Bug 221294 - emulators/virtualbox-ose 5.1.26 r117224 Win7 VM Creation Fails
Summary: emulators/virtualbox-ose 5.1.26 r117224 Win7 VM Creation Fails
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Virtualbox Team (Nobody)
Depends on:
Reported: 2017-08-06 23:33 UTC by papowell
Modified: 2020-10-12 15:31 UTC (History)
7 users (show)

See Also:
bugzilla: maintainer-feedback? (vbox)

TAR of screenshots (373.50 KB, application/x-tar)
2017-08-06 23:33 UTC, papowell
no flags Details
turns off aio completely (1.60 KB, patch)
2020-09-27 10:05 UTC, Gleb Popov
arrowd: maintainer-approval? (vbox)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description papowell 2017-08-06 23:33:01 UTC
Created attachment 185107 [details]
TAR of screenshots

Summary: cannot create a Windows 7 VM with Virtualbox 5.1.26 (FreeBSD 11.[01])
A VM can be created with VirtualBox 4.3.38 (FreeBSD 10.3)

I upgraded to FreeBSD 11.1 and installed VirtualBox 5.1.26 using PKG on 2 Aug 2017. The Windows 7 VMs I had created under FreeBSD 10.3, Virtualbox 4.3.38 OSE r106717 were imported and ran without problems.  However,  when I tried to create a new VM on FreeBSD 11.1/11.0 using VirtualBox 5.1.26, both from the PORTs repository AND when compiled from source,  I got the same set of problems.

When installing Windows 7 you start by creating a virtual machine flagged as Win 7 64amd.  You boot the virtual machine and provide it an ISO image of the Win 7 64 bit Install CD.  The VM then boots from the CD,  goes through a prelimiary setup phase,  and then starts to copy and unpack files from the ISO image.  During this process it displays a progress bar showing the amount of work done.

The unpacking operation appears to fail randomly - 10%, 12%, 20%, 26%, 54%.
If you google the failure message,  then problem (according to MS) is that you have either a bad DVD, or memory and to take corrective actions.

The ISO image checksum matches the published MD5 checksum.  I have tried this on two different systems with the same result.  I am attaching screenshots showing the output of the install and the Vbox.log file.

In the Vbox.log file there appears to be a hint of the cause of the failure:

00:12:32.856035 AioMgr0-N: Request 0x000000a537bd00 failed with rc=VERR_TRY_AGAIN, migrating endpoint /root/VirtualBox VMs/Win7_2017-08-06/Win7_2017-08-06.vdi to failsafe manager.

00:12:34.601955 AIOMgr: I/O manager 0x000000993963a0 encountered a critical error (rc=VERR_FILE_AIO_NO_REQUEST) during operation. Falling back to failsafe mode. Expect reduced performance

00:12:34.602001 AIOMgr: Error happened in /wrkdirs/usr/ports/emulators/virtualbox-ose/work/VirtualBox-5.1.26/src/VBox/VMM/VMMR3/PDMAsyncCompletionFileNormal.cpp:(1673){int pdmacFileAioMgrNormal(RTTHREAD, void *)}

00:12:34.602006 AIOMgr: Please contact the product vendor

Thanks for any help or suggestions on this problem
Comment 1 Jung-uk Kim freebsd_committer 2017-08-08 05:10:44 UTC
It looks like you need some aio(4) tuning.  Please see Bug 212128 for example.
Comment 2 Walter Schwarzenfeld freebsd_triage 2018-01-16 10:39:11 UTC
Is this still relevant? We have version 5.2.4.
Comment 3 papowell 2018-01-18 02:25:27 UTC
After reporting this problem,  I got several suggestions.  Here is the one that appeared to reduce the problem.  The following was added to /etc/sysctl.conf:

# https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212128
# On FreeBSD-11 and on older versions using AIO VirtualBox can lose
# disk access when the disk is heavily loaded.
# The symptoms are either the guest locking up with the guest's system
# "disk" showing continuous access while the host system shows no
# disk activity or by a crash on the guest.
# This can be fixed by adjusting the AIO sysctls to allow added
# capacity before disk requests are lost.
# While I have no fix and this issue has existed for a long time, I
# suggest that pkg-message have an added warning Here is a first-cut
# that could use some word-smithing:
# If AIO is enabled (always in FreeBSD-11.0 and newer), guests may
# lose disk access. This can be mitigated by adjusting AIO sysctls
# as follows:
# Current 11.0 Settings:
# vfs.aio.max_buf_aio: 16
# vfs.aio.max_aio_queue_per_proc: 256
# vfs.aio.max_aio_per_proc: 32
# vfs.aio.aiod_lifetime: 30000
# vfs.aio.num_buf_aio: 0
# vfs.aio.num_queue_count: 0
# vfs.aio.max_aio_queue: 1024
# vfs.aio.target_aio_procs: 4
# vfs.aio.num_aio_procs: 0
# vfs.aio.max_aio_procs: 32
# vfs.aio.unsafe_warningcnt: 1
# vfs.aio.enable_unsafe: 0

# Set:                                  # default
# vfs.aio.aiod_lifetime=30000           # (30000)
# vfs.aio.max_aio_procs=4               # (32)
# vfs.aio.max_aio_queue=65536           # (1024)
# vfs.aio.max_aio_queue_per_proc=65536  # (256)
# vfs.aio.max_aio_per_proc=8192         # (32)
# vfs.aio.max_buf_aio=8192              # (16)
# Alternative:
# # From my sysctl.conf                    Default
# vfs.aio.aiod_lifetime=30000             # (30000)
# vfs.aio.max_aio_procs=4                 # (32)
# vfs.aio.max_aio_queue=8192              # (1024)
# vfs.aio.max_aio_queue_per_proc=1024     # (256)
# vfs.aio.max_aio_per_proc=128            # (32)
# vfs.aio.max_buf_aio=64                  # (16)


Comment 4 Walter Schwarzenfeld freebsd_triage 2018-01-18 02:27:59 UTC
Thanks for reply and detailed answer.
Comment 5 per 2018-01-23 17:29:42 UTC
Same problem (though installing CentOS 7 rather than Windows 7:-), same fix. Installation failed repeatedly with VirtualBox 5.2.4 on FreeBSD 11.1-RELEASE, while it worked in the past with VirtualBox 4.3.34 on FreeBSD 10.2-RELEASE. No problem after applying the aio settings from 212128 - thanks to all for those!

It would of course be great if this info could be included in the pkg-message and/or the https://wiki.freebsd.org/VirtualBox page and/or the Handbook...
Comment 6 elwood 2019-12-13 20:44:22 UTC
(In reply to papowell from comment #3)

Does work for me with freebsd 12.1 and Windows7 guest, thanks
Comment 7 Antoine Delvaux 2020-06-04 19:02:17 UTC
Changing the guest parameters of the Storage Controller to use Host I/O Cache removed this issue for me.  Guests were running official images from Ubuntu with the Host I/O Cache disabled.

Could it be that VirtualBox is not using AIO when in this mode?
Comment 8 Gleb Popov freebsd_committer 2020-09-27 10:05:22 UTC
Created attachment 218352 [details]
turns off aio completely

I'm reuploading my patch from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=168298 there, because I didn't get maintainer's response there and the bug is closed, while this one is still open.

So far I haven't found any problems with my patch and VBox at my $WORK are finally working stable.

I'm requesting an approval for this patch and will commit it if it gets a timeout.
Comment 9 commit-hook freebsd_committer 2020-10-12 15:31:56 UTC
A commit references this bug:

Author: arrowd
Date: Mon Oct 12 15:31:45 UTC 2020
New revision: 552134
URL: https://svnweb.freebsd.org/changeset/ports/552134

  emulators/virtualbox-ose: Turn off aio usage and make VirtualBox use generic Unix implementation.

  This fixes instabilities on some loads involving disk IO.

  PR:		168298, 221294
  Approved by:	vbox (timeout)