Bug 244962 - emulators/virtualbox-ose-additions mount -t vboxsf Operation not supported by device
Summary: emulators/virtualbox-ose-additions mount -t vboxsf Operation not supported by...
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Virtualbox Team (Nobody)
Depends on:
Reported: 2020-03-21 21:30 UTC by mikhail.rokhin
Modified: 2021-06-08 23:51 UTC (History)
6 users (show)

See Also:
bugzilla: maintainer-feedback? (vbox)

stack trace screen snapshot (22.14 KB, image/png)
2020-04-10 14:22 UTC, groenveld
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description mikhail.rokhin 2020-03-21 21:30:00 UTC
When use `mount_vboxvfs ...` it shows:
link_elf_obj: symbol bcmp undefined
linker_load_file: /boot/modules/vboxvfs.ko - unsupported file type

FreeBSD 13-Current
Additions: 5.2.34_2 (installed via pkg, not compiled)
Comment 1 Yuri Pankov 2020-03-22 19:53:34 UTC
Do versions in output of `sysctl kern.osreldate` and `pkg query %Av virtualbox-ose-additions` match?  Try rebuilding from ports if they don't.
Comment 2 mikhail.rokhin 2020-03-23 13:36:17 UTC
(In reply to Yuri Pankov from comment #1)
Identical: 1300084
Comment 3 groenveld 2020-03-27 13:38:00 UTC
The kernel does not offer bcmp:
$ nm /boot/kernel/kernel|grep -i bcmp
ffffffff80cb4610 T timingsafe_bcmp

The port links vboxvfs.ko with a library that references bcmp
$ cat /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.dep
define vboxvfs_CMDS_PREV_

@kmk_builtin_rm -f -- /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.dep /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.ko  /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.map
ld               -d --warn-common   -m elf_x86_64_fbsd   -r -o  /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.ko /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs_vfsops.o /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs_vnops.o /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs_prov.o    /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/lib/additions/VBoxGuestR0Lib.a   /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/lib/additions/RuntimeGuestR0.a
awk -f /usr/src/sys/conf/kmod_syms.awk  /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.ko /dev/null | xargs -J% objcopy % /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/vboxvfs/vboxvfs.ko
$ nm /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/lib/additions/RuntimeGuestR0.a | grep -B13 bcmp
                 U VBoxGuest_RTStrFormat
                 U VBoxGuest_RTStrFormatNumber
0000000000000000 r _ZL13g_szHexDigits
0000000000002060 t _ZL15rtstrFormatIPv6PFmPvPKcmES_PK10RTUINT128U
0000000000000000 r _ZZ13rtstrFormatRtE12s_wszBigNull
0000000000000280 r _ZZ13rtstrFormatRtE8s_aTypes
0000000000000870 r _ZZ13rtstrFormatRtE8s_aTypes_0
0000000000000850 r _ZZ13rtstrFormatRtE8s_szNull
0000000000000857 r _ZZ13rtstrFormatRtE8s_szTrue
000000000000085d r _ZZ13rtstrFormatRtE9s_szFalse
                 U __start_set_pcpu
                 U __stop_set_pcpu
                 U bcmp

Someone with an understanding of FreeBSD kernel modules will have to explain where things went sideways.
Comment 4 Mateusz Guzik freebsd_committer 2020-03-27 21:32:33 UTC
There is no bcmp symbol on amd64 for almost 2 years now. In sys/systm.h there is:

int     bcmp(const void *b1, const void *b2, size_t len);
#define bcmp(b1, b2, len) __builtin_memcmp((b1), (b2), (len))

The first line should probably get removed, but it should not be affecting vbox.

vbox likes to *disable* several warnings. Thus what most likely happens is that some place using bcmp *fails* to include the systm.h header, due to warns disabled nobody is even told and the compiler ends up generating a call to bcmp (instead of memcmp, or no call in the first place).

Thus someone(tm) should make sure the header is included in files which end up doing bcmp. Also will not hurt to make sure the build environment is clean and up to date -- if thep ort pulls in *OLD* files it will only see bcmp declared, but no macro telling it to use __builtin_memcmp and then it will stick to generating the call.
Comment 5 mikhail.rokhin 2020-04-05 16:59:14 UTC
Link to VB bugtracker: https://www.virtualbox.org/ticket/19420
Comment 6 Jung-uk Kim freebsd_committer 2020-04-06 17:48:42 UTC
(In reply to mikhail.rokhin from comment #5)
Note the driver is ours, not theirs.


Please close the upstream PR.
Comment 7 Jung-uk Kim freebsd_committer 2020-04-06 17:53:24 UTC
Mahdi, can you please take a look?
Comment 8 groenveld 2020-04-06 20:15:47 UTC
The invocation of bcmp(3) appears to be hidden in strformatrt.cpp in this switch:
//                                /*
//                                 * Regular hex dump with dittoing.
//                                 */
//                                case 'D':

Comment 9 Mateusz Guzik freebsd_committer 2020-04-06 21:37:01 UTC
Quick grep reveals stuff like:
#else /* else vanilla BSD */
#define memmove(dst,src,len) bcopy((char *)(src),(char *)(dst),(int)(len))
#define memcpy(dst,src,len) bcopy((char *)(src),(char *)(dst),(int)(len))
#define memcmp(b1,b2,len) bcmp((char *)(b1),(char *)(b2),(int)(len))

All defines of the sort should be eliminated from all files -- memmove, memcpy and memcmp are provided as first class citizens both in the kernel and libc. For kernel code, previously mentioned sys/systm.h header needs to be included.
Comment 10 groenveld 2020-04-07 21:12:41 UTC
Any thoughts on why when I compile strformatrt.cpp with -O2 I get the bcmp symbol, but with -O0 I don't?

Would that be a clang10 misfeature?


$ c++ -c -O0 -g -pipe -pedantic -Wshadow -Wall -Wextra -Wno-missing-field-initializers -Wno-unused -Wno-trigraphs -fdiagnostics-show-option -Wno-unused-parameter -Wno-language-extension-token -Wno-gnu-folding-constant -Wno-gnu-anonymous-struct -Wno-string-plus-int -Wno-nested-anon-types -Wno-variadic-macros -Wno-long-long -Wno-return-type-c-linkage -Wno-overloaded-virtual -Wno-variadic-macros -O0 -fno-omit-frame-pointer -fno-strict-aliasing -m64 -I/usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/src/VBox/Runtime/include -I/usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/src/libs/liblzf-3.4 -I/usr/local/include -I/usr/local/include/libxml2 -I/usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/include -I/usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release -DVBOX -DVBOX_OSE -DVBOX_WITH_64_BITS_GUESTS -DVBOX_WITH_REM -DVBOX_WITH_RAW_MODE -DRT_OS_FREEBSD -D__FREEBSD__ -DRT_ARCH_AMD64 -D__AMD64__ -DVBOX_WITH_DEBUGGER -DVBOX_WITH_HARDENING -DRTPATH_APP_PRIVATE=\"/usr/local/lib/virtualbox\" -DRTPATH_APP_PRIVATE_ARCH=\"/usr/local/lib/virtualbox\" -DRTPATH_SHARED_LIBS=\"/usr/local/lib/virtualbox\" -DRTPATH_APP_DOCS=\"/usr/local/lib/virtualbox\" -D_REENTRANT -DIN_RING3 -DLOG_DISABLED -DIN_BLD_PROG -DIN_RT_R3 -DIN_ADV_BLD_PROG -DIN_RT_R3 -DLDR_WITH_NATIVE -DLDR_WITH_ELF32 -DLDR_WITH_LX -DLDR_WITH_MACHO -DLDR_WITH_PE -DRT_WITH_VBOX -DRT_NO_GIP -DRT_WITHOUT_NOCRT_WRAPPERS -DNOFILEID -DRT_WITH_ICONV_CACHE -DIPRT_WITHOUT_LDR_VERIFY -DRT_NO_GIP -Wp,-MD,/usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/RuntimeBldProg/common/string/strformatrt.o.dep -Wp,-MT,/usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/RuntimeBldProg/common/string/strformatrt.o -Wp,-MP -o /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/RuntimeBldProg/common/string/strformatrt.o /usr/ports/emulators/virtualbox-ose-additions/work/VirtualBox-5.2.34/src/VBox/Runtime/common/string/strformatrt.cpp

$ nm ./work/VirtualBox-5.2.34/out/freebsd.amd64/release/obj/RuntimeBldProg/common/string/strformatrt.o | tail -8
0000000000000add r _ZZ13rtstrFormatRtE9s_szFalse
                 U memchr
                 U memcmp
                 U memset
                 U rtStrFormatKernelAddress
0000000000000000 T rtstrFormatRt
                 U strlen
                 U strncmp
Comment 11 Mateusz Guzik freebsd_committer 2020-04-07 21:22:52 UTC
This is probably an optimization which is only there because of something wrong with headers.

There is one fundamental discrepancy between bcmp and memcmp: the former only indicates there is a difference while the latter computes what it is on the first differing byte. Thus, if bcmp is provided and buffers differ, it has less work to do than memcmp and is a better pick if available.

However, FreeBSD headers explicitly define bcmp as builtin_memcmp, therefore the above should not be happening if they are properly included and the defines are not overwritten. Once more this is just some build mess.
Comment 12 groenveld 2020-04-08 12:03:59 UTC
strformatrt.cpp compiles with gcc9 without the bcmp(3) reference but the package fails later due to a clang vs gcc flag conflict that I'll try to track down later.

I would like to try to build the package with clang and -O0.
The second invocation of -O2 is set in Config.kmk which I can patch.
But the first one is a mystery.
I tried setting the port Makefile with CXXFLAGS and CFLAGS but that didn't take.

Comment 13 groenveld 2020-04-10 14:22:10 UTC
Created attachment 213246 [details]
stack trace screen snapshot
Comment 14 groenveld 2020-04-10 20:53:53 UTC
Comment on attachment 213246 [details]
stack trace screen snapshot

VBOXVFS[1]: sfprov_mount: path: [browser]
sfprov_mount(browser): error=0 rc=0
panic: getnewvnode: not registered vector op 0xffffffff82a78418

cpuid = 0
time = 1586551831
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00250ee5c0
vpanic() at vpanic+0x182/frame 0xfffffe00250ee610
panic() at panic+0x43/frame 0xfffffe00250ee670
getnewvnode() at getnewvnode+0x254/frame 0xfffffe00250ee6b0
vboxfs_alloc_vp() at vboxfs_alloc_vp+0x470/frame 0xfffffe00250ee790
vboxfs_root() at vboxfs_root+0x32/frame 0xfffffe00250ee7c0
vfs_domount() at vfs_domount+0x85c/frame 0xfffffe00250ee9f0
vfs_donmount() at vfs_donmount+0x871/frame 0xfffffe00250eea90
sys_nmount() at sys_nmount+0x69/frame 0xfffffe00250eead0
amd64_syscall() at amd64_syscall+0x140/frame 0xfffffe00250eebf0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00250eebf0
--- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x8002f9fea, rsp = 0x7fffffffd3e8, rbp = 0x7fffffffd950 ---
KDB: enter: panic
[ thread pid 862 tid 100464 ]
Stopped at      kdb_enter+0x37: movq    $0,0x10927d6(%rip)
Comment 15 Mateusz Guzik freebsd_committer 2020-04-10 21:12:18 UTC
This one should be easy: it needs VFS_VOP_VECTOR_REGISTER to be added for the argument passed to getnewvnode, grep will show you plenty of examples in the tree.
Comment 16 groenveld 2020-04-10 22:46:16 UTC
Thank you!

+ #if __FreeBSD_version > 1300000
+ VFS_VOP_VECTOR_REGISTER(vboxfs_vnodeops);
+ #endif

Comment 17 groenveld 2020-05-09 22:15:36 UTC
John-Mark Gurney's patch worked for me with FreeBSD 13.0-CURRENT #0 r360723:

Comment 18 groenveld 2020-07-19 17:54:22 UTC
(In reply to groenveld from comment #17)
John-Mark Gurney's patch almost applies to the new virtualbox-ose-5.2.44.
Update patch-src_VBox_Additions_freebsd_vboxvfs_Makefile.kmk

--- src/VBox/Additions/freebsd/vboxvfs/Makefile.kmk.orig        2020-05-13 19:37
:06 UTC
+++ src/VBox/Additions/freebsd/vboxvfs/Makefile.kmk
@@ -33,32 +33,37 @@ vboxvfs_INCS          = \
 vboxvfs_SOURCES       = \
        vboxvfs_vfsops.c \
-       vboxvfs_vnops.c
+       vboxvfs_vnops.c \
+       bcmp.c \
+       vboxvfs_prov.c

Comment 19 groenveld 2021-01-25 20:02:09 UTC
appears to be fixed in ports rev 561713
Comment 20 Graham Perrin 2021-03-14 00:50:28 UTC
mikhail, please, can you confirm that this is no longer an issue for you? 

If it's still an issue: 5.2.44_3 or 6.1.18?
Comment 21 mikhail.rokhin 2021-03-14 01:13:31 UTC
(In reply to Graham Perrin from comment #20)

No, it's the same, except there is no message about bcmp: 
link_elf_obj: symbol bcmp undefined

KLD vboxvfs.ko: depends on kernel - not available or version mismatch
linker_load_file: /boot/modules/vboxvfs.ko - unsupported file type

And earlier during boot:
KLD vboxguest.ko: depends on kernel - not available or version mismatch
linker_load_file: /boot/modules/vboxguest.ko - unsupported file type

virtualbox-ose-additions-6.1.18 (latest from `pkg ins ...`)

Kernel & world revision is latest ( 369447 ; FreeBSD 13.0-CURRENT).
Comment 22 Mateusz Guzik freebsd_committer 2021-06-06 08:14:30 UTC
Is this sorted out now?
Comment 23 mikhail.rokhin 2021-06-06 15:49:09 UTC
(In reply to Mateusz Guzik from comment #22)
No, nothing changed.


Starting vboxservice.
VBoxService: error: VbglR3Init failed with rc=VERR_FILE_NOT_FOUND
/etc/rc: WARNING: failed to start vboxservice
Comment 24 Graham Perrin 2021-06-06 17:29:25 UTC
(In reply to mikhail.rokhin from comment #23)

> VBoxService: error: VbglR3Init failed with rc=VERR_FILE_NOT_FOUND

UFS in the guest?
Comment 25 mikhail.rokhin 2021-06-08 15:09:58 UTC
(In reply to Graham Perrin from comment #24)

No, ZFS and it succeeded 'til some recent changes.
Comment 26 Graham Perrin 2021-06-08 23:51:17 UTC

(In reply to mikhail.rokhin from comment #23)

> … VbglR3Init failed with rc=VERR_FILE_NOT_FOUND

As far as I can tell, probably not related to this bug 244962. 

If you raise the issue elsewhere, please rope me in. Thanks.