Bug 238511 - [regression] after D18038 usr.bin/strings
Summary: [regression] after D18038 usr.bin/strings
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Mark Johnston
Keywords: regression
Depends on:
Reported: 2019-06-11 23:28 UTC by Andrey Fesenko
Modified: 2020-06-08 22:44 UTC (History)
3 users (show)

See Also:

diff build logs (3.50 KB, text/plain)
2019-06-11 23:28 UTC, Andrey Fesenko
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Fesenko 2019-06-11 23:28:09 UTC
Created attachment 204999 [details]
diff build logs


I'm see broken build emulators/i386-wine-devel if revert usr.bin/strings on 325188 build fixed.
Comment 1 Mariusz Zaborski freebsd_committer 2019-06-12 10:35:44 UTC
Could you generate a core dump of string and paste it here?

To do that please set those sysctls:

Thank you,
Comment 2 Andrey Fesenko 2019-06-12 12:43:05 UTC
hm... this crash not generate core :(
test perl -e 'dump' make core success.

if you have time this instruction for build https://wiki.freebsd.org/i386-Wine
Comment 3 Mark Johnston freebsd_committer 2019-07-09 03:26:47 UTC
I managed to reproduce this.  It looks like some strange issue in libnv or the kernel, libnv is seeing recv() return 0 upon attempting to read a new message.  I suspect that there is a problem in our 32-bit compat code for sockets.
Comment 4 Mariusz Zaborski freebsd_committer 2019-07-09 14:27:42 UTC
If the issue is in the libnv and Casper I can look into this.
Could you tell me how to reproduce the problem or can you add the core dump?
Comment 5 Mark Johnston freebsd_committer 2019-07-09 14:32:24 UTC
(In reply to Mariusz Zaborski from comment #4)
I'm pretty sure it's a bug somewhere in our freebsd32 sendmsg, I'm still digging into it.

To repro, run the following in an i386 chroot on amd64:

$ jot -b /bin/cat 85 >/tmp/files
$ strings $(cat /tmp/files) >/dev/null
strings: '/bin/cat': Invalid argument
Comment 6 Mark Johnston freebsd_committer 2019-07-11 20:17:43 UTC
So, the issue is that we have an undocumented limit on the number of descriptors that we can send in a single sendmsg() call, and it's the number that fits in an mbuf cluster (MCLBYTES = 2KB).  And, libnv has this:

 62 #ifdef __linux__                                                                                                                                                          
 63 /* Linux: arbitrary size, but must be lower than SCM_MAX_FD. */                                                                                                           
 64 #define PKG_MAX_SIZE    ((64U - 1) * CMSG_SPACE(sizeof(int)))                                                                                                             
 65 #else                                                                                                                                                                     
 66 #define PKG_MAX_SIZE    (MCLBYTES / CMSG_SPACE(sizeof(int)) - 1)                                                                                                          
 67 #endif

The problem is that libnv passes each fd in a separate cmsg (no idea why), and these cmsgs have a different size on 32-bit vs. 64-bit FreeBSD due to alignment.  Our freebsd32 sendmsg translates the messages to their native size, so it can only accept as many as our 64-bit sendmsg.  But a 32-bit application running on a 64-bit kernel doesn't know that.

We could largely mitigate the problem here by passing all of the fds in a single cmsg.  That would also be more efficient.  But this is really a general problem in our freebsd32 compat, which must be willing to construct an mbuf chain to hold the control message instead of assuming that it all fits in one cluster.
Comment 7 Mark Johnston freebsd_committer 2019-07-12 21:43:52 UTC
Grab this bug since I'm working on it.

Comment 8 commit-hook freebsd_committer 2019-07-16 16:29:26 UTC
A commit references this bug:

Author: markj
Date: Tue Jul 16 16:28:50 UTC 2019
New revision: 350054
URL: https://svnweb.freebsd.org/changeset/base/350054

  Use a platform-independent constant for PKG_MAX_SIZE.

  This constant determines the number of rights libnv will attempt to
  transmit in a given control message.  In practice, the upper limit
  defined by the kernel is machine-dependent and is smaller on 64-bit
  kernels than on 32-bit kernels.  To ensure that a 32-bit libnv works
  as expected when run on a 64-bit kernel, use a limit that will work
  on both 32-bit and 64-bit kernels.

  PR:		238511
  Discussed with:	oshogbo
  MFC after:	3 days
  Sponsored by:	The FreeBSD Foundation
  Differential Revision:	https://reviews.freebsd.org/D20942

Comment 9 commit-hook freebsd_committer 2019-07-19 15:08:24 UTC
A commit references this bug:

Author: markj
Date: Fri Jul 19 15:07:31 UTC 2019
New revision: 350130
URL: https://svnweb.freebsd.org/changeset/base/350130

  MFC r350054:
  Use a platform-independent constant for PKG_MAX_SIZE.

  PR:	238511

_U  stable/12/
Comment 10 Mark Johnston freebsd_committer 2019-07-19 15:12:52 UTC
Thanks for the report.
Comment 11 Andrey Fesenko 2019-10-09 01:12:38 UTC
build of emulators/i386-wine-devel | i386-wine-devel-4.17,1 ended at Tue Oct  8 23:16:21 MSK 2019

>/usr/bin/grep -v '[@%]' /wrkdirs/usr/ports/emulators/i386-wine-devel/work/.PLIST.mktmp | /usr/bin/sed "s!^!/wrkdirs/usr/ports/emulators/i386-wine-devel/work/stage/usr/local/!g" |  /usr/bin/xargs -n1 file -F' ' | /usr/bin/grep ELF | /usr/bin/cut -f1 -d' ' | /usr/bin/xargs strings |  /usr/bin/grep '^lib.*\.so' | /usr/bin/sort -u > /wrkdirs/usr/ports/emulators/i386-wine-devel/work/winesoftlibs
>strings: '/wrkdirs/usr/ports/emulators/i386-wine-devel/work/stage/usr/local/lib32/wine/tapi32.dll.so': Socket is not connected
>xargs: strings: terminated with signal 13; aborting
Comment 12 Mark Johnston freebsd_committer 2019-10-09 01:14:45 UTC
(In reply to Andrey Fesenko from comment #11)
Please provide the host and jail revisions.
Comment 13 Andrey Fesenko 2019-10-09 01:20:50 UTC
Host OSVERSION: 1300048
Jail OSVERSION: 1300048

full log https://suahbsd.org/poudriere-logs/data/13i386-default-wine/2019-10-08_15h55m10s/logs/i386-wine-devel-4.17,1.log
Comment 14 Mark Johnston freebsd_committer 2020-06-08 22:44:55 UTC
(In reply to Andrey Fesenko from comment #13)
My apologies, I never followed up on this. :(

Is it still occurring?  I fixed another related bug recently, 241226, which may be the cause.