Bug 214944

Summary: emulators/qemu-user-static: poudriere bulk with aarch64 creates memory leak
Product: Ports & Packages Reporter: dornergd <gerhard.dorner>
Component: Individual Port(s)Assignee: Sean Bruno <sbruno>
Status: Closed FIXED    
Severity: Affects Only Me CC: audioone.official, kevans, mikael, pi, punasipuli, vivek
Priority: --- Flags: bugzilla: maintainer-feedback? (sbruno)
Version: Latest   
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
c file that will cause the qemu-bsd-user port to fail none

Description dornergd 2016-11-30 08:43:51 UTC 
When running poudriere to create new packages for aarch64 the build process creates a memory leak.

After a short time of building the package "pkg" the whole memory are used.

After downgrade qemu-sbruno to revision 417742 this memory leak doesn't
occur.
 
I haven't tested other revisions to find out the incorrect version.
Comment 1 Sean Bruno freebsd_committer freebsd_triage 2016-12-01 15:46:17 UTC 
Yeah, we noticed this in the freebsd build cluster.  :-(

I'm still trying to find the leak (its not aarch64 specific AFAIK).
Comment 2 Andrew Turner freebsd_committer freebsd_triage 2016-12-05 15:13:46 UTC 
*** Bug 215071 has been marked as a duplicate of this bug. ***
Comment 3 punasipuli 2016-12-05 16:46:04 UTC 
Hi!

Its not specific to an arch. I've had this issue with poudriere packages for armv6. If I create the jail with the nx-tools, than it works for the most packages, but there is still a memory leak with gcc. Also there are some qemu-arm issues with e.g. conftest.
Comment 4 Vick Khera 2016-12-15 14:27:50 UTC 
Doing a poudriere cross-compile on FreeBSD11/amd64 with a Xeon processor to armv6 will never complete a build of m4 (as a dependency trying to build subversion). It stalls in the configuration step:

checking for dirent.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for working C stack overflow detection... make: Working in: /usr/ports/devel/m4
make: Working in: /usr/ports/devel/m4
make: Working in: /usr/ports/devel/m4
make: Working in: /usr/ports/devel/m4
make: Working in: /usr/ports/devel/m4
make: Working in: /usr/ports/devel/m4

Even after 18 hours it just keeps printing that line every so often.

Downgrading qemu-user-static from 2.7.90.g20161116_1 to qemu-user-static-2.6.90.g20160728 allows it to build pretty much immediately (under 2 minutes). I don't know if any newer version of qemu-user-static will work. I just found the package from the quarterly package set. The last time I successfully built the packages was early to mid October.

The jail is configured to use the native cross-building tools (the "-x" option to poudriere jail).
Comment 5 Sean Bruno freebsd_committer freebsd_triage 2016-12-19 00:33:57 UTC 
Created attachment 178081 [details]
c file that will cause the qemu-bsd-user port to fail

Been trying to resolve the threading model/signal handling upstream with our out of tree bsd-user code over the last few weeks.  I'm kind of stuck.  The devel/m4 configure script will try to invoke the code attached and doesn't deal with it very well.
Comment 6 punasipuli 2016-12-19 13:54:51 UTC 
I can confirm the above behaviour.

Poudriere cross-compile on FreeBSD11/amd64 with an Intel processor to armv6 will also never complete a build of ruby22. It stalls in the configuration step:

checking for broken backtrace... make: Working in: /usr/ports/lang/ruby22

lang/gcc fails to build, too:

"(process:87381): GLib-ERROR (recursed) **: gmem.c:166: failed to
allocate 32 bytes{standard input}: Assembler messages:"

The jail is configured to use the native cross-building tools.
I didn`t try to downgrade.
Comment 7 commit-hook freebsd_committer freebsd_triage 2016-12-26 15:08:59 UTC 
A commit references this bug:

Author: sbruno
Date: Mon Dec 26 15:08:18 UTC 2016
New revision: 429531
URL: https://svnweb.freebsd.org/changeset/ports/429531

Log:
  Update to qemu 2.8.50:
  - we should be able to build devel/m4 again, even though the configure
    script will segfault instead of hanging forever.  Progress!

  PR:		214944

Changes:
  head/emulators/qemu-sbruno/Makefile
  head/emulators/qemu-sbruno/distinfo
Comment 8 Sean Bruno freebsd_committer freebsd_triage 2016-12-27 02:03:20 UTC 
It appears now that the host_signal_handler() isn't being registered or invoked in bsd-user.  That's curious.
Comment 9 punasipuli 2016-12-27 18:02:09 UTC 
Now, with qemu 2.8.50 I have had that message in Poudriere:

"echo: write error on stdout"

Also some other packages don't compile successfully without MAKE_JOBS_UNSAFE= yes. E.g. devel/glib20

First time, I was trying the this version, after upgrade the whole VM was hanging. Don't know, if it depends on it?
Comment 10 commit-hook freebsd_committer freebsd_triage 2016-12-29 01:13:14 UTC 
A commit references this bug:

Author: sbruno
Date: Thu Dec 29 01:12:38 UTC 2016
New revision: 429848
URL: https://svnweb.freebsd.org/changeset/ports/429848

Log:
  Bump qemu-sbruno to capture today's removal of signal blocking in
  the bsd-user code. This update should allow normal operation with the
  bugs that we all were familiar with!

  PR:	214944 215552

Changes:
  head/emulators/qemu-sbruno/Makefile
  head/emulators/qemu-sbruno/distinfo
Comment 11 Mikael Urankar freebsd_committer freebsd_triage 2016-12-29 08:35:43 UTC 
Thanks Sean!
Comment 12 punasipuli 2016-12-29 15:08:15 UTC 
Now, it seems to work. So thanks again