Bug 218320

Summary: [exp-run] ino64
Product: Ports & Packages Reporter: Konstantin Belousov <kib>
Component: Individual Port(s)Assignee: Port Management Team <portmgr>
Status: Closed FIXED    
Severity: Affects Only Me CC: emaste, kmoore, kwm, mckusick
Priority: ---    
Version: Latest   
Hardware: Any   
OS: Any   
Bug Depends on: 219667    
Bug Blocks:    
Attachments:
Description Flags
Fixes for llvm39 and llvm40 (ports tree patch)
none
ino64: enlarge ino_t to 64bit and several other ABI forward changes
none
ino64: enlarge ino_t to 64bit and several other ABI forward changes, v2
none
Ports tree patch (llvm, ghc, rust and some minor ports)
none
Ports tree patch (llvm, ghc, rust, webcamd and some minor ports)
none
Update libgtop patch with __FreeBSD_version guards none

Description Konstantin Belousov freebsd_committer freebsd_triage 2017-04-03 10:52:16 UTC
Created attachment 181419 [details]
Fixes for llvm39 and llvm40 (ports tree patch)
Comment 1 Konstantin Belousov freebsd_committer freebsd_triage 2017-04-03 10:53:33 UTC
Created attachment 181420 [details]
ino64: enlarge ino_t to 64bit and several other ABI forward changes
Comment 2 Konstantin Belousov freebsd_committer freebsd_triage 2017-04-03 10:59:50 UTC
I ask for the exp-run of the patch which extends ino_t to 64 bit, and also does related VFS ABI breaking changes.

It is known that llvm* ports are broken due to asan doing dirty things, I attached the ports tree patch which fixes llvm39 and llvm40.  During other attempts to test ports with the patch, it was once reported that devel/libclc is broken, but I was unable to reproduce the issue.

Please see the file testing.txt which appear in the root of the patched src/ tree for details and motivation of the patch.  IMPORTANT: after patching src, you must do
   (cd sys/kern && touch syscalls.master && make sysent)
   (cd sys/compat/freebsd && touch syscalls.master && make sysent)
before rebuilding world and kernel.
Comment 3 Antoine Brodin freebsd_committer freebsd_triage 2017-04-06 05:55:29 UTC
Exp-run results on amd64:

http://pb2.nyi.freebsd.org/build.html?mastername=headamd64PR218320-default&build=2017-04-05_20h35m39s

New failures on amd64:

+ {"origin"=>"devel/libgtop", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/llvm-devel", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/llvm38", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/radare2", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"emulators/qemu-user-static", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"lang/ccl", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"lang/dmd2", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"lang/gcc5-aux", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/gcc6-aux", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/ghc", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"lang/ldc", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"lang/modula3", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/rust", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/rust-nightly", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/twelf", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/urweb", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"multimedia/webcamd", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"net-mgmt/netdata", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"sysutils/py-psutil", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"sysutils/py-psutil121", "phase"=>"build", "errortype"=>"clang"}

Around 650 new ports were skipped due to those failures.

Failure logs:

http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/libgtop-2.32.0.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/llvm-devel-5.0.d20170329_1.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/llvm38-3.8.1_7.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/radare2-1.0.2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/qemu-user-static-2.8.50.g20170307_2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/ccl-1.11.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/dmd2-2.073.2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/gcc5-aux-20160603.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/gcc6-aux-20170202_1.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/ghc-7.10.2_2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/ldc-1.1.0.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/modula3-5.10.0_4.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/rust-1.16.0.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/rust-nightly-1.17.0.20170212.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/twelf-1.7.1.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/urweb-20150520.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/webcamd-4.8.0.4.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/netdata-1.6.0.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/py27-psutil-5.2.1.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-04-05_20h35m39s/logs/errors/py27-psutil121-1.2.1_2.log
Comment 4 Ed Maste freebsd_committer freebsd_triage 2017-04-06 13:34:32 UTC
Some of the errors appear to be Clang 4.0.0 issues or otherwise unrelated to ino64 -- e.g. from databases/mongodb:

| In file included from src/mongo/db/db.cpp:47:
| src/mongo/db/client.h:263:59: error: ordered comparison between pointer and zero ('mongo::Client *' and 'int')
|     inline bool haveClient() { return currentClient.get() > 0; }
|                                       ~~~~~~~~~~~~~~~~~~~ ^ ~
Comment 5 Antoine Brodin freebsd_committer freebsd_triage 2017-04-09 08:23:02 UTC
Exp-run results on i386:

http://pb2.nyi.freebsd.org/build.html?mastername=headi386PR218320-default&build=2017-04-08_06h24m47s

Baseline build is still ongoing (at http://beefy11.nyi.freebsd.org/build.html?mastername=head-i386-default&build=p437978_s316629 )so I will post new failures when i'm back in a week.
Comment 6 Antoine Brodin freebsd_committer freebsd_triage 2017-04-16 12:56:21 UTC
New failures on i386:

+ {"origin"=>"devel/libgtop", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/llvm-devel", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/llvm38", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/radare2", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"emulators/open-vm-tools", "phase"=>"build", "errortype"=>"clang_werror"}
+ {"origin"=>"emulators/open-vm-tools-nox11", "phase"=>"build", "errortype"=>"clang_werror"}
+ {"origin"=>"games/kiki", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"lang/ccl", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"lang/dmd2", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"lang/gcc5-aux", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/gcc6-aux", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/ghc", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"lang/ldc", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"lang/rust", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/rust-nightly", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/twelf", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/urweb", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"multimedia/webcamd", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"net-mgmt/netdata", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"sysutils/py-psutil", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"sysutils/py-psutil121", "phase"=>"build", "errortype"=>"clang"}

Failure logs on i386:

http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/libgtop-2.32.0.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/llvm-devel-5.0.d20170329_1.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/llvm38-3.8.1_7.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/radare2-1.0.2.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/open-vm-tools-10.1.0_1,2.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/open-vm-tools-nox11-10.1.0_1,2.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/kiki-1.0.2_18.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/ccl-1.11.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/dmd2-2.073.2.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/gcc5-aux-20160603.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/gcc6-aux-20170202_1.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/ghc-7.10.2_2.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/ldc-1.1.0.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/rust-1.16.0.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/rust-nightly-1.17.0.20170212.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/twelf-1.7.1.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/urweb-20150520.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/webcamd-4.8.0.4.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/netdata-1.6.0.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/py27-psutil-5.2.1.log
http://pb2.nyi.freebsd.org/data/headi386PR218320-default/2017-04-08_06h24m47s/logs/errors/py27-psutil121-1.2.1_2.log

Around 670 ports were skipped due to those new failures.
Comment 7 Konstantin Belousov freebsd_committer freebsd_triage 2017-05-06 12:02:13 UTC
Created attachment 182332 [details]
ino64: enlarge ino_t to 64bit and several other ABI forward changes, v2
Comment 8 Konstantin Belousov freebsd_committer freebsd_triage 2017-05-06 12:03:04 UTC
Created attachment 182333 [details]
Ports tree patch (llvm, ghc, rust and some minor ports)
Comment 9 Konstantin Belousov freebsd_committer freebsd_triage 2017-05-06 12:04:00 UTC
Updated patch set.  Hopefully, the big ports blocking many others are fixed.

Still not sure what to do with small (leaf) ports.
Comment 10 Antoine Brodin freebsd_committer freebsd_triage 2017-05-07 07:04:00 UTC
lang/ghc fails:  build depends are not installed yet during patch phase so ${CC} is not available

http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-06_15h54m27s/logs/errors/ghc-7.10.2_2.log

Maybe the "${CC} ${CFLAGS} .." part has to be moved to pre-configure target?
Comment 11 Antoine Brodin freebsd_committer freebsd_triage 2017-05-07 07:11:16 UTC
Is there a patch for multimedia/webcamd ?  61 ports (including gnome3 and pidgin) are skipped because of it right now.
Comment 12 Konstantin Belousov freebsd_committer freebsd_triage 2017-05-07 09:26:03 UTC
Created attachment 182355 [details]
Ports tree patch (llvm, ghc, rust, webcamd and some minor ports)

Reorganized ghc patch to compile the hack at pre-configure stage.  Added patch for webcamd.
Comment 13 Antoine Brodin freebsd_committer freebsd_triage 2017-05-11 16:54:02 UTC
Exp-run results on i386:

http://pb2.nyi.freebsd.org/build.html?mastername=headamd64PR218320-default&build=2017-05-11_05h06m10s

New failures on i386:

+ {"origin"=>"devel/llvm-devel", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/llvm38", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"devel/radare2", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"emulators/qemu-user-static", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"lang/ccl", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"lang/dmd2", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"lang/gcc5-aux", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/gcc6-aux", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/ldc", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"lang/modula3", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/rust-nightly", "phase"=>"extract", "errortype"=>"???"}
+ {"origin"=>"lang/twelf", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"lang/urweb", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"sysutils/py-psutil121", "phase"=>"build", "errortype"=>"clang"}

Failure logs on i386:

http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/llvm-devel-5.0.d20170329_2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/llvm38-3.8.1_8.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/radare2-1.0.2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/qemu-user-static-2.8.50.g20170307_2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/ccl-1.11.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/dmd2-2.073.2.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/gcc5-aux-20160603.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/gcc6-aux-20170202_1.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/ldc-1.1.0.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/modula3-5.10.0_4.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/rust-nightly-1.17.0.20170212.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/twelf-1.7.1.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/urweb-20150520.log
http://pb2.nyi.freebsd.org/data/headamd64PR218320-default/2017-05-11_05h06m10s/logs/errors/py27-psutil121-1.2.1_2.log
Comment 14 Antoine Brodin freebsd_committer freebsd_triage 2017-05-11 16:56:10 UTC
The results above were on amd64
Comment 15 Konstantin Belousov freebsd_committer freebsd_triage 2017-05-11 17:26:35 UTC
(In reply to Antoine Brodin from comment #13)
So this looks quite good, in the sense that all ports and dependencies which were supposed to be fixed, are fixed.  Am I right ?

What is the portmgr group opinion on the current state ?  Do you require that I fixed all remaining ports, or do you agree that low-profile ports fixes can be delegated to the ports maintainers ?

Thank you for all the efforts.
Comment 16 Koop Mast freebsd_committer freebsd_triage 2017-05-16 15:32:10 UTC
Created attachment 182641 [details]
Update libgtop patch with __FreeBSD_version guards

The libgtop patch from the big ports patch with __FreeBSD_version code.

Only thing needed is the correct value to be filled in when the ino64 work is committed.
Comment 17 Ed Maste freebsd_committer freebsd_triage 2017-05-25 02:04:53 UTC
For reference, ino64 is __FreeBSD_version 1200031
Comment 18 Ed Maste freebsd_committer freebsd_triage 2018-02-01 15:01:30 UTC
Is there anything further that needs to be done here?
Comment 19 Antoine Brodin freebsd_committer freebsd_triage 2018-02-01 15:07:15 UTC
I believe gcc6-aux is still broken on i386?
Comment 20 Baptiste Daroussin freebsd_committer freebsd_triage 2019-09-05 09:34:15 UTC
The code has landed, long ago, if there still are leftovers, a new PR should be open dedicated to it