Bug 276961 - buildworld artifacts not reproducible between native and cross build
Summary: buildworld artifacts not reproducible between native and cross build
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 13.3-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-11 00:21 UTC by Ed Maste
Modified: 2024-07-31 07:38 UTC (History)
1 user (show)

See Also:


Attachments
list of differing artifacts (6.03 KB, text/plain)
2024-02-11 00:21 UTC, Ed Maste
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer freebsd_triage 2024-02-11 00:21:14 UTC
Created attachment 248338 [details]
list of differing artifacts

On stable/13 cperciva discovered a number of artifacts produced by buildworld that differ between native (e.g. i386 buildworld in i386 jail) and cross (e.g. i386 buildworld on amd64 with TARGET/TARGET_ARCH set) builds.

PR276960 describes one example. Other cases are different stack layouts or instruction ordering, pointing to nondeterminism in Clang. This example is a disassembly of mkfs.o, responsible for the difference in makefs:

- objdump --line-numbers --disassemble --demangle --reloc --no-show-raw-insn --section=.text {}
@@ -24590,15 +24590,15 @@
              movl    $0x0, 0x42ca00    
              movl    $0x0, 0x42cccc    
              movl    $0x1, 0x42cec0    
              movl    $0x1, 0x42cec4    
              xorl    %eax, %eax
              movl    $0x2000, %ecx           # imm = 0x2000
              jmp     0x41e7fa <ffs_mkfs+0x79a>
+             movl    %ebx, 0x2c(%esp)  
-             movl    %ebx, 0x30(%esp)  
              movl    $0x19540119, 0x42ced0   # imm = 0x19540119
              movl    $0x0, 0x42cd60    
              movl    $0x10000, 0x42cd5c      # imm = 0x10000   
              movl    0x4(%esp), %eax   
              movl    %eax, %ecx
              shrl    $0x3, %ecx
              movl    %ecx, %edi
@@ -24611,15 +24611,15 @@
              movl    $0x10000, %ecx          # imm = 0x10000
              movl    0x8(%esp), %edx   
              cmpl    $0x1, 0x70(%edx)  
              movl    %esi, 0x20(%esp)  
              movl    %edi, (%esp)
              jne     0x41e7f6 <ffs_mkfs+0x796>
              orb     $0x2, 0x42ce94    
+             movl    0x2c(%esp), %ebx  
-             movl    0x30(%esp), %ebx  
              movl    0x14(%esp), %edx  
              addl    %ecx, %edx
              adcl    %eax, %ebx
              addl    $0x2000, %edx           # imm = 0x2000
              adcl    $0x0, %ebx
              movl    0xc(%esp), %esi   
              movl    %esi, %eax
@@ -24749,57 +24749,57 @@
              leal    (%ebx,%ecx), %edx 
              leal    (%eax,%edx), %ecx 
              decl    %ecx
              movl    %ecx, %eax
              xorl    %edx, %edx
              divl    %ebx
              subl    %edx, %ecx
+             movl    %ecx, 0x2c(%esp)  
-             movl    %ecx, 0x3c(%esp)  
              movl    %ebx, 0x28(%esp)  
              movl    0xc(%esp), %eax   
              addl    %ebx, %eax
              decl    %eax
              movl    %eax, 0x38(%esp)  
              movl    0x42ca28, %eax    
              addl    %eax, %eax
              leal    (%eax,%eax,2), %eax
              movl    %eax, 0x34(%esp)  
              movl    0x42c9a4, %eax    
              addl    $-0x8, %eax
+             movl    %eax, 0x48(%esp)  
+             movl    0x42c9a8, %eax    
              movl    %eax, 0x40(%esp)  
-             movl    0x42c9a8, %eax    
-             movl    %eax, 0x44(%esp)  
              negl    %eax
+             movl    %eax, 0x30(%esp)  
-             movl    %eax, 0x2c(%esp)  
              movl    %edi, (%esp)
              leal    -0x1(%edi), %eax  
              movl    %eax, 0x4(%esp)   
              movl    0x42ce98, %eax    
+             movl    %eax, 0x3c(%esp)  
-             movl    %eax, 0x48(%esp)  
              leal    (,%eax,4), %eax   
              movl    %eax, 0x50(%esp)  
              movl    %esi, %edx
              movl    0x14(%esp), %ecx
Comment 1 Ed Maste freebsd_committer freebsd_triage 2024-02-11 00:35:53 UTC
Looking at the difference in boot1.efi there's an extra instruction in the cross-built case

--- i386-on-amd64/boot/boot1.efi
+++ i386-on-i386/boot/boot1.efi
├── objdump
│ @@ -55,15 +55,15 @@
│  Entry d 0000000000000000 00000000 Delay Import Directory
│  Entry e 0000000000000000 00000000 CLR Runtime Header
│  Entry f 0000000000000000 00000000 Reserved
│
│
│  Sections:
│  Idx Name          Size     VMA      Type
│ -  0 .text         000121ba 00001000 TEXT
│ +  0 .text         000121b6 00001000 TEXT
│    1 .data         0000a538 00014000 DATA
│    2 .sdata        00000024 0001f000 DATA
...
│ -    221c:            movl    %edi, 0x28(%esp)
│ -    2220:            movl    %eax, (%esp)
│ -    2223:            movl    %ecx, 0x4(%esp)
│ -    2227:            je      0x2252 <.text+0x1252>
│ -    2229:            movl    -0x1ec4(%ebx), %ecx
...
│ +    221c:            movl    %eax, (%esp)
│ +    221f:            movl    %ecx, 0x4(%esp)
│ +    2223:            je      0x224e <.text+0x124e>
│ +    2225:            movl    -0x1ec4(%ebx), %ecx
Comment 2 Dimitry Andric freebsd_committer freebsd_triage 2024-04-11 16:50:34 UTC
An initial analysis seems to indicate that with clang 18, I can get bit-identical output for e.g. mkfs.o and a few other select .o files that I found, when comparing between "cross" and "native" compiler output.

With clang 17 I see the same type of difference, making it likely that some stack alignment setting leaks in from the host architecture. It is going to be very time consuming to figure out the exact upstream commit(s) that fixed this.

For now I am doing 13.3-RELEASE-p1 buildworlds with clang 18.1.3 (as it is in main right now) as host compiler, both in "cross" and "native" mode. I will report back when I have some results there.
Comment 3 Dimitry Andric freebsd_committer freebsd_triage 2024-05-05 19:56:45 UTC
I did the analysis again on 15-CURRENT with clang 18.1.5, and this is now the list, when excluding host binaries (i.e. those that file(1) shows as ELF 64-bit binaries on a amd64 host):

usr.cross/src/i386.i386/cddl/lib/libzfs/libzfs.a
usr.cross/src/i386.i386/cddl/lib/libzfs/libzfs.so
usr.cross/src/i386.i386/cddl/lib/libzfs/libzfs.so.4
usr.cross/src/i386.i386/cddl/lib/libzfs/libzfs.so.4.debug
usr.cross/src/i386.i386/cddl/lib/libzfs/libzfs.so.4.full
usr.cross/src/i386.i386/cddl/lib/libzfs/zfs_fletcher_sse.o
usr.cross/src/i386.i386/cddl/lib/libzfs/zfs_fletcher_sse.pico
usr.cross/src/i386.i386/cddl/lib/libzpool/libzpool.a
usr.cross/src/i386.i386/cddl/lib/libzpool/libzpool.so
usr.cross/src/i386.i386/cddl/lib/libzpool/libzpool.so.2
usr.cross/src/i386.i386/cddl/lib/libzpool/libzpool.so.2.debug
usr.cross/src/i386.i386/cddl/lib/libzpool/libzpool.so.2.full
usr.cross/src/i386.i386/cddl/lib/libzpool/zfs_fletcher_sse.o
usr.cross/src/i386.i386/cddl/lib/libzpool/zfs_fletcher_sse.pico
usr.cross/src/i386.i386/lib/clang/liblldb/Plugins/Process/FreeBSD/NativeRegisterContextFreeBSD_x86_64.o
usr.cross/src/i386.i386/lib/clang/liblldb/liblldb.a
usr.cross/src/i386.i386/lib/libarchive/archive_write_set_format_7zip.pico
usr.cross/src/i386.i386/lib/libarchive/libarchive.so
usr.cross/src/i386.i386/lib/libarchive/libarchive.so.7
usr.cross/src/i386.i386/lib/libarchive/libarchive.so.7.debug
usr.cross/src/i386.i386/lib/libarchive/libarchive.so.7.full
usr.cross/src/i386.i386/rescue/rescue/bectl.lo
usr.cross/src/i386.i386/rescue/rescue/rescue
usr.cross/src/i386.i386/rescue/rescue/zdb.lo
usr.cross/src/i386.i386/rescue/rescue/zfs.lo
usr.cross/src/i386.i386/rescue/rescue/zfsbootcfg.lo
usr.cross/src/i386.i386/rescue/rescue/zpool.lo
usr.cross/src/i386.i386/sbin/ipf/ipmon/ipmon
usr.cross/src/i386.i386/sbin/ipf/ipmon/ipmon.debug
usr.cross/src/i386.i386/sbin/ipf/ipmon/ipmon.full
usr.cross/src/i386.i386/sbin/ipf/ipmon/ipmon.o
usr.cross/src/i386.i386/stand/ficl/softcore.c
usr.cross/src/i386.i386/stand/i386/loader_4th/loader_4th
usr.cross/src/i386.i386/stand/i386/loader_4th/loader_4th.bin
usr.cross/src/i386.i386/stand/i386/loader_4th/loader_4th.sym
usr.cross/src/i386.i386/stand/i386/loader_4th/vers.c
usr.cross/src/i386.i386/stand/i386/loader_4th/vers.o
usr.cross/src/i386.i386/stand/i386/loader_lua/loader_lua
usr.cross/src/i386.i386/stand/i386/loader_lua/loader_lua.bin
usr.cross/src/i386.i386/stand/i386/loader_lua/loader_lua.sym
usr.cross/src/i386.i386/stand/i386/loader_lua/vers.c
usr.cross/src/i386.i386/stand/i386/loader_lua/vers.o
usr.cross/src/i386.i386/stand/i386/loader_simp/loader_simp
usr.cross/src/i386.i386/stand/i386/loader_simp/loader_simp.bin
usr.cross/src/i386.i386/stand/i386/loader_simp/loader_simp.sym
usr.cross/src/i386.i386/stand/i386/loader_simp/vers.c
usr.cross/src/i386.i386/stand/i386/loader_simp/vers.o
usr.cross/src/i386.i386/stand/i386/pxeldr/loader
usr.cross/src/i386.i386/stand/i386/pxeldr/pxeboot
usr.cross/src/i386.i386/toolchain-metadata.mk
usr.cross/src/i386.i386/usr.bin/clang/lld/ELF/SyntheticSections.o
usr.cross/src/i386.i386/usr.bin/clang/lld/ld.lld
usr.cross/src/i386.i386/usr.bin/clang/lld/ld.lld.debug
usr.cross/src/i386.i386/usr.bin/clang/lld/ld.lld.full
usr.cross/src/i386.i386/usr.bin/clang/lldb-server/lldb-server
usr.cross/src/i386.i386/usr.bin/clang/lldb-server/lldb-server.debug
usr.cross/src/i386.i386/usr.bin/clang/lldb-server/lldb-server.full

Note that the differences in .a or .so files, or executables, is due to one or more object files differing.

Also, the stand/i386/loader files differ because vers.c contains a build date and hostname, but I assume that would go away if you use WITH_REPRODUCIBLE_BUILD?

So the list of interesting files can be reduced to (grouped by library/program):

usr.cross/src/i386.i386/cddl/lib/libzfs/zfs_fletcher_sse.o
usr.cross/src/i386.i386/cddl/lib/libzfs/zfs_fletcher_sse.pico

usr.cross/src/i386.i386/cddl/lib/libzpool/zfs_fletcher_sse.o
usr.cross/src/i386.i386/cddl/lib/libzpool/zfs_fletcher_sse.pico

usr.cross/src/i386.i386/lib/clang/liblldb/Plugins/Process/FreeBSD/NativeRegisterContextFreeBSD_x86_64.o

usr.cross/src/i386.i386/lib/libarchive/archive_write_set_format_7zip.pico

usr.cross/src/i386.i386/rescue/rescue/bectl.lo
usr.cross/src/i386.i386/rescue/rescue/zdb.lo
usr.cross/src/i386.i386/rescue/rescue/zfs.lo
usr.cross/src/i386.i386/rescue/rescue/zfsbootcfg.lo
usr.cross/src/i386.i386/rescue/rescue/zpool.lo

usr.cross/src/i386.i386/sbin/ipf/ipmon/ipmon.o

usr.cross/src/i386.i386/usr.bin/clang/lld/ELF/SyntheticSections.o

Of these, most don't have differences in their assembly, but readelf shows that due to slightly larger or smaller sections, various offsets are different.

The left overs are:

usr.cross/src/i386.i386/lib/clang/liblldb/Plugins/Process/FreeBSD/NativeRegisterContextFreeBSD_x86_64.o
usr.cross/src/i386.i386/lib/libarchive/archive_write_set_format_7zip.pico
usr.cross/src/i386.i386/usr.bin/clang/lld/ELF/SyntheticSections.o

I'm going to spend some time looking at the ways these get compiled.
Comment 4 commit-hook freebsd_committer freebsd_triage 2024-07-21 20:42:27 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=397c2693fa66508cb5e6b173650a1f3bc6c4dd4f

commit 397c2693fa66508cb5e6b173650a1f3bc6c4dd4f
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-21 20:37:27 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-21 20:37:27 +0000

    Fix llvm register allocator for native/cross build differences

    Work around an issue in LLVM's register allocator, which can cause
    slightly different i386 object files, when produced by a native or cross
    build of clang.

    This adds another volatile qualifier to a float variable declaration in
    the weightCalcHelper() function, which otherwise produces slightly
    different float results on amd64 and i386 hosts. In turn, this can lead
    to different (but equivalent) register choices, and thus non-identical
    assembly code.

    See https://github.com/llvm/llvm-project/issues/99396 for more details.

    Note this is a temporary fix, meant to merge in time for 13.4. As soon
    as upstream has a permanent solution we will import that.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Comment 5 commit-hook freebsd_committer freebsd_triage 2024-07-24 19:44:43 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=2c75d993783ca4b0d1bf8dcdf424643781326e4b

commit 2c75d993783ca4b0d1bf8dcdf424643781326e4b
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-21 20:37:27 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-24 19:25:53 +0000

    Fix llvm register allocator for native/cross build differences

    Work around an issue in LLVM's register allocator, which can cause
    slightly different i386 object files, when produced by a native or cross
    build of clang.

    This adds another volatile qualifier to a float variable declaration in
    the weightCalcHelper() function, which otherwise produces slightly
    different float results on amd64 and i386 hosts. In turn, this can lead
    to different (but equivalent) register choices, and thus non-identical
    assembly code.

    See https://github.com/llvm/llvm-project/issues/99396 for more details.

    Note this is a temporary fix, meant to merge in time for 13.4. As soon
    as upstream has a permanent solution we will import that.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 397c2693fa66508cb5e6b173650a1f3bc6c4dd4f)

 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Comment 6 commit-hook freebsd_committer freebsd_triage 2024-07-24 19:44:44 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8d87e47b8d1093a264ca954620b9e58b81fb9b34

commit 8d87e47b8d1093a264ca954620b9e58b81fb9b34
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-21 20:37:27 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-24 19:26:11 +0000

    Fix llvm register allocator for native/cross build differences

    Work around an issue in LLVM's register allocator, which can cause
    slightly different i386 object files, when produced by a native or cross
    build of clang.

    This adds another volatile qualifier to a float variable declaration in
    the weightCalcHelper() function, which otherwise produces slightly
    different float results on amd64 and i386 hosts. In turn, this can lead
    to different (but equivalent) register choices, and thus non-identical
    assembly code.

    See https://github.com/llvm/llvm-project/issues/99396 for more details.

    Note this is a temporary fix, meant to merge in time for 13.4. As soon
    as upstream has a permanent solution we will import that.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 397c2693fa66508cb5e6b173650a1f3bc6c4dd4f)

 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Comment 7 commit-hook freebsd_committer freebsd_triage 2024-07-25 18:04:06 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=55a2a91c5e1bb39dd625ba56597608883fbcb318

commit 55a2a91c5e1bb39dd625ba56597608883fbcb318
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-25 11:13:45 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-25 18:03:01 +0000

    Merge commit 28a2b85602a5 from llvm-project (by Kazu Hirata):

      [DeadStoreElimination] Use SmallSetVector (NFC) (#79410)

      The use of SmallSetVector saves 0.58% of heap allocations during the
      compilation of a large preprocessed file, namely X86ISelLowering.cpp,
      for the X86 target.  During the experiment, the final size of ToCheck
      was 8 or less 88% of the time.

    Merge commit 9e95c4947d31 from llvm-project (by Nikita Popov):

      [DSE] Fix non-determinism due to address reuse (#84943)

      The malloc->calloc fold creates a new MemoryAccess, which may end of at
      the same address as a previously deleted access inside SkipStores.

      To the most part, this is not a problem, because SkipStores is normally
      only used together with MemDefs. Neither the old malloc access nor the
      new calloc access will be part of MemDefs, so there is no problem here.

      However, SkipStores is also used in one more place: In the main DSE
      loop, ToCheck entries are checked against it. Fix this by not using
      SkipStores here, and instead using a separate set to track deletions
      inside this loop. This way it is not affected by the calloc optimization
      that happens outside it.

      This is all pretty ugly, but I haven't found another good way to fix it.
      Suggestions welcome.

      No test case as I don't have a reliable DSE-only test-case for this.

      Fixes https://github.com/llvm/llvm-project/issues/84458.

    This fixes another possible difference in output when building i386
    object files with a native or cross build of clang. (Specifically, the
    file sbin/ipf/ipmon/ipmon.o.)

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

 .../lib/Transforms/Scalar/DeadStoreElimination.cpp | 24 ++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
Comment 8 Dimitry Andric freebsd_committer freebsd_triage 2024-07-25 18:08:38 UTC
With both the fixes from comment 4 and comment 7, I could now succesfully build i386 world native and cross, and all the differences were in host-only tools (i.e. below /usr/obj/usr/src/i386.i386/tmp/), or files with generated timestamps.

After the last fix has been MFC'd I think we can close this issue.
Comment 9 Ed Maste freebsd_committer freebsd_triage 2024-07-25 18:12:53 UTC
(In reply to Dimitry Andric from comment #8)
> files with generated timestamps.

Do you have a list of those handy?
Comment 10 Dimitry Andric freebsd_committer freebsd_triage 2024-07-25 18:54:33 UTC
(In reply to Ed Maste from comment #9)

The list:

usr/src/i386.i386/stand/ficl/softcore.c
usr/src/i386.i386/stand/i386/loader_4th/loader_4th
usr/src/i386.i386/stand/i386/loader_4th/loader_4th.bin
usr/src/i386.i386/stand/i386/loader_4th/loader_4th.sym
usr/src/i386.i386/stand/i386/loader_4th/vers.c
usr/src/i386.i386/stand/i386/loader_4th/vers.o
usr/src/i386.i386/stand/i386/loader_lua/loader_lua
usr/src/i386.i386/stand/i386/loader_lua/loader_lua.bin
usr/src/i386.i386/stand/i386/loader_lua/loader_lua.sym
usr/src/i386.i386/stand/i386/loader_lua/vers.c
usr/src/i386.i386/stand/i386/loader_lua/vers.o
usr/src/i386.i386/stand/i386/loader_simp/loader_simp
usr/src/i386.i386/stand/i386/loader_simp/loader_simp.bin
usr/src/i386.i386/stand/i386/loader_simp/loader_simp.sym
usr/src/i386.i386/stand/i386/loader_simp/vers.c
usr/src/i386.i386/stand/i386/loader_simp/vers.o
usr/src/i386.i386/stand/i386/pxeldr/loader
usr/src/i386.i386/stand/i386/pxeldr/pxeboot
usr/src/i386.i386/sys/GENERIC/kernel
usr/src/i386.i386/sys/GENERIC/kernel.debug
usr/src/i386.i386/sys/GENERIC/kernel.full
usr/src/i386.i386/sys/GENERIC/vers.c
usr/src/i386.i386/sys/GENERIC/vers.o
usr/src/i386.i386/toolchain-metadata.mk

Note that all binary file differences are due to the .c file differences. The interesting diff is:

--- usr.cross/src/i386.i386/stand/ficl/softcore.c       2024-07-25 19:19:31.202747000 +0200
+++ usr.native/src/i386.i386/stand/ficl/softcore.c      2024-07-25 15:51:43.365742000 +0200
@@ -4,7 +4,7 @@
 ** Words from CORE set written in FICL
 ** Author: John Sadler (john_sadler@alum.mit.edu)
 ** Created: 27 December 1997
-** Last update: Thu Jul 25 19:19:31 CEST 2024
+** Last update: Thu Jul 25 15:51:43 CEST 2024
 *******************************************************************/
 /*
 ** DO NOT EDIT THIS FILE -- it is generated by softwords/softcore.awk
--- usr.cross/src/i386.i386/stand/i386/loader_4th/vers.c        2024-07-25 19:19:47.305939000 +0200
+++ usr.native/src/i386.i386/stand/i386/loader_4th/vers.c       2024-07-25 15:52:06.222813000 +0200
@@ -1,2 +1,2 @@
-char bootprog_info[] = "FreeBSD/x86 bootstrap loader, Revision 1.1\n(Thu Jul 25 19:19:47 CEST 2024 dim@freebsd14-amd64-dimtest)\n";
+char bootprog_info[] = "FreeBSD/x86 bootstrap loader, Revision 1.1\n(Thu Jul 25 15:52:06 CEST 2024 dim@freebsd14-i386-dimtest)\n";
 unsigned bootprog_rev = 1 * 1000 + 1;
--- usr.cross/src/i386.i386/stand/i386/loader_lua/vers.c        2024-07-25 19:19:48.214825000 +0200
+++ usr.native/src/i386.i386/stand/i386/loader_lua/vers.c       2024-07-25 15:52:06.504217000 +0200
@@ -1,2 +1,2 @@
-char bootprog_info[] = "FreeBSD/x86 bootstrap loader, Revision 1.1\n(Thu Jul 25 19:19:48 CEST 2024 dim@freebsd14-amd64-dimtest)\n";
+char bootprog_info[] = "FreeBSD/x86 bootstrap loader, Revision 1.1\n(Thu Jul 25 15:52:06 CEST 2024 dim@freebsd14-i386-dimtest)\n";
 unsigned bootprog_rev = 1 * 1000 + 1;
--- usr.cross/src/i386.i386/stand/i386/loader_simp/vers.c       2024-07-25 19:19:50.668663000 +0200
+++ usr.native/src/i386.i386/stand/i386/loader_simp/vers.c      2024-07-25 15:52:09.581807000 +0200
@@ -1,2 +1,2 @@
-char bootprog_info[] = "FreeBSD/x86 bootstrap loader, Revision 1.1\n(Thu Jul 25 19:19:50 CEST 2024 dim@freebsd14-amd64-dimtest)\n";
+char bootprog_info[] = "FreeBSD/x86 bootstrap loader, Revision 1.1\n(Thu Jul 25 15:52:09 CEST 2024 dim@freebsd14-i386-dimtest)\n";
 unsigned bootprog_rev = 1 * 1000 + 1;
--- usr.cross/src/i386.i386/sys/GENERIC/vers.c  2024-07-25 19:28:44.623840000 +0200
+++ usr.native/src/i386.i386/sys/GENERIC/vers.c 2024-07-25 16:14:12.929585000 +0200
@@ -34,8 +34,8 @@
  * method is deprecated and will be removed in a future version of FreeBSD. Orgs
  * that use it are encouraged to migrate before then.
  */
-#define SCCSSTR "@(#)FreeBSD 15.0-CURRENT #0 main-n271383-768485a98783: Thu Jul 25 19:28:40 CEST 2024"
-#define VERSTR "FreeBSD 15.0-CURRENT #0 main-n271383-768485a98783: Thu Jul 25 19:28:40 CEST 2024\n    dim@freebsd14-amd64-dimtest:/usr/obj/share/dim/src/freebsd/src/i386.i386/sys/GENERIC\n"
+#define SCCSSTR "@(#)FreeBSD 15.0-CURRENT #0 main-n271383-768485a98783: Thu Jul 25 16:14:00 CEST 2024"
+#define VERSTR "FreeBSD 15.0-CURRENT #0 main-n271383-768485a98783: Thu Jul 25 16:14:00 CEST 2024\n    dim@freebsd14-i386-dimtest:/usr/obj/share/dim/src/freebsd/src/i386.i386/sys/GENERIC\n"
 #define RELSTR "15.0-CURRENT"

 char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR;
--- usr.cross/src/i386.i386/toolchain-metadata.mk       2024-07-25 18:32:55.156214000 +0200
+++ usr.native/src/i386.i386/toolchain-metadata.mk      2024-07-25 14:11:10.914450000 +0200
@@ -1,4 +1,4 @@
-.info Using cached toolchain metadata from build at freebsd14-amd64-dimtest on Thu Jul 25 18:32:55 CEST 2024
+.info Using cached toolchain metadata from build at freebsd14-i386-dimtest on Thu Jul 25 14:11:10 CEST 2024
 _LOADED_TOOLCHAIN_METADATA=t
 COMPILER_VERSION=180106
 X_COMPILER_VERSION=180106


However, I think the default isn't MK_REPRODUCIBLE=yes? I should probably redo the whole thing with that option explicitly turned on.
Comment 11 Ed Maste freebsd_committer freebsd_triage 2024-07-25 20:26:07 UTC
(In reply to Dimitry Andric from comment #10)
REPRODUCIBLE_BUILD is off by default in main as of

commit a58383d257887c5d4f6226bad3c67bead1d3016d
Author: John Baldwin <jhb@FreeBSD.org>
Date:   Sat Aug 3 01:06:17 2019 +0000

    Flip REPRODUCIBLE_BUILD back to off by default in head.
    
    Having the full uname output can be useful on head even with
    unmodified trees or trees that newvers.sh fails to recognize as
    modified.
    
    Reviewed by:    emaste
    Differential Revision:  https://reviews.freebsd.org/D20895
Comment 12 commit-hook freebsd_committer freebsd_triage 2024-07-25 22:01:37 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=a4e0cb2fd99adef4cbd4f778be729a079a5f2dea

commit a4e0cb2fd99adef4cbd4f778be729a079a5f2dea
Author:     Brooks Davis <brooks@FreeBSD.org>
AuthorDate: 2024-07-25 22:00:32 +0000
Commit:     Brooks Davis <brooks@FreeBSD.org>
CommitDate: 2024-07-25 22:00:32 +0000

    devel/llvm18: fix host dependent compiler output for i386

    Merge fixes for a couple cases where the compiler generated different
    i386 code depending on the host.  In the base system this showed up as
    very small differences in a couple object files in buildworld for i386
    depending on the host architecture (i386 or amd64).

    PR:             276961

 devel/llvm18/Makefile                              |   2 +-
 .../files/patch-backport-freebsd-397c2693fa6 (new) |  43 +++++++
 .../files/patch-backport-freebsd-55a2a91c5e1 (new) | 128 +++++++++++++++++++++
 3 files changed, 172 insertions(+), 1 deletion(-)
Comment 13 commit-hook freebsd_committer freebsd_triage 2024-07-25 22:54:42 UTC
A commit in branch 2024Q3 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=20f78eb6a09b06a7258cf2981a7325759a81f486

commit 20f78eb6a09b06a7258cf2981a7325759a81f486
Author:     Brooks Davis <brooks@FreeBSD.org>
AuthorDate: 2024-07-25 22:00:32 +0000
Commit:     Brooks Davis <brooks@FreeBSD.org>
CommitDate: 2024-07-25 22:29:34 +0000

    devel/llvm18: fix host dependent compiler output for i386

    Merge fixes for a couple cases where the compiler generated different
    i386 code depending on the host.  In the base system this showed up as
    very small differences in a couple object files in buildworld for i386
    depending on the host architecture (i386 or amd64).

    PR:             276961
    (cherry picked from commit a4e0cb2fd99adef4cbd4f778be729a079a5f2dea)

 devel/llvm18/Makefile                              |   2 +-
 .../files/patch-backport-freebsd-397c2693fa6 (new) |  43 +++++++
 .../files/patch-backport-freebsd-55a2a91c5e1 (new) | 128 +++++++++++++++++++++
 3 files changed, 172 insertions(+), 1 deletion(-)
Comment 14 Dimitry Andric freebsd_committer freebsd_triage 2024-07-26 09:28:30 UTC
(In reply to Ed Maste from comment #11)
With MK_REPRODUCIBLE=yes, the only differences left are in softcore.c (but it's just a comment, it doesn't influence the object file), and toolchain-metadata.mk:

--- usr.cross/src/freebsd/src/i386.i386/stand/ficl/softcore.c   2024-07-25 23:31:51.712823000 +0200
+++ usr.native/src/freebsd/src/i386.i386/stand/ficl/softcore.c  2024-07-26 00:38:30.897665000 +0200
@@ -4,7 +4,7 @@
 ** Words from CORE set written in FICL
 ** Author: John Sadler (john_sadler@alum.mit.edu)
 ** Created: 27 December 1997
-** Last update: Thu Jul 25 23:31:51 CEST 2024
+** Last update: Fri Jul 26 00:38:30 CEST 2024
 *******************************************************************/
 /*
 ** DO NOT EDIT THIS FILE -- it is generated by softwords/softcore.awk
--- usr.cross/src/freebsd/src/i386.i386/toolchain-metadata.mk   2024-07-25 22:36:43.882535000 +0200
+++ usr.native/src/freebsd/src/i386.i386/toolchain-metadata.mk  2024-07-25 22:59:09.724274000 +0200
@@ -1,4 +1,4 @@
-.info Using cached toolchain metadata from build at freebsd14-amd64-dimtest on Thu Jul 25 22:36:43 CEST 2024
+.info Using cached toolchain metadata from build at freebsd14-i386-dimtest on Thu Jul 25 22:59:09 CEST 2024
 _LOADED_TOOLCHAIN_METADATA=t
 COMPILER_VERSION=180106
 X_COMPILER_VERSION=180106
Comment 15 commit-hook freebsd_committer freebsd_triage 2024-07-28 11:05:07 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=806fbbbbe5d2eb5decfaafba6deb94534655cc8e

commit 806fbbbbe5d2eb5decfaafba6deb94534655cc8e
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-25 11:13:45 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-28 11:04:21 +0000

    Merge commit 28a2b85602a5 from llvm-project (by Kazu Hirata):

      [DeadStoreElimination] Use SmallSetVector (NFC) (#79410)

      The use of SmallSetVector saves 0.58% of heap allocations during the
      compilation of a large preprocessed file, namely X86ISelLowering.cpp,
      for the X86 target.  During the experiment, the final size of ToCheck
      was 8 or less 88% of the time.

    Merge commit 9e95c4947d31 from llvm-project (by Nikita Popov):

      [DSE] Fix non-determinism due to address reuse (#84943)

      The malloc->calloc fold creates a new MemoryAccess, which may end of at
      the same address as a previously deleted access inside SkipStores.

      To the most part, this is not a problem, because SkipStores is normally
      only used together with MemDefs. Neither the old malloc access nor the
      new calloc access will be part of MemDefs, so there is no problem here.

      However, SkipStores is also used in one more place: In the main DSE
      loop, ToCheck entries are checked against it. Fix this by not using
      SkipStores here, and instead using a separate set to track deletions
      inside this loop. This way it is not affected by the calloc optimization
      that happens outside it.

      This is all pretty ugly, but I haven't found another good way to fix it.
      Suggestions welcome.

      No test case as I don't have a reliable DSE-only test-case for this.

      Fixes https://github.com/llvm/llvm-project/issues/84458.

    This fixes another possible difference in output when building i386
    object files with a native or cross build of clang. (Specifically, the
    file sbin/ipf/ipmon/ipmon.o.)

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 55a2a91c5e1bb39dd625ba56597608883fbcb318)

 .../lib/Transforms/Scalar/DeadStoreElimination.cpp | 24 ++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
Comment 16 commit-hook freebsd_committer freebsd_triage 2024-07-28 11:06:09 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c7e5a0d42f20e9a81b96fc89b3b9883cf4be9839

commit c7e5a0d42f20e9a81b96fc89b3b9883cf4be9839
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-25 11:13:45 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-28 11:04:26 +0000

    Merge commit 28a2b85602a5 from llvm-project (by Kazu Hirata):

      [DeadStoreElimination] Use SmallSetVector (NFC) (#79410)

      The use of SmallSetVector saves 0.58% of heap allocations during the
      compilation of a large preprocessed file, namely X86ISelLowering.cpp,
      for the X86 target.  During the experiment, the final size of ToCheck
      was 8 or less 88% of the time.

    Merge commit 9e95c4947d31 from llvm-project (by Nikita Popov):

      [DSE] Fix non-determinism due to address reuse (#84943)

      The malloc->calloc fold creates a new MemoryAccess, which may end of at
      the same address as a previously deleted access inside SkipStores.

      To the most part, this is not a problem, because SkipStores is normally
      only used together with MemDefs. Neither the old malloc access nor the
      new calloc access will be part of MemDefs, so there is no problem here.

      However, SkipStores is also used in one more place: In the main DSE
      loop, ToCheck entries are checked against it. Fix this by not using
      SkipStores here, and instead using a separate set to track deletions
      inside this loop. This way it is not affected by the calloc optimization
      that happens outside it.

      This is all pretty ugly, but I haven't found another good way to fix it.
      Suggestions welcome.

      No test case as I don't have a reliable DSE-only test-case for this.

      Fixes https://github.com/llvm/llvm-project/issues/84458.

    This fixes another possible difference in output when building i386
    object files with a native or cross build of clang. (Specifically, the
    file sbin/ipf/ipmon/ipmon.o.)

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 55a2a91c5e1bb39dd625ba56597608883fbcb318)

 .../lib/Transforms/Scalar/DeadStoreElimination.cpp | 24 ++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
Comment 17 commit-hook freebsd_committer freebsd_triage 2024-07-28 11:14:11 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=52552d7572a6d3d7d3ce0d6862de9a9d203c5d01

commit 52552d7572a6d3d7d3ce0d6862de9a9d203c5d01
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-28 11:08:50 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-28 11:09:03 +0000

    Revert "Fix llvm register allocator for native/cross build differences"

    In preparation for applying the fix that landed upstream, this reverts
    commit 397c2693fa66508cb5e6b173650a1f3bc6c4dd4f:

      Fix llvm register allocator for native/cross build differences

      Work around an issue in LLVM's register allocator, which can cause
      slightly different i386 object files, when produced by a native or cross
      build of clang.

      This adds another volatile qualifier to a float variable declaration in
      the weightCalcHelper() function, which otherwise produces slightly
      different float results on amd64 and i386 hosts. In turn, this can lead
      to different (but equivalent) register choices, and thus non-identical
      assembly code.

      See https://github.com/llvm/llvm-project/issues/99396 for more details.

      Note this is a temporary fix, meant to merge in time for 13.4. As soon
      as upstream has a permanent solution we will import that.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Comment 18 commit-hook freebsd_committer freebsd_triage 2024-07-28 11:14:12 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=1a4b8325f6e3a45c77188343da504fe04495cc46

commit 1a4b8325f6e3a45c77188343da504fe04495cc46
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-28 11:13:37 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-28 11:13:37 +0000

    Merge commit c80c09f3e380 from llvm-project (by Dimitry Andric):

      [CalcSpillWeights] Avoid x87 excess precision influencing weight result

      Fixes #99396

      The result of `VirtRegAuxInfo::weightCalcHelper` can be influenced by
      x87 excess precision, which can result in slightly different register
      choices when the compiler is hosted on x86_64 or i386. This leads to
      different object file output when cross-compiling to i386, or native.

      Similar to 7af3432e22b0, we need to add a `volatile` qualifier to the
      local `Weight` variable to force it onto the stack, and avoid the excess
      precision. Define `stack_float_t` in `MathExtras.h` for this purpose,
      and use it.

    This is the version of the fix for PR276961 that landed upstream.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

 contrib/llvm-project/llvm/include/llvm/Support/MathExtras.h |  8 ++++++++
 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp  | 11 ++++++-----
 2 files changed, 14 insertions(+), 5 deletions(-)
Comment 19 commit-hook freebsd_committer freebsd_triage 2024-07-31 07:31:02 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=96ff33484ee5f44bc3ac743ce235d23495465904

commit 96ff33484ee5f44bc3ac743ce235d23495465904
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-28 11:13:37 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-31 07:29:47 +0000

    Merge commit c80c09f3e380 from llvm-project (by Dimitry Andric):

      [CalcSpillWeights] Avoid x87 excess precision influencing weight result

      Fixes #99396

      The result of `VirtRegAuxInfo::weightCalcHelper` can be influenced by
      x87 excess precision, which can result in slightly different register
      choices when the compiler is hosted on x86_64 or i386. This leads to
      different object file output when cross-compiling to i386, or native.

      Similar to 7af3432e22b0, we need to add a `volatile` qualifier to the
      local `Weight` variable to force it onto the stack, and avoid the excess
      precision. Define `stack_float_t` in `MathExtras.h` for this purpose,
      and use it.

    This is the version of the fix for PR276961 that landed upstream.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 1a4b8325f6e3a45c77188343da504fe04495cc46)

 contrib/llvm-project/llvm/include/llvm/Support/MathExtras.h |  8 ++++++++
 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp  | 11 ++++++-----
 2 files changed, 14 insertions(+), 5 deletions(-)
Comment 20 commit-hook freebsd_committer freebsd_triage 2024-07-31 07:31:04 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=51158386635294dc6cff48090a1a8faa2c97553e

commit 51158386635294dc6cff48090a1a8faa2c97553e
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-28 11:08:50 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-31 07:29:41 +0000

    Revert "Fix llvm register allocator for native/cross build differences"

    In preparation for applying the fix that landed upstream, this reverts
    commit 397c2693fa66508cb5e6b173650a1f3bc6c4dd4f:

      Fix llvm register allocator for native/cross build differences

      Work around an issue in LLVM's register allocator, which can cause
      slightly different i386 object files, when produced by a native or cross
      build of clang.

      This adds another volatile qualifier to a float variable declaration in
      the weightCalcHelper() function, which otherwise produces slightly
      different float results on amd64 and i386 hosts. In turn, this can lead
      to different (but equivalent) register choices, and thus non-identical
      assembly code.

      See https://github.com/llvm/llvm-project/issues/99396 for more details.

      Note this is a temporary fix, meant to merge in time for 13.4. As soon
      as upstream has a permanent solution we will import that.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 52552d7572a6d3d7d3ce0d6862de9a9d203c5d01)

 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Comment 21 commit-hook freebsd_committer freebsd_triage 2024-07-31 07:37:06 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=814641415fbee707b312c334e00a1ebb5157e37b

commit 814641415fbee707b312c334e00a1ebb5157e37b
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-28 11:13:37 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-31 07:29:59 +0000

    Merge commit c80c09f3e380 from llvm-project (by Dimitry Andric):

      [CalcSpillWeights] Avoid x87 excess precision influencing weight result

      Fixes #99396

      The result of `VirtRegAuxInfo::weightCalcHelper` can be influenced by
      x87 excess precision, which can result in slightly different register
      choices when the compiler is hosted on x86_64 or i386. This leads to
      different object file output when cross-compiling to i386, or native.

      Similar to 7af3432e22b0, we need to add a `volatile` qualifier to the
      local `Weight` variable to force it onto the stack, and avoid the excess
      precision. Define `stack_float_t` in `MathExtras.h` for this purpose,
      and use it.

    This is the version of the fix for PR276961 that landed upstream.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 1a4b8325f6e3a45c77188343da504fe04495cc46)

 contrib/llvm-project/llvm/include/llvm/Support/MathExtras.h |  8 ++++++++
 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp  | 11 ++++++-----
 2 files changed, 14 insertions(+), 5 deletions(-)
Comment 22 commit-hook freebsd_committer freebsd_triage 2024-07-31 07:37:07 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c8d0dca5e5f75b98c90ccfb60f8a65523072a8c7

commit c8d0dca5e5f75b98c90ccfb60f8a65523072a8c7
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-07-28 11:08:50 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-07-31 07:29:57 +0000

    Revert "Fix llvm register allocator for native/cross build differences"

    In preparation for applying the fix that landed upstream, this reverts
    commit 397c2693fa66508cb5e6b173650a1f3bc6c4dd4f:

      Fix llvm register allocator for native/cross build differences

      Work around an issue in LLVM's register allocator, which can cause
      slightly different i386 object files, when produced by a native or cross
      build of clang.

      This adds another volatile qualifier to a float variable declaration in
      the weightCalcHelper() function, which otherwise produces slightly
      different float results on amd64 and i386 hosts. In turn, this can lead
      to different (but equivalent) register choices, and thus non-identical
      assembly code.

      See https://github.com/llvm/llvm-project/issues/99396 for more details.

      Note this is a temporary fix, meant to merge in time for 13.4. As soon
      as upstream has a permanent solution we will import that.

    PR:             276961
    Reported by:    cperciva
    MFC after:      3 days

    (cherry picked from commit 52552d7572a6d3d7d3ce0d6862de9a9d203c5d01)

 contrib/llvm-project/llvm/lib/CodeGen/CalcSpillWeights.cpp | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Comment 23 Dimitry Andric freebsd_committer freebsd_triage 2024-07-31 07:38:32 UTC
Everything has now been committed and merged back in time for releng/13.4 and releng/14.2.