Bug 248516 - elftoolchain error when building linux_vdso.so with llvm objcopy
Summary: elftoolchain error when building linux_vdso.so with llvm objcopy
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-toolchain (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-08-07 13:01 UTC by Alex Richardson
Modified: 2020-12-23 02:42 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Richardson freebsd_committer 2020-08-07 13:01:18 UTC
I am seeing "strip: gelf_getshdr failed: Invalid argument"
when building linux_vdso.so (compiled with upstream clang 10 and using llvm-objcopy).
I am not sure if this is a problem with llvm-objcopy 10 or whether we should be passing different flags (llvm-objcopy has a --strip-all-gnu flag that produces compatible output).
Since we invoke strip afterwards so I'm not sure if the -S flag is needed at all.

Whatever we choose to do, I don't think elftoolchain strip should be rejecting the file produced by llvm-objcopy?
It works just fine if I use  `/usr/local/opt/llvm/bin/llvm-strip -N _binary_linux_locore_o_size vdso.llvm-objcopy.so`, but using elftoolchain strip gives me the error

--- vdso.elftoolchain-objcopy.so
+++ vdso.llvm-objcopy.so
├── greadelf --wide --file-header {}
│ @@ -6,15 +6,15 @@
│    OS/ABI:                            UNIX - System V
│    ABI Version:                       0
│    Type:                              REL (Relocatable file)
│    Machine:                           Advanced Micro Devices X86-64
│    Version:                           0x1
│    Entry point address:               0x0
│    Start of program headers:          0 (bytes into file)
│ -  Start of section headers:          3352 (bytes into file)
│ +  Start of section headers:          3328 (bytes into file)
│    Flags:                             0x0
│    Size of this header:               64 (bytes)
│    Size of program headers:           0 (bytes)
│    Number of program headers:         0
│    Size of section headers:           64 (bytes)
│ -  Number of section headers:         5
│ -  Section header string table index: 2
│ +  Number of section headers:         3
│ +  Section header string table index: 1
├── greadelf --wide --sections {}
│ @@ -1,14 +1,12 @@
│ -There are 5 section headers, starting at offset 0xd18:
│ +There are 3 section headers, starting at offset 0xd00:
│  
│  Section Headers:
│    [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
│    [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
│ -  [ 1] .data             PROGBITS        0000000000000000 000040 000cb0 00  WA  0   0  1
│ -  [ 2] .shstrtab         STRTAB          0000000000000000 000cf0 000022 00      0   0  1
│ -  [ 3] .symtab           SYMTAB          0000000000000000 000e58 000078 18      4   2  8
│ -  [ 4] .strtab           STRTAB          0000000000000000 000ed0 000055 00      0   0  1
│ +  [ 1] .strtab           STRTAB          0000000000000000 000040 00000f 00      0   0  1
│ +  [ 2] .data             PROGBITS        0000000000000000 00004f 000cb0 00  WA  0   0  1
│  Key to Flags:
│    W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
│    L (link order), O (extra OS processing required), G (group), T (TLS),
│    C (compressed), x (unknown), o (OS specific), E (exclude),
│    l (large), p (processor specific)
├── greadelf --wide --symbols {}
│ @@ -1,8 +0,0 @@
│ -
│ -Symbol table '.symtab' contains 5 entries:
│ -   Num:    Value          Size Type    Bind   Vis      Ndx Name
│ -     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
│ -     1: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 
│ -     2: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    1 _binary_linux_locore_o_start
│ -     3: 0000000000000cb0     0 NOTYPE  GLOBAL DEFAULT    1 _binary_linux_locore_o_end
│ -     4: 0000000000000cb0     0 NOTYPE  GLOBAL DEFAULT  ABS _binary_linux_locore_o_size
├── strings --all --bytes=8 {}
│ @@ -24,11 +24,7 @@
│  __vdso_gettimeofday
│  __vdso_time
│  clock_gettime
│  gettimeofday
│  linux_platform
│  linux_rt_sigcode
│  _DYNAMIC
│ -.shstrtab
│ -_binary_linux_locore_o_start
│ -_binary_linux_locore_o_end
│ -_binary_linux_locore_o_size
├── greadelf --wide --decompress --hex-dump=.strtab {}
│ @@ -1,9 +1,4 @@
│  
│  Hex dump of section '.strtab':
│ -  0x00000000 005f6269 6e617279 5f6c696e 75785f6c ._binary_linux_l
│ -  0x00000010 6f636f72 655f6f5f 73746172 74005f62 ocore_o_start._b
│ -  0x00000020 696e6172 795f6c69 6e75785f 6c6f636f inary_linux_loco
│ -  0x00000030 72655f6f 5f656e64 005f6269 6e617279 re_o_end._binary
│ -  0x00000040 5f6c696e 75785f6c 6f636f72 655f6f5f _linux_locore_o_
│ -  0x00000050 73697a65 00                         size.
│ +  0x00000000 002e7374 72746162 002e6461 746100   ..strtab..data.

LLVM objcopy with --strip-all-gnu

--- vdso.elftoolchain-objcopy.so
+++ vdso.llvm-objcopy-strip-all-gnu.so
├── greadelf --wide --file-header {}
│ @@ -6,15 +6,15 @@
│    OS/ABI:                            UNIX - System V
│    ABI Version:                       0
│    Type:                              REL (Relocatable file)
│    Machine:                           Advanced Micro Devices X86-64
│    Version:                           0x1
│    Entry point address:               0x0
│    Start of program headers:          0 (bytes into file)
│ -  Start of section headers:          3352 (bytes into file)
│ +  Start of section headers:          3328 (bytes into file)
│    Flags:                             0x0
│    Size of this header:               64 (bytes)
│    Size of program headers:           0 (bytes)
│    Number of program headers:         0
│    Size of section headers:           64 (bytes)
│ -  Number of section headers:         5
│ -  Section header string table index: 2
│ +  Number of section headers:         3
│ +  Section header string table index: 1
├── greadelf --wide --sections {}
│ @@ -1,14 +1,12 @@
│ -There are 5 section headers, starting at offset 0xd18:
│ +There are 3 section headers, starting at offset 0xd00:
│  
│  Section Headers:
│    [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
│    [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
│ -  [ 1] .data             PROGBITS        0000000000000000 000040 000cb0 00  WA  0   0  1
│ -  [ 2] .shstrtab         STRTAB          0000000000000000 000cf0 000022 00      0   0  1
│ -  [ 3] .symtab           SYMTAB          0000000000000000 000e58 000078 18      4   2  8
│ -  [ 4] .strtab           STRTAB          0000000000000000 000ed0 000055 00      0   0  1
│ +  [ 1] .strtab           STRTAB          0000000000000000 000040 00000f 00      0   0  1
│ +  [ 2] .data             PROGBITS        0000000000000000 00004f 000cb0 00  WA  0   0  1
│  Key to Flags:
│    W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
│    L (link order), O (extra OS processing required), G (group), T (TLS),
│    C (compressed), x (unknown), o (OS specific), E (exclude),
│    l (large), p (processor specific)
├── greadelf --wide --symbols {}
│ @@ -1,8 +0,0 @@
│ -
│ -Symbol table '.symtab' contains 5 entries:
│ -   Num:    Value          Size Type    Bind   Vis      Ndx Name
│ -     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
│ -     1: 0000000000000000     0 SECTION LOCAL  DEFAULT    1 
│ -     2: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    1 _binary_linux_locore_o_start
│ -     3: 0000000000000cb0     0 NOTYPE  GLOBAL DEFAULT    1 _binary_linux_locore_o_end
│ -     4: 0000000000000cb0     0 NOTYPE  GLOBAL DEFAULT  ABS _binary_linux_locore_o_size
├── strings --all --bytes=8 {}
│ @@ -24,11 +24,7 @@
│  __vdso_gettimeofday
│  __vdso_time
│  clock_gettime
│  gettimeofday
│  linux_platform
│  linux_rt_sigcode
│  _DYNAMIC
│ -.shstrtab
│ -_binary_linux_locore_o_start
│ -_binary_linux_locore_o_end
│ -_binary_linux_locore_o_size
├── greadelf --wide --decompress --hex-dump=.strtab {}
│ @@ -1,9 +1,4 @@
│  
│  Hex dump of section '.strtab':
│ -  0x00000000 005f6269 6e617279 5f6c696e 75785f6c ._binary_linux_l
│ -  0x00000010 6f636f72 655f6f5f 73746172 74005f62 ocore_o_start._b
│ -  0x00000020 696e6172 795f6c69 6e75785f 6c6f636f inary_linux_loco
│ -  0x00000030 72655f6f 5f656e64 005f6269 6e617279 re_o_end._binary
│ -  0x00000040 5f6c696e 75785f6c 6f636f72 655f6f5f _linux_locore_o_
│ -  0x00000050 73697a65 00                         size.
│ +  0x00000000 002e7374 72746162 002e6461 746100   ..strtab..data.



GNU objcopy output:


--- vdso.elftoolchain-objcopy.so
+++ vdso.gnu-objcopy.so
├── greadelf --wide --file-header {}
│ @@ -6,15 +6,15 @@
│    OS/ABI:                            UNIX - System V
│    ABI Version:                       0
│    Type:                              REL (Relocatable file)
│    Machine:                           Advanced Micro Devices X86-64
│    Version:                           0x1
│    Entry point address:               0x0
│    Start of program headers:          0 (bytes into file)
│ -  Start of section headers:          3352 (bytes into file)
│ +  Start of section headers:          3552 (bytes into file)
│    Flags:                             0x0
│    Size of this header:               64 (bytes)
│    Size of program headers:           0 (bytes)
│    Number of program headers:         0
│    Size of section headers:           64 (bytes)
│    Number of section headers:         5
│ -  Section header string table index: 2
│ +  Section header string table index: 4
├── greadelf --wide --sections {}
│ @@ -1,14 +1,14 @@
│ -There are 5 section headers, starting at offset 0xd18:
│ +There are 5 section headers, starting at offset 0xde0:
│  
│  Section Headers:
│    [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
│    [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
│    [ 1] .data             PROGBITS        0000000000000000 000040 000cb0 00  WA  0   0  1
│ -  [ 2] .shstrtab         STRTAB          0000000000000000 000cf0 000022 00      0   0  1
│ -  [ 3] .symtab           SYMTAB          0000000000000000 000e58 000078 18      4   2  8
│ -  [ 4] .strtab           STRTAB          0000000000000000 000ed0 000055 00      0   0  1
│ +  [ 2] .symtab           SYMTAB          0000000000000000 000cf0 000078 18      3   2  8
│ +  [ 3] .strtab           STRTAB          0000000000000000 000d68 000055 00      0   0  1
│ +  [ 4] .shstrtab         STRTAB          0000000000000000 000dbd 000021 00      0   0  1
│  Key to Flags:
│    W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
│    L (link order), O (extra OS processing required), G (group), T (TLS),
│    C (compressed), x (unknown), o (OS specific), E (exclude),
│    l (large), p (processor specific)
├── strings --all --bytes=8 {}
│ @@ -24,11 +24,11 @@
│  __vdso_gettimeofday
│  __vdso_time
│  clock_gettime
│  gettimeofday
│  linux_platform
│  linux_rt_sigcode
│  _DYNAMIC
│ -.shstrtab
│  _binary_linux_locore_o_start
│  _binary_linux_locore_o_end
│  _binary_linux_locore_o_size
│ +.shstrtab
├── greadelf --wide --decompress --hex-dump=.shstrtab {}
│ @@ -1,6 +1,6 @@
│  
│  Hex dump of section '.shstrtab':
│ -  0x00000000 00002e73 796d7461 62002e73 74727461 ...symtab..strta
│ -  0x00000010 62002e73 68737472 74616200 2e646174 b..shstrtab..dat
│ -  0x00000020 6100                                a.
│ +  0x00000000 002e7379 6d746162 002e7374 72746162 ..symtab..strtab
│ +  0x00000010 002e7368 73747274 6162002e 64617461 ..shstrtab..data
│ +  0x00000020 00                                  .
Comment 1 Alex Richardson freebsd_committer 2020-08-07 13:13:31 UTC
I've uploaded the workaround we use in CheriBSD here: https://reviews.freebsd.org/D25988
Comment 2 Mark Johnston freebsd_committer 2020-08-07 13:50:18 UTC
Can you share the SO in question?
Comment 3 Alex Richardson freebsd_committer 2020-08-07 13:56:41 UTC
I've uploaded them all to https://people.freebsd.org/~arichardson/pr248516/
Comment 4 Jessica Clarke freebsd_committer 2020-12-23 02:31:18 UTC
Did you try loading the final linux64 module? It looks from your diff like llvm-objcopy hasn't created the _binary_$file symbols in the first place and so the symbol table doesn't exist (hence the error). I expect you will get an error like `link_elf: symbol _binary_linux_locore_o_end undefined` when trying to load the resulting module, something which I see downstream in CheriBSD built with LLVM for Morello.

so I think there are two bugs in play:

1. elftoolchain's strip doesn't like being fed an ELF with no .symtab

2. llvm-objcopy isn't creating the _binary_$file symbols

Looking more closely at 2, it appears that llvm-objcopy erroneously applies -S to the output file when using -I binary; it's meant to skip copying relocation and symbol information from the source file, but presumably it gets confused and forgets that the relocation and symbol information it's generating is self-constructed.
Comment 5 Jessica Clarke freebsd_committer 2020-12-23 02:37:04 UTC
Though passing -S with -I binary doesn't really make any sense, it's not parsing the input. It's debatable whether LLVM's behaviour is actually correct or not (and fixing it is awkward since it does the ELF wrapping on the way in, not on the way out, though someone with a better knowledge of the internals there could probably figure out where exactly the checks need to go, and what).
Comment 6 Jessica Clarke freebsd_committer 2020-12-23 02:42:06 UTC
Ok this is unique to linux64; linux(32) doesn't pass -S -g to objcopy, so linux64 should just drop those, and I can confirm that then the _binary_linux_locore_o_{start,end} symbols reappear.