Bug 229708

Summary: Don't require the program header for ELF files fit in the first page
Product: Base System Reporter: Alex Arslan <ararslan>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: New ---    
Severity: Affects Some People CC: cem, emaste, grahamperrin, iwtcex, kib, lantw44, lwhsu, trueos
Priority: ---    
Version: 11.2-RELEASE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
executable none

Description Alex Arslan 2018-07-11 20:33:48 UTC
According to a comment in libexec/rtld-elf/map_object.c,

"We rely on the program header being in the first page. This is not strictly required by the ABI specification, but it seems to always true in practice. And, it simplifies things considerably."

This assumption is quickly violated when using tools such as patchelf, which often relocates the program header to the end of the ELF header, causing it to no longer fit within the first page. While perhaps not the most ideal behavior, it doesn't appear to violate any ABI specifications.

The build system for the Julia language (https://julialang.org), which supports FreeBSD, and our new dependency cross-compilation system (https://github.com/JuliaPackaging/BinaryBuilder.jl), which supports building for FreeBSD, make heavy use of patchelf for setting RPATHs. That has caused a number of issues for us on FreeBSD though, because we run into FreeBSD's incomplete implementation of the ABI specification, making shared libraries unloadable.
Comment 1 Konstantin Belousov freebsd_committer freebsd_triage 2020-09-04 13:04:30 UTC
https://reviews.freebsd.org/D26323

Can somebody provide a usable test case ?  I mean, the binary and dep library, where library has phdrs outside page 0 ?

Also, do you want same for binaries ?  Our kernel activator does not handle it either.  If yes, this would be a separate change.
Comment 2 Alex S 2020-09-04 16:48:34 UTC
(In reply to Konstantin Belousov from comment #1)

Well, supposedly patchelf produces such executables, although reading [1] https://github.com/NixOS/patchelf/issues/153 and especially [2] gives me the impression this might have been fixed in the meantime.

Another potential use case is Widevine CDM (obtainable from [3] and [4]), which I intend to test with Firefox/Chromium and my glibc ABI compatibility hack ([5], think linuxpluginwrapper, but more reliable) to see how feasible that approach actually is.

% env LD_PRELOAD=$PWD/libwidevinecdm.so true
ld-elf.so.1: .../libwidevinecdm.so: program header too large
% readelf -h libwidevinecdm.so 
...
  Start of program headers:          9300548 (bytes into file)
  Start of section headers:          9301776 (bytes into file)

Since this is a proprietary closed source library, I don't know how it was built.

> Also, do you want same for binaries ?

Likely not.

[1] https://github.com/NixOS/patchelf/issues/153
[2] https://github.com/NixOS/patchelf/issues/165#issuecomment-477084014
[3] https://dl.google.com/widevine-cdm/4.10.1582.2-linux-x64.zip
[4] https://dl.google.com/widevine-cdm/4.10.1582.2-linux-ia32.zip
[5] https://github.com/shkhln/nvshim
Comment 3 Konstantin Belousov freebsd_committer freebsd_triage 2020-09-04 17:36:55 UTC
orion% LD_PRELOAD=/tmp/libwidevinecdm.so obj/usr/home/kostik/work/build/bsd/DEV/src/amd64.amd64/libexec/rtld-elf/ld-elf.so.1 /bin/ls
ld-elf.so.1: Shared object "libpthread.so.0" not found, required by "libwidevinecdm.so"

Which indicates that phdrs were parsed relatively ok-ish.  Still I need a *FreeBSD* amd64 or i386 library with phdrs out of page 0, that I can link to some binary or dlopen().  If there is some function symbol that can be called (so I see that other processing was fine) it would be ideal.
Comment 4 Konstantin Belousov freebsd_committer freebsd_triage 2020-09-04 17:40:22 UTC
(In reply to Konstantin Belousov from comment #3)
Or rebuild your instance of rtld with the patch in D26323 and check.
Comment 5 Alex S 2020-09-04 19:50:44 UTC
(In reply to Konstantin Belousov from comment #3)

> Still I need a *FreeBSD* amd64 or i386 library with phdrs out of page 0,
> that I can link to some binary or dlopen().

Downgrading patchelf to 0.9 is the only option then.

(In reply to Konstantin Belousov from comment #4)

> Or rebuild your instance of rtld with the patch in D26323 and check.

The patch works with my simplistic tests. Widevine eventually crashes for some other rtld-related reason. (Should I file a bug specifically for it? I'm doing this out of curiosity, not strict necessity.)
Comment 6 Konstantin Belousov freebsd_committer freebsd_triage 2020-09-04 20:08:44 UTC
(In reply to Alex S from comment #5)
So will you supply me with the library ?

Feel free to report additional issue there, or in new PR.  In the later case, add me to PR, so I note it.
Comment 7 Alex S 2020-09-04 20:34:19 UTC
Created attachment 217754 [details]
executable

(In reply to Konstantin Belousov from comment #6)

> So will you supply me with the library ?

% uname -a
FreeBSD desktop 12.1-RELEASE-p8 FreeBSD 12.1-RELEASE-p8 GENERIC  amd64
% cat libhello.c 
#include <stdio.h>

__attribute__((constructor))
void hello() {
  printf("Hello there\n");
}
% cc -fPIC -shared libhello.c -o libhello.so
% .../patchelf --version
patchelf 0.9
% .../patchelf --set-soname hellothere.so libhello.so
% readelf -h libhello.so
...
  Start of program headers:          20480 (bytes into file)
  Start of section headers:          15048 (bytes into file)
Comment 8 Alex S 2020-09-04 21:26:52 UTC
(In reply to Konstantin Belousov from comment #6)

> Feel free to report additional issue there, or in new PR.  In the later case, add me to PR, so I note it.

Looks like libwidevinecdm.so in a typical Google fashion overrides (c|m|re|)alloc, so the crash is 100% their fault.
Comment 9 Konstantin Belousov freebsd_committer freebsd_triage 2020-09-04 21:53:31 UTC
Thank you, this was useful:

orion% ./test               ~/build/bsd/DEV/stuff/tests/RTLD/phdr_not_zero_page
Hello there
Hello there
Comment 10 commit-hook freebsd_committer freebsd_triage 2020-09-05 10:17:10 UTC
A commit references this bug:

Author: kib
Date: Sat Sep  5 10:16:24 UTC 2020
New revision: 365360
URL: https://svnweb.freebsd.org/changeset/base/365360

Log:
  rtld: Handle ELF dso with program headers outside the first page.

  Reported by:	Alex Arslan <alex.arslan@julialang.org>
  PR:	229708
  Reviewed by:	dim (previous version), emaste
  Sponsored by:	The FreeBSD Foundation
  Differential revision:	https://reviews.freebsd.org/D26323

Changes:
  head/libexec/rtld-elf/map_object.c
Comment 11 commit-hook freebsd_committer freebsd_triage 2020-09-11 10:06:45 UTC
A commit references this bug:

Author: kib
Date: Fri Sep 11 10:05:44 UTC 2020
New revision: 365627
URL: https://svnweb.freebsd.org/changeset/base/365627

Log:
  MFC r365360, r365370:
  rtld: Handle ELF dso with program headers outside the first page.

  PR:	229708

Changes:
_U  stable/12/
  stable/12/libexec/rtld-elf/map_object.c
Comment 12 commit-hook freebsd_committer freebsd_triage 2020-09-11 10:11:50 UTC
A commit references this bug:

Author: kib
Date: Fri Sep 11 10:11:04 UTC 2020
New revision: 365629
URL: https://svnweb.freebsd.org/changeset/base/365629

Log:
  MFC r365360, r365370:
  rtld: Handle ELF dso with program headers outside the first page.

  PR:	229708

Changes:
_U  stable/11/
  stable/11/libexec/rtld-elf/map_object.c