Bug 193009

Summary: java/openjdk8 does not build with OPTIONS_FILE_SET+=DEBUG
Product: Ports & Packages Reporter: Kamil Szczesny <mailto.kamils>
Component: Individual Port(s)Assignee: freebsd-java (Nobody) <java>
Status: Closed FIXED    
Severity: Affects Only Me CC: glewis, jkim, kib, openjdk
Priority: ---    
Version: Latest   
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
poudriere build log file
none
dynamically link
none
dynamically link
none
A patch for rtld(1) to fix the root cause
none
A revised patch to use realpath(3) instead of readlink(2).
none
An optimized patch, i.e., realpath(3) returns an absolute path. none

Description Kamil Szczesny 2014-08-26 09:08:27 UTC
java/openjdk8 fails to build in poudriere (10-RELEASE, amd64) with DEBUG option set. However, build succeeds with RELEASE option set. Log is attached.
Comment 1 Kamil Szczesny 2014-08-26 09:09:10 UTC
Created attachment 146294 [details]
poudriere build log file
Comment 2 John Marino freebsd_committer freebsd_triage 2014-08-26 12:49:34 UTC
over to maintainer
Comment 3 openjdk 2014-12-01 06:59:50 UTC
Created attachment 150060 [details]
dynamically link

new file
Comment 4 openjdk 2014-12-01 07:00:28 UTC
Created attachment 150061 [details]
dynamically link

This file will replace the existing one
Comment 5 Kamil Szczesny 2015-01-19 20:31:39 UTC
patch makes to port build with option DEBUG.
Comment 6 commit-hook freebsd_committer freebsd_triage 2015-02-15 21:18:39 UTC
A commit references this bug:

Author: glewis
Date: Sun Feb 15 21:18:13 UTC 2015
New revision: 379049
URL: https://svnweb.freebsd.org/changeset/ports/379049

Log:
  . Fix the DEBUG build.

  PR:		193009
  Submitted by:	Brian Gardner (openjdk@getsnappy.com)

Changes:
  head/java/openjdk8/files/patch-jdk-make-lib-CompileLaunchers.gmk
  head/java/openjdk8/files/patch-jdk-make-lib-ServiceabilityLibraries.gmk
Comment 7 Greg Lewis freebsd_committer freebsd_triage 2015-02-15 21:20:06 UTC
Fix committed.
Comment 8 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-26 20:41:00 UTC
Please note statically linking jli was a hack to work around a launcher problem.

$ /usr/local/openjdk8/bin/java -version
openjdk version "1.8.0_31"
OpenJDK Runtime Environment (build 1.8.0_31-b13)
OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)

Let's make a symlink and try the link.

$ ln -fs /usr/local/openjdk8/bin/java .
$ ./java -version
openjdk version "1.8.0_31"
OpenJDK Runtime Environment (build 1.8.0_31-b13)
OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)

This works.  However, the following fails:

$ $PWD/java -version
Shared object "libjli.so" not found, required by "java"

That's because the actual executable was linked with '-z origin', i.e.,

$ objdump -p /usr/local/openjdk8/bin/java | grep ORIGIN
  RPATH       $ORIGIN/../lib/amd64/jli:$ORIGIN/../lib/amd64
  RUNPATH     $ORIGIN/../lib/amd64/jli:$ORIGIN/../lib/amd64
$ ldd /usr/local/openjdk8/bin/java | grep libjli
	libjli.so => /usr/local/openjdk8/bin/../lib/amd64/jli/libjli.so (0x800a36000)
$ ldd $PWD/java | grep libjli
	libjli.so => not found (0)

As you can see, our rtld(1) resolves $ORIGIN differently if it starts with `/'.  Arguably, it is a bug in rtld, though.
Comment 9 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-26 22:22:37 UTC
Created attachment 153565 [details]
A patch for rtld(1) to fix the root cause
Comment 10 commit-hook freebsd_committer freebsd_triage 2015-02-27 06:55:08 UTC
A commit references this bug:

Author: jkim
Date: Fri Feb 27 06:54:33 UTC 2015
New revision: 380064
URL: https://svnweb.freebsd.org/changeset/ports/380064

Log:
  - Revert all damages done in r372338 and r379049.
  - Really fix non-release builds. [1]

  PR:		193009 [1]

Changes:
  head/java/openjdk8/Makefile
  head/java/openjdk8/files/patch-bsd
  head/java/openjdk8/files/patch-configure
  head/java/openjdk8/files/patch-jdk-make-lib-Awt2dLibraries.gmk
  head/java/openjdk8/files/patch-jdk-make-lib-CompileLaunchers.gmk
  head/java/openjdk8/files/patch-jdk-make-lib-CoreLibraries.gmk
  head/java/openjdk8/files/patch-jdk-make-lib-ServiceabilityLibraries.gmk
Comment 11 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-27 07:01:29 UTC
Note r379049 was reverted, i.e., all Java lauchers are build with static jli.  Now the original problem is handled differently, i.e., use newer objcopy(1) from devel/binutils.  Ideally, the rtld problem should be fixed but we cannot do anything here.
Comment 12 Konstantin Belousov freebsd_committer freebsd_triage 2015-02-27 09:35:41 UTC
(In reply to Jung-uk Kim from comment #9)
The patch looks good, although I am curious how the symlink survived the whole path from the execve to obj->path.  Could you build your rtld as

(cd libexec/rtld-elf && make DEBUG_FLAGS=-g DEBUG=-DDDEBUG clean all install)

and show the LD_DEBUG=1 output, in particular, AT_EXECPATH line ?
In fact, I wonder if this should be realpath(3) instead of readlink(2).

From what I read above, it is one the shells which tries to track the symlinks in the PWD ?  Anyway.
Comment 13 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-27 17:06:01 UTC
(In reply to Konstantin Belousov from comment #12)
% ls
build.sh	func.c		main.c		rtld.h
% cat build.sh
#!/bin/sh
mkdir -p bin lib obj
cc -c -fPIC -o obj/func.o func.c
cc -shared -o lib/librtld.so obj/func.o
cc -c -o obj/main.o main.c
cc -o bin/rtld -Wl,-rpath='$ORIGIN/../lib' -Wl,-z,origin -Llib -lrtld obj/main.o
ln -fs $(realpath $(pwd))/bin/rtld link
echo -n "Test 1: "
bin/rtld
echo -n "Test 2: "
./link
echo -n "Test 3: "
$(realpath $(pwd))/link
echo "rtld trace:"
env LD_DEBUG=1 $(realpath $(pwd))/link
rm -rf bin lib link obj
% cat func.c
#include <stdio.h>
#include "rtld.h"

void
func(void)
{

	printf("%s called\n", __func__);
}
% cat main.c
#include "rtld.h"

int
main(void)
{

	func();
	return (0);
}
% cat rtld.h 
void	func(void);
% sh build.sh
Test 1: func called
Test 2: func called
Test 3: Shared object "librtld.so" not found, required by "link"
rtld trace:
/libexec/ld-elf.so.1 is initialized, base address = 0x800600000
RTLD dynamic = 0x80081e908
RTLD pltgot  = 0
initializing thread locks
_rtld_thread_init: done
processing main program's program header
note osrel 1100062
note crt_no_init
AT_EXECPATH 0x7fffffffefc7 /usr/home/jkim/rtld/link
obj_main path /usr/home/jkim/rtld/link
Filling in DT_DEBUG entry
/usr/home/jkim/rtld/link valid_hash_sysv 1 valid_hash_gnu 1 dynsymcount 11
lm_init("(null)")
loading LD_PRELOAD libraries
loading needed objects
 Searching for "librtld.so"
lm_find("(null)", "/usr/home/jkim/rtld/../lib")
lmp_find("$DEFAULT$")
  Trying "/usr/home/jkim/rtld/../lib/librtld.so"
search_library_pathfds('librtld.so', '(null)', fdp)
lm_find("(null)", "/lib")
lmp_find("$DEFAULT$")
  Trying "/lib/librtld.so"
lm_find("(null)", "/usr/lib")
lmp_find("$DEFAULT$")
  Trying "/usr/lib/librtld.so"
lm_find("(null)", "/usr/lib/compat")
lmp_find("$DEFAULT$")
  Trying "/usr/lib/compat/librtld.so"
lm_find("(null)", "/usr/local/lib")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/librtld.so"
lm_find("(null)", "/usr/local/lib/gcc48")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/gcc48/librtld.so"
lm_find("(null)", "/usr/local/lib/graphviz")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/graphviz/librtld.so"
lm_find("(null)", "/usr/local/lib/libxul")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/libxul/librtld.so"
lm_find("(null)", "/usr/local/lib/nss")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/nss/librtld.so"
lm_find("(null)", "/usr/local/lib/opencollada")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/opencollada/librtld.so"
lm_find("(null)", "/usr/local/lib/qt4")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/qt4/librtld.so"
lm_find("(null)", "/usr/local/lib/virtualbox")
lmp_find("$DEFAULT$")
  Trying "/usr/local/lib/virtualbox/librtld.so"
lm_find("(null)", "/usr/local/llvm34/lib")
lmp_find("$DEFAULT$")
  Trying "/usr/local/llvm34/lib/librtld.so"
lm_find("(null)", "/usr/local/llvm35/lib")
lmp_find("$DEFAULT$")
  Trying "/usr/local/llvm35/lib/librtld.so"
lm_find("(null)", "/lib")
lmp_find("$DEFAULT$")
  Trying "/lib/librtld.so"
lm_find("(null)", "/usr/lib")
lmp_find("$DEFAULT$")
  Trying "/usr/lib/librtld.so"
Shared object "librtld.so" not found, required by "link"
Comment 14 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-27 17:24:39 UTC
(In reply to Konstantin Belousov from comment #12)
> In fact, I wonder if this should be realpath(3) instead of readlink(2).

Yes, I agree that realpath(3) is more desirable here.  Unfortunately, it becomes little slower because it calls series of lstat(2) and readlink(2), though.
Comment 15 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-27 17:26:51 UTC
Created attachment 153588 [details]
A revised patch to use realpath(3) instead of readlink(2).
Comment 16 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-27 17:44:27 UTC
Created attachment 153589 [details]
An optimized patch, i.e., realpath(3) returns an absolute path.
Comment 17 Konstantin Belousov freebsd_committer freebsd_triage 2015-02-27 18:22:51 UTC
(In reply to Jung-uk Kim from comment #16)
Your patch is fine.

I could only suggest add an dbg() line into rtld_dirname_abs() to log old and resolved path.

Thank you for finding and fixing this.
Comment 18 Jung-uk Kim freebsd_committer freebsd_triage 2015-02-27 19:11:52 UTC
(In reply to Konstantin Belousov from comment #17)
The rtld patch committed:

https://svnweb.freebsd.org/changeset/base/279364

We may be able to remove the workaround when all currently supported releases reach their EOLs, expected two years from now. ;-)

Thanks for the review!