Bug 203327 - graphics/openjpeg: latest changes (from 2.1.0_1 to 2.1.1) break building of mupdf (and possibly other ports)
Summary: graphics/openjpeg: latest changes (from 2.1.0_1 to 2.1.1) break building of m...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Sunpoet Po-Chuan Hsieh
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-25 09:18 UTC by dennis.noordsij
Modified: 2015-10-09 18:33 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (sunpoet)


Attachments
patch-load-jpx.c (552 bytes, patch)
2015-09-27 08:28 UTC, Walter Schwarzenfeld
no flags Details | Diff
Simplify last change (2.78 KB, patch)
2015-10-06 15:48 UTC, wjenkner
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description dennis.noordsij 2015-09-25 09:18:29 UTC
One of the recent changes to graphics/openjpeg causes graphics/mupdf to no longer build, with the following errors:

<.. snip ..>
cc  -fstack-protector -o build/release/mujstest build/release/platform/x11/jstest_main.o build/release/platform/x11/pdfapp.o build/release/libmupdf.a -L/usr/local/lib  -lm -L/usr/local/lib -lfreetype   -ljbig2dec -ljpeg -L/usr/local/lib -lopenjp2    -lz
build/release/libmupdf.a(load-jpx.o): In function `fz_load_jpx':
source/fitz/load-jpx.c:(.text+0x56): undefined reference to `opj_set_default_decoder_parameters'
source/fitz/load-jpx.c:(.text+0x69): undefined reference to `opj_create_decompress'
source/fitz/load-jpx.c:(.text+0x84): undefined reference to `opj_set_info_handler'
source/fitz/load-jpx.c:(.text+0x94): undefined reference to `opj_set_warning_handler'
source/fitz/load-jpx.c:(.text+0xa4): undefined reference to `opj_set_error_handler'
source/fitz/load-jpx.c:(.text+0xb1): undefined reference to `opj_setup_decoder'
source/fitz/load-jpx.c:(.text+0xc3): undefined reference to `opj_stream_default_create'
source/fitz/load-jpx.c:(.text+0xe5): undefined reference to `opj_stream_set_read_function'
source/fitz/load-jpx.c:(.text+0xf2): undefined reference to `opj_stream_set_skip_function'
source/fitz/load-jpx.c:(.text+0xff): undefined reference to `opj_stream_set_seek_function'
source/fitz/load-jpx.c:(.text+0x10e): undefined reference to `opj_stream_set_user_data'
source/fitz/load-jpx.c:(.text+0x119): undefined reference to `opj_stream_set_user_data_length'
source/fitz/load-jpx.c:(.text+0x129): undefined reference to `opj_read_header'
source/fitz/load-jpx.c:(.text+0x141): undefined reference to `opj_decode'
source/fitz/load-jpx.c:(.text+0x14b): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x153): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x42d): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x4e1): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x4f8): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x500): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x514): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x534): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x548): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x55c): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x570): undefined reference to `opj_image_destroy'
build/release/libmupdf.a(load-jpx.o):source/fitz/load-jpx.c:(.text+0x593): more undefined references to `opj_image_destroy' follow
build/release/libmupdf.a(load-jpx.o)build/release/libmupdf.a(load-jpx.o): In function `fz_load_jpx':
source/fitz/load-jpx.c:(.text+0x56): undefined reference to `opj_set_default_decoder_parameters'
source/fitz/load-jpx.c:(.text+0x69): undefined reference to `opj_create_decompress'
source/fitz/load-jpx.c:(.text+0x84): undefined reference to `opj_set_info_handler'
source/fitz/load-jpx.c:(.text+0x94): undefined reference to `opj_set_warning_handler'
source/fitz/load-jpx.c:(.text+0xa4): undefined reference to `opj_set_error_handler'
source/fitz/load-jpx.c:(.text+0xb1): undefined reference to `opj_setup_decoder'
source/fitz/load-jpx.c:(.text+0xc3): undefined reference to `opj_stream_default_create'
source/fitz/load-jpx.c:(.text+0xe5): undefined reference to `opj_stream_set_read_function'
source/fitz/load-jpx.c:(.text+0xf2): undefined reference to `opj_stream_set_skip_function'
source/fitz/load-jpx.c:(.text+0x: In function `fz_load_jpx':
source/fitz/load-jpx.c:(.text+0x56): undefined reference to `opj_set_default_decoder_parameters'
source/fitz/load-jpx.c:(.text+0x69): undefined reference to `opj_create_decompress'
source/fitz/load-jpx.c:(.text+0x84): undefined reference to `opj_set_info_handler'
source/fitz/load-jpx.c:(.text+0x94): undefined reference to `opj_set_warning_handler'
source/fitz/load-jpx.c:(.text+0xa4): undefined reference to `opj_set_error_handler'
source/fitz/load-jpx.c:(.text+0xb1): undefined reference to `opj_setup_decoder'
source/fitz/load-jpx.c:(.text+0xc3): undefined reference to `opj_stream_default_create'
source/fitz/load-jpx.c:(.text+0xe5): undefined reference to `opj_stream_set_read_function'
source/fitz/load-jpx.c:(.text+0xf2): undefined reference to `opj_stream_set_skip_function'
source/fitz/load-jpx.c:(.text+0xff): undefined reference to `opj_stream_set_seek_function'
source/fitz/load-jpx.c:(.text+0x10e): undefined reference to `opj_stream_set_user_data'
source/fitz/load-jpx.c:(.text+0x119): undefined reference to `opj_stream_set_user_data_length'
source/fitz/load-jpx.c:(.text+0x129): undefined reference to `opj_read_header'
source/fitz/load-jpx.c:(.text+0x141): undefined reference to `opj_decode'
source/fitz/load-jpx.c:(.text+0x14b): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x153): undefined reference to `opj_destroy_codec'ff
): undsource/fitz/load-jpx.c:e(f.texti+n0exd42d )r:e fuenrednecfei nteod  `refopj_stream_set_seek_functione'r
encesource/fitz/load-jpx.c: (t.texto+ 0`x10eopj_image_destroy)':
 undefinsource/fitz/load-jpx.c:e(d.text +r0x4e1): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x4f8): undefined reference to `opj_stream_destroy'
eference to `opj_stream_set_user_data'
source/fitz/load-jpx.c:(.text+0x119): undefined reference to `opj_stream_set_user_data_length'
source/fitz/load-jpx.c:(.text+0x129): undefined reference to `opj_read_header'
source/fitz/load-jpx.c:(.text+0x141): undefined reference to `opj_decode'
source/fitz/load-jpx.c:(.text+0x14b): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x153): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x42d): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x4e1): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x4f8): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x500): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x514): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x534): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x548): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x55c): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x570): undefined reference to `opj_image_destroy'
build/release/libmupdf.a(load-jpx.o):source/fitz/load-jpx.c:(.text+0x593): more undefined references to `opj_image_destroy' follow
source/fitz/load-jpx.c:(.text+0x500): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x514): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x534): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x548): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x55c): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x570): undefined reference to `opj_image_destroy'
build/release/libmupdf.a(load-jpx.o):source/fitz/load-jpx.c:(.text+0x593): more undefined references to `opj_image_destroy' follow
/usr/bin/ld: build/release/mutool: hidden symbol `opj_setup_decoder' isn't defined
/usr/bin/ld: final link failed: Nonrepresentable section on output
/usr/bin/ld: build/release/mujstest: hidden symbol `opj_setup_decoder' isn't defined
/usr/bin/ld: final link failed: Nonrepresentable section on output
/usr/bin/ld: build/release/mudraw: hidden symbol `opj_setup_decoder' isn't defined
/usr/bin/ld: final link failed: Nonrepresentable section on output
cc: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:248: recipe for target 'build/release/mutool' failed
gmake[2]: *** [build/release/mutool] Error 1
gmake[2]: *** Waiting for unfinished jobs....
cc: error: linker command failed with exit code 1 (use -v to see invocation)
cc: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:260: recipe for target 'build/release/mujstest' failed
gmake[2]: *** [build/release/mujstest] Error 1
Makefile:241: recipe for target 'build/release/mudraw' failed
gmake[2]: *** [build/release/mudraw] Error 1
build/release/libmupdf.a(load-jpx.o): In function `fz_load_jpx':
source/fitz/load-jpx.c:(.text+0x56): undefined reference to `opj_set_default_decoder_parameters'
source/fitz/load-jpx.c:(.text+0x69): undefined reference to `opj_create_decompress'
source/fitz/load-jpx.c:(.text+0x84): undefined reference to `opj_set_info_handler'
source/fitz/load-jpx.c:(.text+0x94): undefined reference to `opj_set_warning_handler'
source/fitz/load-jpx.c:(.text+0xa4): undefined reference to `opj_set_error_handler'
source/fitz/load-jpx.c:(.text+0xb1): undefined reference to `opj_setup_decoder'
source/fitz/load-jpx.c:(.text+0xc3): undefined reference to `opj_stream_default_create'
source/fitz/load-jpx.c:(.text+0xe5): undefined reference to `opj_stream_set_read_function'
source/fitz/load-jpx.c:(.text+0xf2): undefined reference to `opj_stream_set_skip_function'
source/fitz/load-jpx.c:(.text+0xff): undefined reference to `opj_stream_set_seek_function'
source/fitz/load-jpx.c:(.text+0x10e): undefined reference to `opj_stream_set_user_data'
source/fitz/load-jpx.c:(.text+0x119): undefined reference to `opj_stream_set_user_data_length'
source/fitz/load-jpx.c:(.text+0x129): undefined reference to `opj_read_header'
source/fitz/load-jpx.c:(.text+0x141): undefined reference to `opj_decode'
source/fitz/load-jpx.c:(.text+0x14b): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x153): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x42d): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x4e1): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x4f8): undefined reference to `opj_stream_destroy'
source/fitz/load-jpx.c:(.text+0x500): undefined reference to `opj_destroy_codec'
source/fitz/load-jpx.c:(.text+0x514): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x534): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x548): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x55c): undefined reference to `opj_image_destroy'
source/fitz/load-jpx.c:(.text+0x570): undefined reference to `opj_image_destroy'
build/release/libmupdf.a(load-jpx.o):source/fitz/load-jpx.c:(.text+0x593): more undefined references to `opj_image_destroy' follow
/usr/bin/ld: build/release/mupdf-x11: hidden symbol `opj_setup_decoder' isn't defined
/usr/bin/ld: final link failed: Nonrepresentable section on output
cc: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:268: recipe for target 'build/release/mupdf-x11' failed
gmake[2]: *** [build/release/mupdf-x11] Error 1
gmake[2]: Leaving directory '/usr/ports/graphics/mupdf/work/mupdf-1.7-source'
===> Compilation failed unexpectedly.



If I install the openjpeg binary pkg version (2.1.0_1) mupdf builds fine. If I install openjpeg from ports (version 2.1.1), the problem happens. (Tried in a poudriere box, jails, and vanilla FreeBSD, all on up to date 10.2-RELEASE).

I don't really know where to look for the problem, usually undefined reference means the library is not installed or can not be found, but that is not the case here. 


(Portsnap up to date as of  Fri Sep 25 11:13:39 CEST 2015)


Happy to provide more information
Comment 1 dennis.noordsij 2015-09-25 21:04:31 UTC
Using git bisect the problem was introduced in:

  commit 3ed5858902055d3500a6ab183f1395686921d026
  Date:   Sun Jan 25 16:59:39 2015 +0000  

  [trunk] use hidden visibility for static library (fixes issue 354)


which references the following issue:
  https://github.com/uclouvain/openjpeg/issues/354


mupdf defines OPJ_STATIC, compiles static libraries libmupdf.a which contain openjp2, etc, and then links these into binaries (mutool, etc).

The same problem happens on a Linux distribution with the gcc toolchain.

I don't know what the correct place is to fix this.

Workarounds:
 - remove the visibility ("hidden") in include/openjp2/openjpeg.h  (basically partially undo the above commit)
 - (untested) do not define OPJ_STATIC in mupdf (but according to comments in load-jpx.c this breaks windows compilation)
 - do not remove the thirdparty/ folder in the mupdf sources and let it statically link against those instead of the system libraries (mupdf already needs a patch to be able to use the system openjpeg)
 - ifdef hacks


The easiest workaround for others who run into this:

--- usr/ports/graphics/mupdf/Makefile.old 2015-09-25 22:58:18.963166563 +0200
+++ usr/ports/graphics/mupdf/Makefile     2015-09-25 22:58:25.875451670 +0200
@@ -51,7 +51,6 @@
                ${WRKSRC}/Makerules
        @${REINPLACE_CMD} '/^CFLAGS/s|$$| -I${LOCALBASE}/include|' \
                ${WRKSRC}/Makefile
-       @${RM} -r ${WRKSRC}/thirdparty/*
 
 post-build:
        (cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} ${MAKE_CMD} ${MAKE_FLAGS} \
Comment 2 Walter Schwarzenfeld 2015-09-27 08:28:01 UTC
Created attachment 161454 [details]
patch-load-jpx.c
Comment 3 Walter Schwarzenfeld 2015-09-27 08:28:35 UTC
This patch should fix it.
Comment 4 Walter Schwarzenfeld 2015-09-27 08:33:29 UTC
May interesting: I the last time I found different errors in some ports, where the solution was the same (extern c). Something seems to went wrong (maybe, only with older ports).
Comment 5 Sunpoet Po-Chuan Hsieh freebsd_committer 2015-09-28 07:57:25 UTC
Add graphics/mupdf maintainer (udvzsolt@gmail.com) to CC List.
Comment 6 Zsolt Udvari 2015-09-28 08:40:32 UTC
If Walter's patch fix the build I'll accept. Now I don't have time to test.
Comment 7 wjenkner 2015-09-28 15:43:50 UTC
(In reply to dennis.noordsij from comment #1)
>  - (untested) do not define OPJ_STATIC in mupdf (but according to comments in load-jpx.c this breaks windows compilation)

Does it make sense at all to define OPJ_STATIC in load-jpx.c when linking with a libopenjp2.so which wasn't actually compiled with the kind of symbol visibility implied by that switch (by default, mupdf compiles its own private static versions of this and other libraries, see Makethird)?

If not, your suggestion above might be the best solution.
Comment 8 commit-hook freebsd_committer 2015-10-04 18:02:47 UTC
A commit references this bug:

Author: sunpoet
Date: Sun Oct  4 18:01:54 UTC 2015
New revision: 398615
URL: https://svnweb.freebsd.org/changeset/ports/398615

Log:
  - Fix build with openjpeg 2.1.1
  - Bump PORTREVISION
  - While I'm here:
    - Update MASTER_SITES: make 1.7 fetchable
    - Use DISTVERSIONSUFFIX instead of EXTRACT_SUFX
    - Convert to new options helper

  PR:		203327
  Submitted by:	Walter Schwarzenfeld <w.litter@aon.at>
  Approved by:	Zsolt Udvari <udvzsolt@gmail.com> (maintainer)

Changes:
  head/graphics/mupdf/Makefile
  head/graphics/mupdf/files/patch-source__fitz__load-jpx.c
Comment 9 Sunpoet Po-Chuan Hsieh freebsd_committer 2015-10-04 18:04:40 UTC
Committed. Thanks!
Comment 10 wjenkner 2015-10-05 14:12:56 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #9)
As I tried to convey in my comment above, I think this fix is wrong; it happens to work because load-jpx.c is compiled as a .c file by plain cc (not c++) and so the #ifdef __cplusplus ...#endif is just a fancy way to comment out the stuff in between.  IIUC, the link error happens because of symbol visibility, not because of symbol mangling (neither the symbols in load-jpx.o nor in libopenjp2.so are mangled).
Comment 11 Walter Schwarzenfeld 2015-10-05 16:35:49 UTC
o..O sorry, I make nonsense and nearly nobody see it.
The wrong header is included in load-jpx.c. Change it to #include <openjpeg-2.1/openjpeg.h>.
And the there is another error:
source/fitz/load-jpx.c:120:38: error: too few arguments to function call, expected 3, have 2
        opj_stream_set_user_data(stream, &sb);

I am not sure, but I think the line 120 should be:
opj_stream_set_user_data(stream, sb.data, &sb);
Comment 12 Walter Schwarzenfeld 2015-10-05 16:41:47 UTC
Thank you @wjenkner
Comment 13 Walter Schwarzenfeld 2015-10-05 17:32:52 UTC
I correct the line 120 should be
opj_stream_set_user_data(stream, &sb, NULL);
Comment 14 dennis.noordsij 2015-10-05 17:35:21 UTC
(In reply to Walter Schwarzenfeld from comment #11)

These 2 differences were already handled by files/patch-source__fitz__load-jpx.c of graphics/mupdf, and also in the new patch? Where do you run into problems?

What is the consensus on the proper fix? I agree with the reasoning of #10.
Comment 15 Walter Schwarzenfeld 2015-10-05 18:34:18 UTC
You are  right, the change of the include is in the patch.
Comment 16 wjenkner 2015-10-06 15:43:43 UTC
(In reply to dennis.noordsij from comment #14)
I think deleting the OPJ_STATIC definition in load-lpx.c is fine.

Effectively, this change is a no-op, but the point is to supply a (hopefully) correct explanation of the issue, see the attached patch.
Comment 17 wjenkner 2015-10-06 15:48:26 UTC
Created attachment 161766 [details]
Simplify last change
Comment 18 Walter Schwarzenfeld 2015-10-06 16:04:57 UTC
You forget the change I posted above in line 120 - in your attachment l think it is line 117.
Comment 19 commit-hook freebsd_committer 2015-10-09 18:33:40 UTC
A commit references this bug:

Author: sunpoet
Date: Fri Oct  9 18:33:09 UTC 2015
New revision: 398949
URL: https://svnweb.freebsd.org/changeset/ports/398949

Log:
  - Fix patch

  PR:		203327
  Submitted by:	Wolfgang Jenkner <wjenkner@inode.at>
  Approved by:	Zsolt Udvari <udvzsolt@gmail.com> (maintainer, implicit)

Changes:
  head/graphics/mupdf/files/patch-source__fitz__load-jpx.c