Bug 251185 - graphics/mozjpeg conflicts with graphics/jpeg-turbo
Summary: graphics/mozjpeg conflicts with graphics/jpeg-turbo
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Po-Chuan Hsieh
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2020-11-16 14:16 UTC by Keren Sky
Modified: 2022-06-03 13:30 UTC (History)
7 users (show)

See Also:
bugzilla: maintainer-feedback? (sunpoet)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Keren Sky 2020-11-16 14:16:33 UTC
Hi,

This port fails to install when upgrading to the latest version. The error message indicates a conflict w/ graphics/jpeg-turbo. 

Is it no longer possible to have the conflict resolved like so: https://www.freshports.org/commit.php?category=graphics&port=mozjpeg&files=yes&message_id=201705131459.v4DExAq8088390@repo.freebsd.org

Best regards,
Comment 1 Keren Sky 2020-11-16 14:19:38 UTC
I forgot to copy and paste the exact terminal output:
===>  Installing for mozjpeg-4.0.0
===>  Checking if mozjpeg is already installed
===>   Registering installation for mozjpeg-4.0.0
Installing mozjpeg-4.0.0...
pkg-static: mozjpeg-4.0.0 conflicts with jpeg-turbo-2.0.5 (installs files into the same place).  Problematic file: /usr/local/bin/cjpeg
*** Error code 70
Comment 2 Dani I. 2020-11-16 14:45:54 UTC
Same problem here. graphics/jpeg-turbo is needed as dependency by multiple other pkgs, so there should be a solution to either excloude some stuff from mozjpeg or install in a different place.
Comment 3 Dani I. 2020-11-20 16:16:14 UTC
Bump..
Comment 4 buganini 2020-11-24 14:51:52 UTC
BTW

With both mozjpeg-3.3.1 and jpeg-turbo installed, I have
246:-ljpeg.8 => /usr/local/lib/libjpeg.so.8
439:-ljpeg.8 => /usr/local/lib/mozjpeg/libjpeg.so.8
from `ldconfig -r`

USE_LDCONFIG is removed in 4.0.0
https://svnweb.freebsd.org/ports/head/graphics/mozjpeg/Makefile?r1=555266&r2=555265&pathrev=555266


May we have some symbolic links like /usr/local/lib/libjpeg-mozjpeg.so.8 => /usr/local/lib/mozjpeg/libjpeg.so.8
to make it easier to create a py-pillow-mozjpeg slave port.
Comment 5 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-11-24 20:00:31 UTC
Upstream changed to CMake which does not have --program-prefix.
But why do you need both ports at the same time?

(In reply to buganini from comment #4)
I'll add USE_LDCONFIG back. Thanks!
Comment 6 SolarCatcher 2020-12-17 11:37:21 UTC
As others pointed out already: quite a few other ports directly or indiectly depend on jpeg-turbo. I cannot upgrade mozjpeg to 4.0.0 because pkg then wants to uninstall quite a few other packages on my servers:
ImageMagick7-nox11: 7.0.10.24_1
ghostscript9-agpl-base: 9.52_11
jpeg-turbo: 2.0.6
lcms2: 2.11_1
libgd: 2.3.0,1
libraw: 0.19.5
libwmf-nox11: 0.2.8.4_15
openjpeg: 2.3.1
php74-gd: 7.4.13_2
php74-pecl-imagick-im7: 3.4.4_2
phpMyAdmin5-php74: 5.0.4
tiff: 4.1.0
webp: 1.1.0

I would really need the old program-prefix back. If CMAKE does not offer this option, we should build a port mozjpeg3 keeping version 3 available, which seems to work for many of us.
Comment 7 Daniel Dowse 2021-03-28 06:49:57 UTC
Hello,

i have made some patches they can be pulled from this github repo 

https://github.com/ddowse/mozjpeg

Please check and review if the patches can be used `as is` or not. 

Greetings.
Comment 8 SolarCatcher 2021-03-28 10:52:57 UTC
Great! Many thanks! I just built this Port on 13.0-RC2 and it does not conflict with jpeg-turbo. I was able to losslessly compress a test.jpg with

/usr/local/bin/mozjpeg-jpegtran -optimize -copy none -outfile test_optimized.jpg test.jpg

Could others confirm that these patches work for their use cases? Then we could ask the maintainer to accept the patches and have a new working mozjpeg which does not conflict with jpeg-turbo!
Comment 9 Keren Sky 2021-04-12 15:42:01 UTC
I'm not that familiar with Cmake. Does this offer possibility of renaming binaries?
https://cmake.org/cmake/help/latest/command/set_target_properties.html

If not, apologies for the noise,
Comment 10 Morgan Davis 2021-04-30 01:40:45 UTC
What is the procedure for requesting fixes/changes for the prebuilt packages so we can have both mozjpeg and webp tools installed on a single machine together without this cjpeg conflict? Since these two formats are important for modern web sites, not having these tools be able to coexist is creating some toolchain headaches.  If the package maintainer(s) can't resolve this, is there a work-around that doesn't require building patched versions from sources?
Comment 11 SolarCatcher 2021-04-30 07:17:18 UTC
I think it would help if more people could test the updated port by Daniel: https://github.com/ddowse/mozjpeg . Please report back whether everything works as expected or whether you found any problems.

Then we might be able to either convince the port maintainer or someone else to commit the changes. I am not familiar with the procedures, but I have sometimes seen changes merged into the ports tree with the comment "maintainer time-out", which I interpret as: If maintainer fails to react within a certain time span, the commit can be integrated anyway.

I had tried to contact sunpoet@ directly, but without success, so far. Just sent another mail to him... Let's hope this can get solved, soon. Until then, I build this port with poudriere. I use it in production since the beginning of April.
Comment 12 Morgan Davis 2021-04-30 08:06:05 UTC
In fact, that's what I ended up doing -- I built Daniel's port.  I rolled it out to six servers and processed over 315,000 jpgs (43GB in all) and it worked perfectly, coexisting with the webp package (and all the stuff that comes with it). Based on this (and after previous use of the mozjpeg's cjpeg), this gets my approval. Thank you, Daniel!
Comment 13 Morgan Davis 2021-04-30 17:33:56 UTC
One side-effect of the patched mozjpeg package along with the webp package is that if you do a "pkg upgrade" you'll still be dealing with this:

Installed packages to be REINSTALLED:
        mozjpeg-4.0.3 (provided shared library changed)

Fetching jpeg-turbo-2.0.6.txz: 100%  345 KiB 352.9kB/s    00:01    
[1/3] Deinstalling jpeg-turbo-2.0.6...
[1/3] Deleting files for jpeg-turbo-2.0.6: 100%
[1/3] Installing jpeg-turbo-2.0.6...
[1/3] Extracting jpeg-turbo-2.0.6: 100%
[2/3] Reinstalling mozjpeg-4.0.3...
pkg: mozjpeg-4.0.3 conflicts with jpeg-turbo-2.0.6 (installs files into the same place).  Problematic file: /usr/local/bin/cjpeg

I haven't been able to pin down the fallout from this, but I did get into a state where doing the above caused me to lose both webp and the custom (patch) mozjpeg, leaving me with neither. I had to manually "pkg install webp" to get it back and then do the "pkg add /path/to/patched/mozjpeg.txz" to restore everything back to working order.  But if I do another "pkg upgrade" I get this:


Installed packages to be REINSTALLED:
        mozjpeg-4.0.3 (provided shared library changed)


(repeating these steps is just an infinite loop of removals and reinstalls)

This is a pain, and if you forget this you might find automated image management tools that depended on their existence to become rather upset.

Seems that even with the patched version, they are still sharing references to a library or conflicting executable.  Not sure if Daniel's patch is completely isolating mozjpeg as one would hope.
Comment 14 Daniel Dowse 2021-04-30 18:10:59 UTC
ACK, i will try to look into it in the coming week but can't promise anything

-cheers
Comment 15 SolarCatcher 2021-05-02 10:48:35 UTC
@Morgan I don't think what you see is a problem with the newly-patched mozjpeg package. What happens is that you upgrade your packges and the custom built mozjpeg is overwritten with the version from the standard repo, which cannot co-exist with jpeg-turbo.

You can save yourself a lot of trouble by locking the package after install: 
pkg lock mozjpeg

Then, it cannot be overwritten with the incompatible version. You just have to remember that you have to unlock it when you want to upgrade that package (pkg unlock mozjpeg).

I have the newly-patched mozjpeg package running on several servers and experience not problem with jpeg-turbo. I even forced a re-install of jpeg-turbo without any problems.
Comment 16 Morgan Davis 2021-05-03 17:23:32 UTC
@SolarCatcher Great advice/workaround.  Implemented the lock and all is well, even forcing a re-install of webp (jpeg-turbo) works as you said. Many thanks to you and Daniel.
Comment 17 Po-Chuan Hsieh freebsd_committer freebsd_triage 2021-07-13 16:50:00 UTC
(In reply to Po-Chuan Hsieh from comment #5)

I still do not understand why would you need both ports.
It seems the only reason is that jpeg-turbo automatically being installed by the ports framework (USES=jpeg).

mozjpeg can be used as jpeg-turbo by setting JPEG_PORT=graphics/mozjpeg in /etc/make.conf, See bug #257028.

Furthermore, I do not want to maintain a local patch.
It would be better to submit it upstream.

Thanks.
Comment 18 SolarCatcher 2021-07-15 10:57:46 UTC
(In reply to Po-Chuan Hsieh from comment #17)

I need mozjpeg solely to optimize the size of JPEGs. It is THE recommended library to losslessly reduce the size of JPEGs (used by many other tools such as Google's Sqoosh, https://squoosh.app/). Therefore it is installed on all my web servers - alongside zopflipng and gifsicle to reduce PNGs and GIFs.

Until v3 it was possible to generally use the packages from the official repo which - by default - depend on jpeg-turbo AND to install mozjpeg for the image optimziation. This is no longer possible.

Can anyone say if the changes proposed by Daniel could be adapted so that they could be upstreamed?