Comment 1 Kubilay Kocak 2019-04-21 04:51:11 UTC

Thank you Kai

Given backward incompatibilities, could you review dependent ports/packages setup.py:*_requires to evaluate their compatibility. (a list of OK/NOT OK should be fine)

Comment 2 Kai Knoblich 2019-04-21 17:03:00 UTC

No problem, here comes the list:

astro/py-astLib: pillow>=0
audio/decibel-audio-player: pillow>0
cad/k40-whisperer: pillow>0
comms/apitran: pillow>0
databases/py-mongoengine: pillow>=2.0.0
deskutils/calibre: pillow>=0
deskutils/gourmet: pillow>0
deskutils/griffith: pillow>=4.3
devel/py-asciimatics: pillow>=2.7.0
devel/py-mwlib: pillow>0
devel/py-pydenticon: pillow>=0
games/fretsonfire: pillow>=0
games/hypatia_engine: pillow>0
games/openage: pillow>0
games/py-mnemosyne: pillow>0
games/pysolfc: pillow>0
games/pythonsudoku: pillow>=0
graphics/caffe: pillow>=1.7 (>= 2.3.0 / Makefile should be updated to reflect the requirements)
graphics/imgtops: pillow>0
graphics/imgv: pillow>0
graphics/impressive: pillow>0
graphics/py-PyOpenGL: pillow>0
graphics/py-actdiag: pillow>=2.2.1
graphics/py-blockdiag: pillow>=2.2.1
graphics/py-django-easy-thumbnails: pillow>0
graphics/py-imageio: pillow>=0
graphics/py-imageio24: pillow>0
graphics/py-mcomix: pillow>0    (>= 1.15 / Makefile should be updated)
graphics/py-nwdiag: pillow>=2.2.1
graphics/py-photocollage: pillow>0
graphics/py-pyinsane2: pillow>0
graphics/py-pyocr: pillow>0
graphics/py-pypillowfight: pillow>0
graphics/py-sane: pillow>0
graphics/py-scikit-image: pillow>=1.7 (>= 4.3.0 / Makefile should be updated)
graphics/py-seqdiag: pillow>=2.2.1
graphics/py-sorl-thumbnail: pillow>0
graphics/py-soya3d: pillow>0
graphics/py-willow: pillow>=0
graphics/sk1: pillow>0
graphics/variety: pillow>=2.6.0
lang/mono: pillow>=0
math/asymptote: pillow>0
math/py-PyWavelets: pillow>=1.7 (>= 3.0.0 / Makefile should be updated)
math/py-plastex: pillow>=0
misc/cs: pillow>0
misc/ironscanner: pillow>0
misc/pylize: pillow>0
misc/wotsap: pillow>0   (>= 1.1.4 / Makefile should be updated)
multimedia/freevo: pillow>0
multimedia/openshot: pillow>=0
net-im/py-matrix-synapse: pillow>=3.1.2
net-mgmt/netbox: pillow>=5.3.0
net/py-rainbowstream: pillow>0
net/tucan: pillow>0
net-p2p/deluge-cli: pillow>0
net-p2p/tribler: pillow>0
print/hplip: pillow>0
print/pkpgcounter: pillow>0
print/py-reportlab: pillow>0 (>= 4.0.0 / Makefile should be updated)
print/py-trml2pdf: pillow>0
science/gramps: pillow>0
science/psychopy: pillow>0
science/rdkit: pillow>0
security/py-volatility: pillow>0
security/vinetto: pillow>0 (>= 1.1.5 / Makefile should be updated)
textproc/py-qrcode: pillow>=2.6
textproc/py-xhtml2pdf: pillow>=0
www/google-appengine: pillow>0
www/go-appengine-sdk: pillow>0
www/py-Products.CMFPlone: pillow>0
www/py-django-filer: pillow>=2.3.0
www/py-django-markdownx: pillow>0
www/py-django-mezzanine: pillow>0
www/py-django-photologue: pillow>=4.3.0
www/py-django-simple-captcha: pillow>=2.2.2
www/py-instabot: pillow>=0
www/py-pywikibot: pillow>0
www/py-weboob: pillow>=0
www/seahub: pillow>=0 (== 4.3.0 / Makefile should be updated)
www/twms: pillow>0
www/web2ldap: pillow>0
www/xist: pillow>=2.6
x11-toolkits/py-kivy: pillow>0
x11/cinnamon: pillow>0
x11/py-pyvirtualdisplay: pillow>0
x11/xcurs: pillow>0
x11/xpra: pillow>=3.0

As far I can tell everything looks ok so far. One exception is www/seahub where the dependency for graphics/py-pillow is pinned to 4.3.0 in the requirements.txt.

Comment 3 Charlie Li 2019-04-21 23:27:53 UTC

(In reply to Kai Knoblich from comment #2)
> x11/cinnamon: pillow>0

When it comes to released versions, Cinnamon breaks at runtime. Specifically in cinnamon-settings, a long-deprecated syntax for checking PIL/Pillow version was found, for a long-irrelevant purpose. Since PIL is not only dead in favour of Pillow but also never supported Python 3 (which x11/cinnamon requires), upstream removed the check. It only landed 16 days ago, after the latest release version, so I will pull in that change to the GNOME devel repo where Cinnamon 4.0 continues to incubate (and I continue to dogfood).

https://github.com/linuxmint/cinnamon/commit/c843f3664064742e2672e0fea528571a882d84ad

Comment 4 Charlie Li 2019-04-22 00:12:18 UTC

(In reply to Charlie Li from comment #3)

Fix applied in the GNOME devel repo.

https://github.com/freebsd/freebsd-ports-gnome/pull/33/commits/609cdb436d113ecd40bf1187d33f1f0b3256ff41

Comment 5 Kubilay Kocak 2019-04-23 04:40:28 UTC

Comment on attachment 203846 [details]
py-pillow-6.0.0.patch

Approved by: koobs (maintainer)

Comment 6 Kai Knoblich 2019-04-26 19:03:24 UTC

Somehow I missed the following port:

www/py-wagtail: >=4.0.0, <6.0 (taken from setup.py)

I've asked the maintainer if there's a way to relax the settings. Once that has been resolved the patch is ready to commit.


(In reply to Charlie Li from comment #3)

Thank you for your info and the fix in the GNOME repo.

Comment 7 Kai Knoblich 2019-05-12 13:05:18 UTC

The requirements for www/py-wagtail were relaxed in ports r500975. Meanwhile I did a deeper QA check by checking the sources for possible runtime failures.

There are some ports which we need to take care of:

- graphics/impressive (makes use of Image.VERSION in impressive.py)
- graphics/py-mcomix (makes use of Image.VERSION in run.py)
- graphics/py-sane (makes use of Image.VERSION and PIL.OleFileIO in several files, makes use of PIL 1.1.7 -> deprecate?)
- print/hplip (makes use of Image.VERSION in installer/dcheck.py)
- x11/cinnamon (makes use of Image.VERSION in files/usr/lib/cinnamon-settings/bin/imtools.py, already pointed out by Charlie Li)

Comment 8 commit-hook 2019-05-18 11:16:01 UTC

A commit references this bug:

Author: kai
Date: Sat May 18 11:15:01 UTC 2019
New revision: 501950
URL: https://svnweb.freebsd.org/changeset/ports/501950

Log:
  graphics/py-mcomix: Fix runtime with graphics/py-pillow 6.0

  * Prevent runtime breakage when graphics/py-pillow is updated to 6.0 where
    the deprecated constant "Image.VERSION" is no longer available. [1]

  * Add a workaround for the behavior of the "_getexif" function that is still
    buggy with PNG files that contain EXIF meta data. [2]

  Also while I'm here:
  * Add "gnome" to USES as using USE_GNOME alone is deprecated.

  PR:		237887, 237426 (related)
  Approved by:	ashish (maintainer)
  Obtained from:	upstream [1], https://github.com/multiSnow/mcomix3/issues/76 [2]

Changes:
  head/graphics/py-mcomix/Makefile
  head/graphics/py-mcomix/files/
  head/graphics/py-mcomix/files/patch-mcomix_image__tools.py
  head/graphics/py-mcomix/files/patch-mcomix_run.py

Comment 9 commit-hook 2019-05-22 06:40:50 UTC

A commit references this bug:

Author: kai
Date: Wed May 22 06:40:02 UTC 2019
New revision: 502255
URL: https://svnweb.freebsd.org/changeset/ports/502255

Log:
  graphics/impressive: Update to 0.12.0

  * Switch to MuPDF as default renderer for PDF files and give users the
    choice to choose alternate or additional PDF renderers (Ghostscript,
    Poppler and Xpdf)
  * Make further dependencies optional but leave them as default to keep in
    sync with upstream's defaults
  * Limit the port to Python 2.7 because Python 3 is not (yet) supported and
    add "python" to CATEGORIES to make searching somewhat easier
  * Introduce a patch to prevent possible runtime errors if graphics/py-pillow
    is updated to 6.0.0 [1] [2]
  * Pet portlint
  * Take maintainership

  Notable changes since 0.11.1:

  * Many zoom mode related changes
  * Many video related changes
  * Made Impressive suitable for Raspberry Pi-based digital signage
  * Fixed --auto mode that was broken in 0.11.2 (only worked properly for the
    first page)
  * Fixed excessive mouse sensitivity in full-screen mode
  * Fixed crashes when clicking hyperlinks in some PDFs
  * Fixed non-working hyperlinks when more than one input document is specified
  * Added new MuPDF renderer, uses no temporary files or named pipes on POSIX
    with MuPDF >= 1.4
  * Can now use MuPDF's mutool instead of pdftk to get basic file information
    and unpack PDF files (for hyperlink decoding)
    * pdftk is no longer required to open "modern" PDF files
    * Only page title extraction still needs pdftk
  * Input files are now checked for playability (to prevent users from
    accidentally playing .info files)
  * Fixed non-responsiveness to any controls on slow systems where rendering a
    page takes longer than the --auto interval
  * First page is now shown as early as possible (before fonts etc. have been
    preloaded)
  * Fixed occasional re-rendering of pages that have already been rendered
  * Fixed 'keys' PageProp that was broken since 0.11.0
  * Fixed crash in --verbose mode
  * Changed shebang line to 'python2' to avoid issues with Linux distributions
    that use Python 3.x by default

  http://impressive.sourceforge.net/news.php

  PR:		237426 (related) [1]
  Obtained from:	Upstream [2]

Changes:
  head/graphics/impressive/Makefile
  head/graphics/impressive/distinfo
  head/graphics/impressive/files/
  head/graphics/impressive/files/patch-impressive.py

Comment 10 Kai Knoblich 2019-05-22 08:20:14 UTC

Actual progress so far is:

- graphics/impressive -> Updated and fixed
- graphics/py-mcomix -> Fixed
- graphics/py-sane -> No actions required, doesn't make use of any removed functions.

Remaining ports:

print/hplip:
Still waiting for feedback from woodsb02@

x11/cinnamon:
Version in ports tree seems to be a bit stale and needs some tweaks to get it actually working because it depends on Python 2 and has some shebang issues (poudriere bulk -t always fails). Question here is, should we patch anyway or leave that port out?

Comment 11 Charlie Li 2019-05-22 16:11:53 UTC

(In reply to Kai Knoblich from comment #10)
> x11/cinnamon:
> Version in ports tree seems to be a bit stale and needs some tweaks to get
> it actually working because it depends on Python 2 and has some shebang
> issues (poudriere bulk -t always fails). Question here is, should we patch
> anyway or leave that port out?

Leave it out. It is getting updated to 4.0 anyway, and work in the gnome@ devel repo has the fix for this bug.

Comment 12 commit-hook 2019-05-29 11:42:07 UTC

A commit references this bug:

Author: koobs
Date: Wed May 29 11:42:03 UTC 2019
New revision: 502951
URL: https://svnweb.freebsd.org/changeset/ports/502951

Log:
  x11/cinnamon: Remove unnecessary version check

  Bug 237426 updates Pillow to 6.0.0, which removed the Image.VERSION
  variable, after a period of deprecation.

  Cinnamon currently uses this variable, so backport upstream commit
  c843f3664064742e2672e0fea528571a882d84ad [1] to compensate, so we don't
  need to wait for the Gnome teams Cinnamon 4 update, which has already
  pulled it in. Thanks to Charli Li for pointing this commit out.

  While I'm here, fix all stage-qa errors and ports/framework compliance
  issues:

  	- Add missing LIB_DEPENDS
  	- Add missing USE_{GNOME,X11} components
  	- Add USES=gnome, gl
  	- Fix gstreamer dependencies (GSTREAMER1, not GSTREAMER)
  	- Add LICENCE{_FILE}
  	- Fix more/all shebangs, remove custom REINPLACE_CMD section
  	- Remove GH_ACCOUNT (no longer necessary)

  [1] https://github.com/linuxmint/cinnamon/pull/8496
  [2] https://github.com/linuxmint/cinnamon/issues/8495

  PR:		238070, 237426
  Reported by:	kai
  Tested by:	Michal Kilijanek (via Twitter)
  Approved by:	kwm (gnome, maintainer)
  MFH:		2019Q2

Changes:
  head/x11/cinnamon/Makefile
  head/x11/cinnamon/distinfo
  head/x11/cinnamon/files/patch-files_usr_lib_cinnamon-settings_bin_imtools.py

Comment 13 commit-hook 2019-06-02 21:01:06 UTC

A commit references this bug:

Author: kai
Date: Sun Jun  2 21:00:07 UTC 2019
New revision: 503333
URL: https://svnweb.freebsd.org/changeset/ports/503333

Log:
  print/hplip: Fix runtime with graphics/py-pillow 6.0.0

  Fix a possible runtime error in installers/dcheck.py when graphics/py-pillow
  is updated to 6.0.0, which removed the previously deprecated constant
  Image.VERSION .

  PR:		238072, 237426 (related)
  Approved by:	woodsb02 (maintainer)

Changes:
  head/print/hplip/Makefile
  head/print/hplip/files/patch-installer_dcheck.py

Comment 14 Kai Knoblich 2019-06-03 09:34:53 UTC

Meanwhile two new ports were added that also depends on graphics/py-pillow:

x11-toolkits/py-easygui: Uses no backwards incompatible code, should be ok
x11/py-pyscreenshot: Makes uses of Image.VERSION in pyscreenshot/plugins/pil.py

Comment 15 Kai Knoblich 2019-06-03 18:49:18 UTC

No actions required for x11/py-pyscreenshot because the code in pyscreenshot/plugins/pil.py makes use of the ImageGrab feature that is only available for macOS/Windows.

So all things are set to bring Pillow 6.0.0 into the Ports tree once bug #238070 is closed.

Comment 16 Kubilay Kocak 2019-06-04 03:48:59 UTC

@Kai Bug 238070 has landed in head in ports r502951, and since Pillow 6.0.0 wont be merged to quarterly (no security fixes), it can be committed to head

However, I just noted the following in the release notes:

PIL.OleFileIO was removed as a vendored file and in Pillow 4.0.0 (2017-01) in favour of the upstream olefile Python package, and replaced with an ImportError. The deprecated file has now been removed from Pillow. If needed, install from PyPI (eg. pip install olefile).

The olefile module is no longer a required dependency when installing Pillow. Support for plugins requiring olefile will not be loaded if it is not installed. This allows library consumers to avoid installing this dependency if they choose. Some library consumers have little interest in the format support and would like to keep dependencies to a minimum.

We should add a RUN_DEPENDS on devel/py-olefile to maintain compatibility with existing installations/users that rely on that modules functionality, at least until it can be made an OPTION

Comment 17 Kai Knoblich 2019-06-04 20:28:34 UTC

Created attachment 204825 [details]
py-pillow-6.0.0-with-OLE-as-non-default-option.patch

Comment 18 Kai Knoblich 2019-06-04 20:29:28 UTC

Created attachment 204826 [details]
py-pillow-6.0.0-with-OLE-as-default-option.patch

Comment 19 Kai Knoblich 2019-06-04 20:49:09 UTC

@Kubilay: Alright, I had got used to the workflow here on BZ to process further once the dependent bug(s) is/are closed.

I created two patches each introducing a new option, OLE, that adds devel/py-olefile to RUN_DEPENDS.

The second one also sets the OLE option as default. I'll let you decide what the best approach would be.

QA:
~~~

Pillow 6.0.0 without devel/py-olefile:

> 1111 passed, 199 skipped

Pillow 6.0.0 with devel/py-olefile:

> 1112 passed, 198 skipped

I also did comparisons with Pillow 5.3.0 that is still in the Ports tree:

Pillow 5.3.0 without devel/py-olefile:

> 1 failed, 880 passed, 124 skipped

Pillow 5.3.0 with devel/py-olefile:

> 1 failed, 881 passed, 123 skipped

The overall tests failed for Pillow 5.3.0 because there are incompatibilities with print/freetype2 that was recently updated to 2.10 in ports r503401 .

To be absolutely sure I also tested both versions with the previous release of print/freetype2 -> 2.9.1:

Pillow 6.0.0 (with devel/py-olefile):

> 1112 passed, 198 skipped

Pillow 5.3.0 (with devel/py-olefile):

> 1042 passed, 198 skipped

Comment 20 Antoine Brodin 2019-06-04 21:03:33 UTC

I don't think olefile should be an option,  this is kind of a niche port that can easily break and depending on it for a widely used port like pillow is a idea.

Comment 21 Antoine Brodin 2019-06-04 21:04:07 UTC

(In reply to Antoine Brodin from comment #20)
a bad idea

Comment 22 Kubilay Kocak 2019-06-13 12:02:22 UTC

Comment on attachment 204826 [details]
py-pillow-6.0.0-with-OLE-as-default-option.patch

Patch approved, with one change request:

Expand MS -> Microsoft in OLE_DESC

Comment 23 commit-hook 2019-06-14 04:07:03 UTC

A commit references this bug:

Author: kai
Date: Fri Jun 14 04:06:17 UTC 2019
New revision: 504155
URL: https://svnweb.freebsd.org/changeset/ports/504155

Log:
  graphics/py-pillow: Update to 6.0.0

   * Introduce a new option OLE and add it to the default options to maintain
     compatibility for a while with existing installations/users that rely on
     the previously deprecated function PIL.OleFileIO which has been removed
     now.

   * Remove obsolete CONFLICTS_INSTALL entry because graphics/py-imaging no
     longer exists in the Ports tree since 2016.

  Release Notes:

   * https://pillow.readthedocs.io/en/latest/releasenotes/index.html

  Backward Incompatible Changes (6.0.0):

   * https://pillow.readthedocs.io/en/latest/releasenotes/6.0.0.html#backwards-incompatible-changes

  Detailed Changelog:

   * https://github.com/python-pillow/Pillow/blob/6.0.0/CHANGES.rst

  PR:		237426
  Reviewed by:	antoine, koobs, Charlie Li
  Approved by:	koobs (maintainer)

Changes:
  head/graphics/py-pillow/Makefile
  head/graphics/py-pillow/distinfo

Comment 24 commit-hook 2019-06-14 06:56:15 UTC

A commit references this bug:

Author: kai
Date: Fri Jun 14 06:55:59 UTC 2019
New revision: 504161
URL: https://svnweb.freebsd.org/changeset/ports/504161

Log:
  graphics/py-pillow: Retain the alphabetic order of options

  PR:		237426
  Reported by:	koobs

Changes:
  head/graphics/py-pillow/Makefile

Comment 25 Kubilay Kocak 2019-06-26 07:18:25 UTC

Good to close this?

Comment 26 Kai Knoblich 2019-06-26 07:33:50 UTC

(In reply to Kubilay Kocak from comment #25)

Yes, if bug 238070 that is still open doesn't prevent it, everything seems fine then.

Comment 27 Kubilay Kocak 2019-06-26 07:45:13 UTC

(In reply to Kai Knoblich from comment #26)

Ah yes, I'll sort that out, sorry for the noise and thank you for the reminder Kai

Comment 28 commit-hook 2019-07-01 06:18:31 UTC

A commit references this bug:

Author: koobs
Date: Mon Jul  1 06:17:51 UTC 2019
New revision: 505538
URL: https://svnweb.freebsd.org/changeset/ports/505538

Log:
  MFH: r502951 x11/cinnamon: Remove unnecessary version check

  Bug 237426 updates Pillow to 6.0.0, which removed the Image.VERSION
  variable, after a period of deprecation.

  Cinnamon currently uses this variable, so backport upstream commit
  c843f3664064742e2672e0fea528571a882d84ad [1] to compensate, so we don't
  need to wait for the Gnome teams Cinnamon 4 update, which has already
  pulled it in. Thanks to Charli Li for pointing this commit out.

  While I'm here, fix all stage-qa errors and ports/framework compliance
  issues:

  	- Add missing LIB_DEPENDS
  	- Add missing USE_{GNOME,X11} components
  	- Add USES=gnome, gl
  	- Fix gstreamer dependencies (GSTREAMER1, not GSTREAMER)
  	- Add LICENCE{_FILE}
  	- Fix more/all shebangs, remove custom REINPLACE_CMD section
  	- Remove GH_ACCOUNT (no longer necessary)

  [1] https://github.com/linuxmint/cinnamon/pull/8496
  [2] https://github.com/linuxmint/cinnamon/issues/8495

  PR:		238070, 237426
  Reported by:	kai
  Tested by:	Michal Kilijanek (via Twitter)
  Approved by:	kwm (gnome, maintainer)

  Approved by:	ports-secteam (blanket(s): bugfixes, dependencies)

Changes:
_U  branches/2019Q2/
  branches/2019Q2/x11/cinnamon/Makefile
  branches/2019Q2/x11/cinnamon/distinfo
  branches/2019Q2/x11/cinnamon/files/patch-files_usr_lib_cinnamon-settings_bin_imtools.py

Comment 29 Kubilay Kocak 2019-07-01 06:44:03 UTC

Closing, all dependent bugs committed & merged. Thank you Kai for your help!