Bug 237135 - [exp-run] Update graphics/opencv to 3.4.7
Summary: [exp-run] Update graphics/opencv to 3.4.7
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Port Management Team
URL: https://github.com/freebsd/freebsd-po...
Keywords:
Depends on:
Blocks:
 
Reported: 2019-04-09 07:11 UTC by Hiroo Ono
Modified: 2021-01-01 09:28 UTC (History)
6 users (show)

See Also:
tcberner: exp-run?


Attachments
opencv-3.4.5 patch. (198.36 KB, patch)
2019-04-09 07:11 UTC, Hiroo Ono
no flags Details | Diff
update to the patch to update to opencv 3.4.5 (208.28 KB, patch)
2019-04-14 05:54 UTC, Hiroo Ono
no flags Details | Diff
patch to update graphics/opencv 3.4.1 -> 3.4.5 (207.99 KB, patch)
2019-04-15 12:07 UTC, Hiroo Ono
no flags Details | Diff
patch to update graphics/opencv 3.4.1 -> 3.4.5 (208.55 KB, patch)
2019-04-16 08:38 UTC, Hiroo Ono
no flags Details | Diff
patch to update graphics/opencv 3.4.1 -> 3.4.5 (177.49 KB, patch)
2019-04-20 09:56 UTC, Hiroo Ono
no flags Details | Diff
opencv3 update patch 3.4.1 -> 3.4.5 (211.19 KB, patch)
2019-04-25 00:37 UTC, Hiroo Ono
no flags Details | Diff
List of files to remove from graphics/opencv3 (866 bytes, text/plain)
2019-04-25 00:39 UTC, Hiroo Ono
no flags Details
repocopy and update patch from opencv 3.4.1 to 3.4.6 (196.33 KB, patch)
2019-05-03 00:45 UTC, Hiroo Ono
no flags Details | Diff
ffmpeg port update regarding opencv 3.4.6 update (3.36 KB, patch)
2019-05-03 00:46 UTC, Hiroo Ono
no flags Details | Diff
files to be removed after applying opencv3 patch (555 bytes, text/plain)
2019-05-03 00:53 UTC, Hiroo Ono
no flags Details
repocopy and update patch graphics/opencv 3.4.1 -> 3.4.6 (196.33 KB, patch)
2019-05-03 19:18 UTC, Hiroo Ono
no flags Details | Diff
repocopy and update patch graphics/opencv 3.4.1 -> 3.4.6 (179.46 KB, patch)
2019-05-09 09:04 UTC, Hiroo Ono
no flags Details | Diff
patch to ports that depends on opencv3 (17.26 KB, patch)
2019-05-22 15:28 UTC, Hiroo Ono
no flags Details | Diff
repocopy and update patch graphics/opencv 3.4.1 -> 3.4.6 (179.82 KB, patch)
2019-05-22 15:32 UTC, Hiroo Ono
no flags Details | Diff
patch to opencv 3.4.7 (127.01 KB, patch)
2020-12-19 10:16 UTC, Arno de Vries
no flags Details | Diff
Additional file needed by opencv 3.4.7 (1.33 KB, text/plain)
2020-12-19 10:20 UTC, Arno de Vries
no flags Details
v1 (124.57 KB, patch)
2020-12-20 12:45 UTC, Tobias C. Berner
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hiroo Ono 2019-04-09 07:11:26 UTC
Created attachment 203513 [details]
opencv-3.4.5 patch.

Here is a patch to update opencv 3.4.1 to 3.4.5.
Regarding bug #234147, repocopy
  from graphics/opencv to graphics/opencv3,
  from graphics/py-opencv to graphics/py-opencv3,
  from graphics/opencv3-java to graphics/opencv3-java,
and apply the patch.

I am working on 3.4.1 -> 3.4.5 update, and arnov is working on 4.0.1 port.
(see: https://forums.freebsd.org/threads/make-extract-extracts-two-files-to-the-same-location.69940/ )
So, I decided to make subdirectories for 3.4.5 as below, to be able to also install 4.0.1.

/usr/local/bin/opencv_annotation -> /usr/local/bin/opencv-3.4.5/opencv_annotation
/usr/local/lib/libopencv_core.so -> /usr/local/lib/opencv-3.4.5/libopencv_core.so
/usr/local/share/OpenCV -> /usr/local/lib/OpenCV/3.4.5

Problem 1: How should I separate opencv python library between 3.4.5 and 4.0.1?
I have no idea and left the directory layout as it was.
Thus, graphics/py-opencv3 will confict with (coming) opencv 4.0.1 python library.

Todo:
1) There are about 42 ports that have direct dependency on graphics/opencv.
   I need to make change and test if these ports work, but it will take time.
2) I did not yet verified arnov's OpenCV 4.0.1 port do not really conflict
   with this 3.4.5 update. I will need to check and make change to the patch
   once he files the 4.0.1 port.
3) Though it is an option, there is dependency on graphics/ogre3d, but version
   1.10 or later. To update graphics/ogre3d, games/opendungeons,
   games/stuntrarry and misc/visp needs update and patches. The former two
   seem to have pull requests to make them work with ogre 1.11, but I am not
   sure about misc/visp (for visp, graphics/ogre3d is also an option.)
Comment 1 Hiroo Ono 2019-04-14 05:54:07 UTC
Created attachment 203655 [details]
update to the patch to update to opencv 3.4.5

Some improvements on the patch.

- graphics/py-opencv3 made for allflavors as it provides library to python (like math/py-numpy).
  Problem of conflict between (coming) 4.0.1 remains.
- Previous patch had plist issue with python 3.x.
- There is mutual dependency between multimedia/ffmpeg. So patch to ffmpeg/Makefile is included.
- Changed to use USES=blaslapack and refined around options a little.
- Previous patch failed to include java examples.
Comment 2 Hiroo Ono 2019-04-15 12:07:12 UTC
Created attachment 203694 [details]
patch to update graphics/opencv  3.4.1 -> 3.4.5

Update to the patch again. Sorry. This time:

- changes inside ${LOCALBASE}/libdata/pkgconfig/opencv(-core)-3.4.pc
  (I realized when I was building multimedia/lives)
- Qt buildtools and qmake are needed only in _build stage.
Comment 3 Kurt Jaeger freebsd_committer 2019-04-15 18:59:22 UTC
The PKGNAME for graphics/opencv and graphics/opencv3 is the same if the patch
is applied. I think this is not allowed.

PKGNAMESUFFIX=  3

is probably needed, the same for all the other abc3 variants.
Comment 4 Kurt Jaeger freebsd_committer 2019-04-15 19:00:18 UTC
See for example lang/gcc7.

Test with

make -V PKGNAME

PKGNAME must be uniq across all ports, that's why.
Comment 5 Hiroo Ono 2019-04-16 08:38:17 UTC
Created attachment 203713 [details]
patch to update graphics/opencv  3.4.1 -> 3.4.5

Thank you for your comment. I updated the patch.

- The package names are now: opencv3-3.4.5, opencv3-core-3.4.5, py*-opencv3-3.4.5, opencv3-java-3.4.5.
- Changed pkg-config file names to: opencv3.pc, opencv3-core.pc
Comment 6 Hiroo Ono 2019-04-20 09:56:50 UTC
Created attachment 203820 [details]
patch to update graphics/opencv  3.4.1 -> 3.4.5

Patch update.

- Corrected a mistake on IPP option.
- Made TESSERACT an option. (as is VTK and OGRE)

I am glancing over the ports that have direct dependency on graphics/opencv.
I have no yet finished but so far,

- ffmpeg and ffmpeg0 use C API so that they need to depend on OpenCV 3.4.x.
- p5-Image-ObjectDetect and rubygem-objectdetect also use C API.
- Other ports (that I checked so far) use C++ API. they should be first checked if they work with OpenCV 4.0.x, when there is OpenCV 4.0.x port. (OpenCV 4.0.x has no more C API.)
Comment 7 Hiroo Ono 2019-04-25 00:37:22 UTC
Created attachment 203992 [details]
opencv3 update patch 3.4.1 -> 3.4.5

Though I added the Ceres solver to the depend list, I forgot to remove the patch to disable Ceres.
Comment 8 Hiroo Ono 2019-04-25 00:39:36 UTC
Created attachment 203993 [details]
List of files to remove from graphics/opencv3

svn diff does not seem to record file removal (because I did svn copy + svn remove?).
I listed the file to rename or remove under graphics/opencv3.
Comment 9 Kurt Jaeger freebsd_committer 2019-04-30 21:23:47 UTC
I tried to apply the patch like this:

fetch -o /tmp/ocv3 https://bugs.freebsd.org/bugzilla/attachment.cgi?id=203992
cd graphics
svn copy opencv opencv3
svn copy opencv-core opencv3-core
svn copy py-opencv py-opencv3
svn copy opencv-java opencv3-java
cd ..
svn patch /tmp/ocv3

This caused this effect:

C         graphics/opencv3/Makefile
>         rejected hunk @@ -2,8 +2,8 @@
U         graphics/opencv3/distinfo
C         graphics/opencv3/files/FindOpenCV3Core.cmake.in
>         rejected hunk @@ -15,7 +15,8 @@
[...]
C         graphics/py-opencv3/Makefile
>         rejected hunk @@ -1,18 +1,16 @@
U         graphics/py-opencv3/pkg-plist
U         graphics/opencv3-java/Makefile
>         applied hunk @@ -2,13 +2,13 @@ with fuzz 2

So I guess the patch needs a small update due to other changes to the ports tree ?

But it looks very promising!
Comment 10 Hiroo Ono 2019-05-03 00:45:12 UTC
Created attachment 204176 [details]
repocopy and update patch from opencv 3.4.1 to 3.4.6

updated the diff.

* update to opencv 3.4.6
* took the diff against r500577
* made separate diff for ffmpeg
Comment 11 Hiroo Ono 2019-05-03 00:46:19 UTC
Created attachment 204177 [details]
ffmpeg port update regarding opencv 3.4.6 update

Made ffmpeg part a separate patch.
Comment 12 Hiroo Ono 2019-05-03 00:53:43 UTC
Created attachment 204178 [details]
files to be removed after applying opencv3 patch

Applying patches is rather complicated. Subversion does not seem to record filename change or deletion without comitting between svn copy and svn remove etc.

1. svn copy graphics/opencv graphics/opencv3
2. svn copy graphics/opencv-core graphics/opencv3-core
3. svn copy graphics/opencv-java graphics/opencv3-java
4. svn copy graphics/py-opencv graphics/py-opencv3
5. svn rename graphics/opencv3/files/FindOpenCVCore.cmake.in graphics/opencv3/files/FindOpenCV3Core.cmake.in
6. apply the patches (attachment 204176 [details] and 204177)
7. remove files listed in this patch from graphics/opencv3/files.

Then, making opencv3 port should work.
Comment 13 Hiroo Ono 2019-05-03 19:18:28 UTC
Created attachment 204201 [details]
repocopy and update patch graphics/opencv 3.4.1 -> 3.4.6

Sorry, there was one more mistake in dependency when OPENBLAS option was enabled.
Comment 14 Hiroo Ono 2019-05-09 09:04:37 UTC
Created attachment 204284 [details]
repocopy and update patch graphics/opencv 3.4.1 -> 3.4.6

- It turned out that I forgot to svn add some new patch files. Added these files.
- Changed the option DOCS to DOXYGEN and generate documents with doxygen. (Previously, documents were not generated.)
Comment 15 Hiroo Ono 2019-05-22 15:28:31 UTC
Created attachment 204540 [details]
patch to ports that depends on opencv3

Among the ports depending on graphics/opencv(-core):

astro/libkgeomap did not really depend on opencv (I filed dependency removal patch in bug #237934 ).
graphics/gstreamer-plugins-opencv becomes broken (a C source file refers to CV_RGB, which is defined to cv::Scalar, apparently a C++ symbol.)
graphics/pfstools with ALIGN option is broken (depends on deprecated libopencv_nonfree.so)

in the category graphics, these ports depends on opencv3 (this patch):
graphics/cimg
graphics/fei0r-plugins-opencv
graphics/gimp-gmic-plugin
graphics/gmic
graphics/gstreamer1-plugins-opencv
multimedia/gstreamer1-plugins  (included by graphics/gstreamer1-plugins-opencv)
multimedia/gstreamer1-plugins-bad (included by graphics/gstreamer1-plugins-opencv)
graphics/p5-Image-ObjectDetect
graphics/rubygem-objectdetect
graphics/shotwell

I could not build graphics/php-facedetect.

in the categories astro and graphics, these ports depends on opencv4 (will attach a patch to bug #237847 ):
astro/siril
graphics/caffe
graphics/digikam
graphics/nomacs
graphics/openimageio
graphics/waifu2x-converter-cpp
graphics/yafaray

Still remain to be patched:
math/saga
misc/auto-multiple-choice
misc/actiona
misc/visp
multimedia/ccextractor
multimedia/chilitags
multimedia/ffmpeg0
multimedia/libav
multimedia/lives
multimedia/mlt
multimedia/vlc
net/freeswitch
net-im/tox
net-p2p/retroshare
textproc/zxing-cpp
x11/xpra
x11-toolkits/nucleo
www/mod_pagespeed
Comment 16 Hiroo Ono 2019-05-22 15:32:42 UTC
Created attachment 204541 [details]
repocopy and update patch graphics/opencv 3.4.1 -> 3.4.6

Some more correction to the patch.

- Restored NONFREE option.
- A little more patching was needed with IPP option.
Comment 17 Kubilay Kocak freebsd_committer freebsd_triage 2020-07-03 02:30:25 UTC
^Triage: [tags] in issue Titles are deprecated
Comment 18 Tobias C. Berner freebsd_committer 2020-10-03 14:56:10 UTC
Moin moin 

@Hiroo Ono -- could you rebase your patch against the current tree?


mfg Tobias
Comment 19 Arno de Vries 2020-12-19 10:16:12 UTC
Created attachment 220712 [details]
patch to opencv 3.4.7

I have changed Hiroo Ono's opencv patch so it can be applied to the current tree. It is also updated to opencv 3.4.7 I do not have the infrastructure to test all depending ports. opencv-java and py-opencv build.
Comment 20 Arno de Vries 2020-12-19 10:20:25 UTC
Created attachment 220713 [details]
Additional file needed by opencv 3.4.7

This file is needed for the patch to 3.4.7. it should be in files
Comment 21 Kurt Jaeger freebsd_committer 2020-12-19 10:30:10 UTC
(In reply to Arno de Vries from comment #20)
Can you suggest a filename for attachment 220713 [details] ?
Comment 22 Arno de Vries 2020-12-19 10:32:14 UTC
Comment on attachment 220713 [details]
Additional file needed by opencv 3.4.7

Name of this file should be FindopencvCore.cmake.in
Comment 23 Kurt Jaeger freebsd_committer 2020-12-19 12:28:30 UTC
Hmm, the patch is no longer a repo-copy to have opencv3. Does this still matter
or will we do a repo-copy to have opencv4, if that patch comes along ?
Comment 24 Arno de Vries 2020-12-19 13:20:30 UTC
I tried to create a opencv4 port about a year ago. It was a bit to ambitious for my first port. It requires a lot of CMake knowledge. So opencv4 is not going to arrive anytime soon.
Comment 25 Tobias C. Berner freebsd_committer 2020-12-19 21:44:21 UTC
(In reply to Hiroo Ono from comment #6)
Moin moin

I will remove the VTK option again -- I think this port has already too many (most likely unused options), and is therefore hard to maintain. 

I'm preparing a slimmed down version of it at the moment, less options, but hopefully easier to keep up to date.


mfg Tobias
Comment 26 commit-hook freebsd_committer 2020-12-20 09:06:50 UTC
A commit references this bug:

Author: tcberner
Date: Sun Dec 20 09:06:20 UTC 2020
New revision: 558733
URL: https://svnweb.freebsd.org/changeset/ports/558733

Log:
  graphics/frei0r-plugins-opencv: prepare for OpenCV 3.4.7

  PR:		237135
  Obtained from:	https://github.com/dyne/frei0r/commit/b0a06d52e39438fae2afbf98bafe6c794d13b83e

Changes:
  head/graphics/frei0r/Makefile
  head/graphics/frei0r/files/extra-patch-src_filter_facebl0r_facebl0r.cpp
Comment 27 Tobias C. Berner freebsd_committer 2020-12-20 12:45:58 UTC
Created attachment 220746 [details]
v1

Moin moin 

I would like to ask for an exp-run of the attached patch, which can also be found here:
https://people.freebsd.org/~tcberner/patches/opencv-3.4.7.v1.diff


mfg Tobias
Comment 28 Antoine Brodin freebsd_committer 2020-12-21 13:26:18 UTC
multimedia/py-PySceneDetect has to be adjusted:

[00:01:57] Error: compute_deps_pkg failed to lookup pkgname for graphics/py-opencv@py27 processing package py27-PySceneDetect-0.5.4 from multimedia/py-PySceneDetect@py27 -- Is SUBDIR+=py-opencv@py27 missing in graphics/Makefile and does the port provide the 'py27' FLAVOR?
Comment 29 Antoine Brodin freebsd_committer 2020-12-25 07:44:10 UTC
This breaks ffmpeg which blocks a lot of ports too:

http://package22.nyi.freebsd.org/data/114amd64-default-foo/2020-12-24_10h05m43s/logs/errors/ffmpeg-4.3.1_9,1.log