Bug 253110 - graphics/opencv: where is options!?
Summary: graphics/opencv: where is options!?
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Tobias C. Berner
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-01-30 23:03 UTC by Ivan Rozhuk
Modified: 2023-08-20 17:29 UTC (History)
6 users (show)

See Also:
bugzilla: maintainer-feedback? (tcberner)


Attachments
patch-Makefile_Atlas_Option (2.05 KB, patch)
2021-02-01 01:37 UTC, alt2600
no flags Details | Diff
patch-Makefile (8.36 KB, patch)
2021-02-04 01:29 UTC, alt2600
no flags Details | Diff
patch-distinfo (1.02 KB, patch)
2021-02-04 01:30 UTC, alt2600
no flags Details | Diff
patch-pkg-plist (7.70 KB, patch)
2021-02-04 01:31 UTC, alt2600
no flags Details | Diff
BUILD_WRKSRC_CMakeDownloadLog.txt (5.65 KB, text/plain)
2021-02-04 01:31 UTC, alt2600
no flags Details
Makefile (6.98 KB, text/plain)
2021-02-04 01:32 UTC, alt2600
no flags Details
distinfo (1.20 KB, text/plain)
2021-02-04 01:32 UTC, alt2600
no flags Details
pkg-plist (31.07 KB, text/plain)
2021-02-04 01:32 UTC, alt2600
no flags Details
pacth-Makefile (8.93 KB, patch)
2021-02-20 07:05 UTC, alt2600
no flags Details | Diff
Makefile (7.14 KB, text/plain)
2021-02-20 07:06 UTC, alt2600
no flags Details
full commit patch of all affected files (17.65 KB, patch)
2021-02-20 07:07 UTC, alt2600
no flags Details | Diff
Add VTK to bsd.default-versions.mk (694 bytes, patch)
2021-02-20 08:39 UTC, Vladimir Druzenko
no flags Details | Diff
Patch for FLAVOR support of graphics/opencv (16.06 KB, patch)
2021-06-13 22:15 UTC, Tatsuki Makino
tatsuki_makino: maintainer-approval-
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Rozhuk 2021-01-30 23:03:48 UTC
1. This is a library port, this mean that some one want use it for develop own apps/libs with very unusual requirements.
I spent many time to make 3.xx work with all options set.
On past work I use many of these options, and do not use some of them.
For most users many of these default options not required, for developers - probably some of them missed.
Personally I never use gphoto, and do not remember that things from p.2 was required in my work.


2. As I understand new port is download tiny-dnn-tiny-dnn, opencv-opencv_3rdparty and some other staff during configure stage, see make configure:
...
-- IPPICV: Download: ippicv_2020_lnx_intel64_20191018_general.tgz
...
-- xfeatures2d/boostdesc: Download: boostdesc_bgm.i
-- xfeatures2d/boostdesc: Download: boostdesc_bgm_bi.i
-- xfeatures2d/boostdesc: Download: boostdesc_bgm_hd.i
-- xfeatures2d/boostdesc: Download: boostdesc_binboost_064.i
-- xfeatures2d/boostdesc: Download: boostdesc_binboost_128.i
-- xfeatures2d/boostdesc: Download: boostdesc_binboost_256.i
-- xfeatures2d/boostdesc: Download: boostdesc_lbgm.i
-- xfeatures2d/vgg: Download: vgg_generated_48.i
-- xfeatures2d/vgg: Download: vgg_generated_64.i
-- xfeatures2d/vgg: Download: vgg_generated_80.i
-- xfeatures2d/vgg: Download: vgg_generated_120.i
-- data: Download: face_landmark_model.dat
...


I am glad to see that OpenCV is updated, -core removed, in past is was hard to test all depend ports, but I am very disappointed that port lost all its options.
Comment 1 Tobias C. Berner freebsd_committer freebsd_triage 2021-01-31 06:01:31 UTC
Moin moin 

As I noted in the commit message the previous port was just unmaintainable in its state, so I cut it back to have something we can work with again.



Please let me know if there are any concrete parts that you require, that are now unavailable.

 



mfg Tobias
Comment 2 alt2600 2021-01-31 21:43:11 UTC
(In reply to Tobias C. Berner from comment #1)

how about the ability to select the BLAS libraries we want to use. OPENBLAS is not the default library, and while the port doesn't seem to support netlib it does support Atlas. Atlas and OpenBLAS are mutually exclusive conflicting ports doing the same basic thing. 

I am working on a patch for this now. Looking over the cmake files it seems as though you are missing a dependancy to get the LAPACK features properly functioning, this port doesn't seem to depend on math/lapacke but both atlas and openblas show it is needed if the build detects either as the blas library. Still early on patching the makefile to see if this will effect things and I will have to add that dependancy, but I can only test for atlas, I cannot rebase my entire BLAS world to openblas. I wish these two libraries played nice together instead of both being rooted directly in /usr/local/lib; /usr/local/include . I would think a dedicated directory for both would help, but maybe the loader would still get confused with the same symbols, this is beyond my knowledge.

hopefully I'll work through the patch in a day or so tops and I'll reference this post on it. If its not too taxing given these are just CMAKE booleans and added library dependencies I might look at the overall options issues, there are a ton like GDAL which do not have much to do with things like kdenlive, which is the only thing I really have that use this library, but I happened to install gdal for some CAD/CAM thing I was trying to install so i didn't notice until I just checked the makefile for what the poster might be concerned with. Anyone can likely make options for this if they want them, all I care about is atlas. the whole OPENCV_ENABLE_NONFREE Cmake option sounds interesting, if that somehow bars binary distribution and this is being used to make packages for the masses.

OCV_OPTION(OPENCV_ENABLE_NONFREE "Enable non-free algorithms" OFF) 
certainly sounds like it should not be enabled by default
Comment 3 Ivan Rozhuk 2021-01-31 22:24:27 UTC
(In reply to Tobias C. Berner from comment #1)

> previous port was just unmaintainable in its state, so I cut it back to have
> something we can work with again.

If you return all features that port have before - you will get almost same makefile.

I agree that is was hard to maintain, but it can not be easy to maintain so complex lib with a lot options.
It is a good thing that -core was dropped, but other is not.

Current version include too many features, that other ports will never use and do not allow control options and understanding what is happening for users who want to use opencv as lib in his project.


> Please let me know if there are any concrete parts that you require, that are now unavailable.

- all options back
- new options from 4.x
- do not download crap at configure state - IMHO this violates some ports rules?
Comment 4 alt2600 2021-01-31 22:44:40 UTC
(In reply to rozhuk.im from comment #3)


not sure it violates rules, but it makes patching next to impossible to fix simple things like this. Its not during configure, its downloading packages during build for me. I even have it finding ATLAS during configure stage, but I'm going to have to find the option to disable this IPP thing to make sure it works ok. Also math/lapacke is definitely a requirement, likely for OpenBLAS to work as well.

https://docs.opencv.org/master/d2/dca/group__xfeatures2d__nonfree.html
"This section describes two popular algorithms for 2d feature detection, SIFT and SURF, that are known to be patented. You need to set the OPENCV_ENABLE_NONFREE option in cmake to use those. Use them at your own risk."

haven't look at the new makefiles yet, just noticed some comments, but this should definitely not be set by default. I'll have to pick up the new ones to stage ATLAS patches.

===>  Building for opencv-4.5.1
[1/1443] cd /usr/ports/graphics/opencv/work/.build && /usr/local/bin/cmake -DCMAKE_HELPER_SCRIPT=/usr/ports/graphics/opencv/work/.build/OpenCVGenPkgConfig.info.cmake -P /usr/ports/graphics/opencv/work/opencv-4.5.1/cmake/OpenCVGenPkgconfig.cmake
[2/1443] /usr/bin/cc -DICV_BASE -DIW_BUILD -I3rdparty/ippicv/ippicv_lnx/iw/include -I3rdparty/ippicv/ippicv_lnx/icv/include -isystem . -O2 -pipe -march=westmere  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winconsistent-missing-override -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -Qunused-arguments -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -Wno-unused-function -Wno-missing-braces -Wno-missing-field-initializers -Wno-self-assign -O2 -pipe -march=westmere  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -DNDEBUG -fPIC -MD -MT 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o -MF 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o.d -o 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o -c 3rdparty/ippicv/ippicv_lnx/iw/src/iw_core.c
FAILED: 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o 
/usr/bin/cc -DICV_BASE -DIW_BUILD -I3rdparty/ippicv/ippicv_lnx/iw/include -I3rdparty/ippicv/ippicv_lnx/icv/include -isystem . -O2 -pipe -march=westmere  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winconsistent-missing-override -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -Qunused-arguments -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -Wno-unused-function -Wno-missing-braces -Wno-missing-field-initializers -Wno-self-assign -O2 -pipe -march=westmere  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -DNDEBUG -fPIC -MD -MT 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o -MF 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o.d -o 3rdparty/ippiw/CMakeFiles/ippiw.dir/src/iw_core.c.o -c 3rdparty/ippicv/ippicv_lnx/iw/src/iw_core.c
In file included from 3rdparty/ippicv/ippicv_lnx/iw/src/iw_core.c:63:
/usr/include/malloc.h:3:2: error: "<malloc.h> has been replaced by <stdlib.h>"
#error "<malloc.h> has been replaced by <stdlib.h>"
Comment 5 Ivan Rozhuk 2021-01-31 22:53:43 UTC
(In reply to alt2600 from comment #4)

> Its not during configure, its downloading packages during build for me.

make configure
Will show "downloading" from my prev message here.
As I remember some files was downloaded later...


> but I'm going to have to find the option to disable this IPP thing to make sure
> it works ok. Also math/lapacke is definitely a requirement, likely for OpenBLAS
> to work as well.

This all was done in 3.5x port.
Comment 6 alt2600 2021-01-31 23:02:33 UTC
(In reply to rozhuk.im from comment #5)

my apologies, you are correct, I scanned my console window too quick and got confused. I added WITH_IPP to the CMAKE_OFF list and it is building so far with ATLAS. 

perhaps it can be pre-downloaded like the other third party libraries are being downloaded so they can be patched like the other ones are being down with those post-extract statements so patches can apply. I don't know what this IPP thing is other then some kind of intel optimizations. Not sure they are important or not.
Comment 7 Ivan Rozhuk 2021-01-31 23:37:12 UTC
IPP crap was disabled by:
-DWITH_IPP:BOOL=OFF \
-DWITH_IPP_A:BOOL=OFF \
-DBUILD_WITH_DYNAMIC_IPP:BOOL=OFF \

Do not remember details why I disable it.


Other custom downloaded things was downloaded by ports system and moved to places where cmake downloader found them and use:
EXTRA_MODULES_DESC=		Extra modules
EXTRA_MODULES_GH_ACCOUNT=	tiny-dnn:extra_mod_3rdparty_tiny_dnn
EXTRA_MODULES_GH_PROJECT=	opencv_contrib:extra_mod \
				tiny-dnn:extra_mod_3rdparty_tiny_dnn \
				opencv_3rdparty:extra_mod_3rdparty_face_landmark_model \
				opencv_3rdparty:extra_mod_3rdparty_boost_descr \
				opencv_3rdparty:extra_mod_3rdparty_vgg_descr
EXTRA_MODULES_GH_TAGNAME=	${PORTVERSION}:extra_mod \
				2a2b50caa437a5964a61e45ffc83e70558e2bc68:extra_mod_3rdparty_tiny_dnn \
				8afa57abc8229d611c4937165d20e2a2d9fc5a12:extra_mod_3rdparty_face_landmark_model \
				34e4206aef44d50e6bbcd0ab06354b52e7466d26:extra_mod_3rdparty_boost_descr \
				fccf7cd6a4b12079f73bbfb21745f9babcd4eb1d:extra_mod_3rdparty_vgg_descr

post-extract-EXTRA_MODULES-on:
	@${MV} ${WRKSRC_extra_mod}/doc/tutorials/* ${WRKSRC}/doc/tutorials/
	@${MV} ${WRKSRC_extra_mod}/modules/* ${WRKSRC}/modules/
	@${CP} -RpP ${WRKSRC_extra_mod}/samples/* ${WRKSRC}/samples/
	@${MKDIR} ${CONFIGURE_WRKSRC}/3rdparty/tinydnn/tiny-dnn-1.0.0a3/
	@${MV} ${WRKSRC_extra_mod_3rdparty_tiny_dnn}/* ${CONFIGURE_WRKSRC}/3rdparty/tinydnn/tiny-dnn-1.0.0a3/
	@${MKDIR} ${CONFIGURE_WRKSRC}/share/OpenCV/testdata/cv/face/
	@${MV} ${WRKSRC_extra_mod_3rdparty_face_landmark_model}/* ${CONFIGURE_WRKSRC}/share/OpenCV/testdata/cv/face/
	@${MKDIR} ${CONFIGURE_WRKSRC}/downloads/xfeatures2d/
	@${MV} ${WRKSRC_extra_mod_3rdparty_boost_descr}/* ${CONFIGURE_WRKSRC}/downloads/xfeatures2d/
	@${MV} ${WRKSRC_extra_mod_3rdparty_vgg_descr}/* ${CONFIGURE_WRKSRC}/downloads/xfeatures2d/
Comment 8 alt2600 2021-02-01 01:37:38 UTC
Created attachment 222058 [details]
patch-Makefile_Atlas_Option

comment #7

yeah IPP had to go due to using malloc.h on my end, I used a slightly different method then you, but effectively the same thing in reality. I also added a dependency to math/lapacke so Atlas was fully detected and usable. I suspect Openblas wants this library too its referenced in ${WRKSRC}cmake/OpenCVFindLAPACK.cmake . I left the default for openblas, cannot test that on my end, but only real change was having to make edits to the CmakeFindAtlas scripts to use /usr/local essentially by setting it to the prefix by using a REINPLACE_CMD call post patch. I opted for a direct Library dependency instead of a uses:blaslapack:atlas as the cmake scripts don't seem to read any of those environment variables set, but still may be cleaner to use uses, I have in the past done it that way.

Are you working on adding back the remaining options? or have something working you can attach so I can add/patch in the Atlas stuff?

I have confirmed this builds installs, and the binaries run, can't test beyond execution. frei0r_plugins-opencv built against the installed version I did with my patched Makefile. kdenlive runs, I also rebuilt it just for good measure and it runs fine and seems to accept the new opencv version in both cases. ffmpeg seems to be happy with finding the core files as part of the single port, and reinstalled fine and seems to be running fine.
Comment 9 Ivan Rozhuk 2021-02-01 01:45:08 UTC
No, I am not working now.
If I will - it will be makefile from 3.4 that I will update to use with 4.xx.
Comment 10 alt2600 2021-02-01 01:53:16 UTC
(In reply to rozhuk.im from comment #9)

then I might take a crack at modifying the current makefile just to flip the the current CMAKE_ON modules into port knobs assuming the blizzard in my area doesn't take my power/internet out tomorrow. Shouldn't take too long. I will not worry about all the third party items. I will leave it fully building with all the depends by default as set up now except for the patented stuff, and keeping IPP disabled.
Comment 11 Tobias C. Berner freebsd_committer freebsd_triage 2021-02-01 04:55:18 UTC
> If you return all features that port have before - you will get almost same makefile.

If that is what you want, then you need to maintain it :) -- I can give it to you if you want.


mfg Tobias
Comment 12 Vladimir Druzenko freebsd_committer freebsd_triage 2021-02-01 12:15:56 UTC
Check patch from this PR:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253118
Comment 13 alt2600 2021-02-01 17:49:59 UTC
(In reply to Tobias C. Berner from comment #11)


No thank you, I looked at the old Makefile and it is a nightmare most unholy. No way will I re-implement that. A reasonable middleground that doesn't require hacking in patches to third party configured downloads seems to be possible. 

I just want the port to build on my machine, As I said, all I care about is adding Atlas, as openblas is a non-starter for me, Atlas was easy to support, just needed to correct searchpaths in the cmake files. I did make a few corrections to my posted patch my next one will cover LOCALBASE instead of PREFIX. Option wise make simple knobs with the WITH_ cmake booleans into options, and turn off whatever is causing problems, or just found by cmake regardless of being enabled. I an already planning on killing things like opengl, unless it proves trivial to get GTK or QT going with uses directives. Its amazing how many optional features are set to enable by default, this things cmake scripts find and try to link with everything on your system without tracking the dependency by pkgng. And if if cannot find it decide to just build its own copy of it. I have the basic options framed out, now I am confirming they configure/build correctly and package. If someone wants all that other third party stuff let them step up to maintain. I promise I'm not trying to deliver a nightmare of a Makefile.
Comment 14 Tobias C. Berner freebsd_committer freebsd_triage 2021-02-02 09:03:13 UTC
(In reply to alt2600 from comment #13)
> A reasonable middleground that doesn't require hacking in patches to third party configured downloads seems to be possible. 

This is already done for example for 'ade', which is downlaoded and moved to the proper location, with some patching of the init-cmake file.

This should be added for the others too (note, that inside poudirere no downloading will happen).


mfg Tobias
Comment 15 alt2600 2021-02-02 18:47:36 UTC
(In reply to Tobias C. Berner from comment #14)

I totally see why you might not want to maintain this one lol...

Yeah I noticed and found that the files have to go somewhere else then what is in the current make file and can confirm no downloads happen anymore, as well as unplugged my ethernet to reconfirm the build finds them and uses the ones pre-downloaded. What an evil build system. I also added the one missing one for facial recognition data.

Basically at this point I am cleaning up the plist subs for the various options and going through testing each feature. Will need to pick up the recent changes made for python 3.8 the other day and retest a bit depending on how that change was made. Also plan to patch CmakeDownload.cmake or whatever it is called that does the downloading to not allow downloading at all by swapping the cmake file() call with a message() to FATAL ERROR out with message to report the error and provide the CMakeDownloadLog.txt from the build directory to stop cold any actual downloading during configure at all.

I'd post an updated patch, but I'm so close to being done I'd rather do some testing and sync it with head to make sure things still work ok to give a complete product. Its probably another day or two given all the options to confirm. I will give you this, it will stop the configure downloading stuff. the face alignment dat and the locations to copy them need updating. It will still claim it is downloading if you look at make configure in graphics/opencv, but if you look at the download log it will show it found the "download" confirmed its checksum and copied it to where it wanted in the build directory. You can use MV instead of CP I was getting tired of not knowing what to look for when I was googling... If you want, you can ignore the atlas lines but the libpng lines for post-patch will allow the port to find png and add support for it. But for now it doesn't hurt anything for it not to find it, so your call.

GH_PROJECT=	opencv_contrib:contrib \
		ade:ade \
		opencv_3rdparty:extra_mod_3rdparty_boost_descr \
		opencv_3rdparty:extra_mod_3rdparty_vgg_descr \
		opencv_3rdparty:extra_mod_3rdparty_face_alignment_dat
GH_TAGNAME=	v0.1.1f:ade \
		34e4206aef44d50e6bbcd0ab06354b52e7466d26:extra_mod_3rdparty_boost_descr \
		fccf7cd6a4b12079f73bbfb21745f9babcd4eb1d:extra_mod_3rdparty_vgg_descr \
		8afa57abc8229d611c4937165d20e2a2d9fc5a12:extra_mod_3rdparty_face_alignment_dat


post-extract:
	${MV} ${WRKSRC_contrib} ${WRKSRC}/contrib
	${MKDIR} ${BUILD_WRKSRC}/3rdparty/ade
	${MV} ${WRKSRC_ade} ${BUILD_WRKSRC}/3rdparty/ade
	${MKDIR} ${BUILD_WRKSRC}/downloads/xfeatures2d
	${CP} ${WRKSRC_extra_mod_3rdparty_boost_descr}/* ${BUILD_WRKSRC}/downloads/xfeatures2d
	${CP} ${WRKSRC_extra_mod_3rdparty_vgg_descr}/* ${BUILD_WRKSRC}/downloads/xfeatures2d
	${MKDIR} ${BUILD_WRKSRC}/share/opencv4/testdata/cv/face
	${CP} ${WRKSRC_extra_mod_3rdparty_face_alignment_dat}/* ${BUILD_WRKSRC}/share/opencv4/testdata/cv/face
post-patch:
	${REINPLACE_CMD} -e 's|/usr/lib/atlas-base|${LOCALBASE}/lib|g' ${WRKSRC}/cmake/OpenCVFindAtlas.cmake
	${REINPLACE_CMD} -e 's|/usr/include/atlas|${LOCALBASE}/include|g' ${WRKSRC}/cmake/OpenCVFindAtlas.cmake
	${REINPLACE_CMD} -e 's|/libpng/png.h|/libpng16/png.h|g' ${WRKSRC}/cmake/OpenCVFindLibsGrfmt.cmake
	${REINPLACE_CMD} -e 's|<libpng/png.h>|<libpng16/png.h>|g' ${WRKSRC}/modules/imgcodecs/src/grfmt_png.cpp
Comment 16 alt2600 2021-02-04 01:29:57 UTC
Created attachment 222147 [details]
patch-Makefile
Comment 17 alt2600 2021-02-04 01:30:38 UTC
Created attachment 222148 [details]
patch-distinfo
Comment 18 alt2600 2021-02-04 01:31:02 UTC
Created attachment 222149 [details]
patch-pkg-plist
Comment 19 alt2600 2021-02-04 01:31:52 UTC
Created attachment 222150 [details]
BUILD_WRKSRC_CMakeDownloadLog.txt
Comment 20 alt2600 2021-02-04 01:32:10 UTC
Created attachment 222151 [details]
Makefile
Comment 21 alt2600 2021-02-04 01:32:26 UTC
Created attachment 222152 [details]
distinfo
Comment 22 alt2600 2021-02-04 01:32:45 UTC
Created attachment 222153 [details]
pkg-plist
Comment 23 alt2600 2021-02-04 01:50:33 UTC
Ok marked old patch obsolete and added the latest set. I synced my ports and found I need to focus on updating my 12.1 system... So here is what I was able to do, again I can only test NOBLAS and ATLAS not OPENBLAS. I added the knob to have no blas at all as opencv allows that

First I fixed the locations for the third party downloads going into the tree to prevent configure downloading things, I attached the download log to confirm its not actually downloading the files. Also added the facial recognition data that was missing.

for the build options I set hard CMAKE_OFF to disable all surprises that were not going to be ultimately set with CMAKE_BOOL directives by selecting the options and bringing in found dependencies pkg cannot track. I turned on Build Hardening, and Strict Configure checking so any set option that is missing the libraries to complete will stop configure for troubleshooting, how i found the libpng issue.

I set plist sub settings into the pkg-plist, and ordered them into groups at the bottom with the existing ones to make them easy to find. I checked various build option sets for orphans along the way and found none. This includes the python tweaks made on that other 3.8 python bug report.

given the new 3rd party facial recognition stuff I had to update distinfo.

For the build I left TBB off due to seeing some other bugs showing oneTBB coming into the picture potentially, so left optional. Also I left FFMPEG off as it could create a circular dependency if you say build your ffmpeg with opencv support as I have, but i can adapt on my end depending on your thoughts. I turned off the patented 2d features by default. I kept openBLAS as the default, but maybe NOBLAS would be more appropriate. I had to make some REINPLACE substitutions mentioned previously to get the build finding our libpng includes headers.

I think I can add DOCS, EXAMPLES, and VTK but I use VTK8 and not sure how to cleanly let the build system detect VTK>6 is fine, need to review the porters handbook a bit. I do want to also look at patching the build to totally block allowing downloading at all during configure by patching that code and perhaps creating a make variable that can be defined to allow downloading to happen so the CmakeDownloadLog.txt can be checked and things added as needed. but no time right now. As other modules come in, other random 3rdparty modules will start building which may bring in more random downloads, which needs to stop, but be easy to troubleshoot.

My issue is I need to focus on getting to 12.2 and rebuilding my system which is always fun. My gamble of just waiting on 13.0 didn't pan out so well. I wanted to stop at a place that meets the spirit of this bug report by giving back options, but didn't want to get bogged down during the updates and not being able to report back so this can possibly be closed for the time being. Let me know what you think, I do not think this added too much to making this hard to maintain.
Comment 24 alt2600 2021-02-04 02:01:15 UTC
oh, and want to point out, configure will show in the log they are trying to download these files, but its not. if it finds the copy it will use it but it uses the same function to do all the checks or download basically. The CMakedDownloadLog.txt in the build directory will show the downloading status with a progress hash if it actually downloads things. That is why I attached the clean log. The same log is produced when my machine is unplugged physically from the internet.
Comment 25 Vladimir Druzenko freebsd_committer freebsd_triage 2021-02-04 07:10:59 UTC
(In reply to alt2600 from comment #23)
> I think I can add DOCS, EXAMPLES, and VTK but I use VTK8 and not sure how to cleanly let the build system detect VTK>6 is fine, need to review the porters handbook a bit.
I have VTK9 installed, opencv found it and use as dependency check my PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253118
I create patch for opencv with VTK option: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=222068&action=diff
This patch work fine for me on 12.2 amd64 with VTK9 - I can turn ON and OFF this option.
Comment 26 Tobias C. Berner freebsd_committer freebsd_triage 2021-02-04 08:03:59 UTC
(In reply to alt2600 from comment #23)
Moin moin 

Thank you very much for the work -- I will surely commit it, if the consensus is that all these options are required.

I would also suggest, that you step up as the new maintainer of the port :) 


mfg Tobias
Comment 27 alt2600 2021-02-04 15:59:48 UTC
(In reply to VVD from comment #25)

I suspected it would be easy to get going. My issue was I need/use vtk8, whereas not sure if folks would want vtk9 or vtk6 even. I think based on a quick glance at freshports, if you have cad like things they seem to use vtk8, graphics oriented seems to use vtk9, and didn't check what vtk6 might be doing. Only thing I could come up with is flavors to track the dependent libraries soas to support the full range opencv says it would accept, but didn't have time to check it all out, or if there is an easier way with the vtk version to massage what is being looked for as a dependency.
Comment 28 alt2600 2021-02-04 16:30:03 UTC
(In reply to Tobias C. Berner from comment #26)

I've thought about that more. I would like to get more involved with things, given how long I've enjoyed using FreeBSD.

I guess I would be less opposed, but I've never been one, not sure what the process is. Don't want my private email in makefiles... But there are some other ports I'd like to see created, I would imagine being a maintainer on one makes it easier to create/be a maintainer on others. Despite that this build system is just evil, I've gotten fairly familiar with taming it. My concern remains that messing with makefiles is one thing, if it came to patching code tremendously I'd be lost. Not that this one seems to require too much of that at least for this particular version, compared to trying to resurrect grub2... :)

how does one go about this? I'll have to do some looking on my end too I suppose.
Comment 29 Tobias C. Berner freebsd_committer freebsd_triage 2021-02-11 05:48:48 UTC
(In reply to alt2600 from comment #28)
Moin moin 

I think you're motivated enough to step up as the maintainer.

Please let me know, if you agree.


mfg Tobias
Comment 30 alt2600 2021-02-20 07:05:38 UTC
Created attachment 222659 [details]
pacth-Makefile

reconciled one more time with new port update on LTO and HARDENING jumping as options. did not enable by default, but LTO works fine, I had HARDENING by default prior left as an option not in default now. also bumped portrevision as we really aught consider flushing out patented code being distributed on our repos without seemingly having licenses.
Comment 31 alt2600 2021-02-20 07:06:32 UTC
Created attachment 222660 [details]
Makefile

full make for those that want to try without using a patch
Comment 32 alt2600 2021-02-20 07:07:44 UTC
Created attachment 222661 [details]
full commit patch of all affected files

hopefully the proper concatenation of all the patches into a single patch file for ease of comitting
Comment 33 alt2600 2021-02-20 07:24:58 UTC
(In reply to Tobias C. Berner from comment #29)

I was hoping you could commit this before another jumps the line again. especially when that commit only made options but didn't do anything to correct configure downloading missing files.

I attempted VTK but I have to use 8 and this thing didn't seem to accept VTK_DIR being set so I gave up for now. Cannot test vtk 9 the other user suggested as it would break my entire system just to install it given my firm dependancies on vtk8. And that brings up why I'm still on the fence about being a maintainer with this one. I'll need to stage a bhyve to properly test options which break by production system. Also, during my 12.2 upgrade i noticed atlas was broken with gcc10, had no maintainer, and my patches have gone uncommitted to fix it with gfortran10 so I was thinking that might be a place I'd get my feet wet with maintaining things given this one is maintained. Being a zealot about atlas anyway, and i don't want to see it end up being deleted.

I may come back to this one once I setup a clean bhyve, and likely a poudriere system again to make my life a little easier with spare packages lying around. Just wish it took portconf port config syntax so have to script a translator first or hack in the ability to use directly... 

anyway I think I've had enough fun with this one for a while, let someone else have a go at it for a bit. If you do add VTK I would ask you use VTK9, to leave room for VTK8. I might come back to it, but I'm getting rather sick of it, and all I ever really wanted was for it to allow using atlas, which is definitely accomplished... :) I do hope you consider committing this. let me know if the formats are bad, but it does look like the combined patch file is properly showing the diff on the website, so I think its all complete and tidy.
Comment 34 Vladimir Druzenko freebsd_committer freebsd_triage 2021-02-20 08:39:12 UTC
Created attachment 222662 [details]
Add VTK to bsd.default-versions.mk

(In reply to alt2600 from comment #33)
> I attempted VTK but I have to use 8 and this thing didn't seem to accept VTK_DIR being set so I gave up for now.
> Cannot test vtk 9 the other user suggested as it would break my entire system just to install it given my firm dependancies on vtk8. 
We can add default version of VTK.
Comment 35 alt2600 2021-02-22 18:38:11 UTC
(In reply to VVD from comment #34)
 I don;t think this helps at all. opencascade will not build without VTK8, it looks like as of 11/2020 it doesn't support VTK 9, possibly to to some name conflicts with X11. Its getting a little bit gibberishy for me.

https://bugzilla.redhat.com/show_bug.cgi?id=1840501

https://gitlab.kitware.com/vtk/vtk/-/issues/18048

this will just make that port break by default. Not sure if it would be more appropriate to do a USES setup that can preload the library paths, and where to find the Cmake files in a way that cmake will actually accept.

The core issue is the ports install stuff in different places, 9 appears directly into /usr/local/lib, 8 into /usr/local/lib/vtk-8.2 . Now it would seem if they both installed prefixed into their own /vtk-${VTK_VERSION} we could install multiple versions of it instead of being locked to one version in ports. Or since they conflict anyway have vtk 8 install rooted in /usr/local/lib like vtk9 does. Similarly if we had out BLAS libraries each in their own directories we should be able to install Atlas, OpenBLAS, etc without conflicts, but that's another topic.

one user reports opencv can find the cmake files for vtk-9 when the cmake files are off the lib folder, but the opencv build system ignores -DVTK_DIR:PATH=/usr/local/lib/vtk-8.2/cmake/vtk-8.2 for finding the VTKConfig.cmake file, I've tried shortening it to just -DVTK_DIR:PATH=/usr/local/lib/vtk-8.2 but in all cases the Cmakecache.txt file says VTK_DIR not found.
Comment 36 Vladimir Druzenko freebsd_committer freebsd_triage 2021-04-25 11:30:26 UTC
opencv still silently grab vtk if it's installed.
Comment 37 Tatsuki Makino 2021-06-13 22:15:33 UTC
Created attachment 225777 [details]
Patch for FLAVOR support of graphics/opencv

Is this the spearhead of opencv maintenance?

This requires FLAVOR support as %%PYTHON_SITELIBDIR%% appears in pkg-plist.
The package name is changed by USE_PYTHON=optsuffix when it is not the default python version. (See databases/rrdtool, multimedia/openshot, etc.)
This includes an update to 4.5.2 (which I need right away), so if you use this as-is, you'll need to do some more work here.
Comment 38 Tatsuki Makino 2021-06-13 22:28:45 UTC
If USE_JAVA=yes with JAVA option is used, then share/java in pkg-plist will be replaced by %%JAVASHAREDIR%%.
However, if we do that replacement, we need to link it with OPENCV_JAR_INSTALL_PATH defined in ${WRKSRC}/cmake/OpenCVInstallLayout.cmake.
Comment 39 Tatsuki Makino 2021-06-14 09:18:02 UTC
(In reply to Tatsuki Makino from comment #37)
> USE_PYTHON=optsuffix

Mk/Uses/python.mk says USE_PYTHON=optsuffix is deprecated.
Then there is the trouble with using optsuffix for libraries.
For example, it will not be able to be used in the following ways

RUN_DEPENDS= opencv${PYTHON_PKGNAMESUFFIX}>=4.5.2:graphics/opencv@${PY_FLAVOR} # PYTHON_PKGNAMESUFFIX is always defined as -py38
RUN_DEPENDS= opencv${PKGNAMESUFFIX}>=4.5.2:graphics/opencv@${PY_FLAVOR} # optsuffix must be added to this port
Comment 40 Vladimir Druzenko freebsd_committer freebsd_triage 2021-06-21 08:54:18 UTC
Bug with VTK still exist.
Comment 41 Vladimir Druzenko freebsd_committer freebsd_triage 2021-07-18 12:41:16 UTC
Add, plz, WITH_VTK to CMAKE_OFF.
Comment 42 Tobias C. Berner freebsd_committer freebsd_triage 2021-07-18 12:47:33 UTC
(In reply to VVD from comment #41)
yes, will be in
Comment 43 Tobias C. Berner freebsd_committer freebsd_triage 2021-07-18 12:50:41 UTC
(In reply to Tatsuki Makino from comment #39)
Moin moin 

Flavoring this package by pyXX does not look sensible to me -- I think only supporting the default python version is appropriate. 
If multiple python versions are required, then the python part needs to be split out, and flavoured there.


mfg Tobias
Comment 44 commit-hook freebsd_committer freebsd_triage 2021-07-18 13:54:12 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=c582239e8a2b795fc0a7f51a3463cee9998cfab8

commit c582239e8a2b795fc0a7f51a3463cee9998cfab8
Author:     Tobias C. Berner <tcberner@FreeBSD.org>
AuthorDate: 2021-07-18 12:13:52 +0000
Commit:     Tobias C. Berner <tcberner@FreeBSD.org>
CommitDate: 2021-07-18 13:52:58 +0000

    graphics/opencv: add makefile options

    Bring back lot of the options that were available in the old port.

    Reported by:    Ivan Rozhuk <rozhuk.im@gmail.com>
    Original by:    alt2600@icloud.com
                    VVD <vvd@unislabs.com>
    PR:             253110
    PR:             255446

 graphics/opencv/Makefile                           | 194 ++++++++++++++++-----
 .../patch-cmake_OpenCVFindLibsGrfmt.cmake (new)    |  14 ++
 ...atch-modules_imgcodecs_src_grfmt__png.cpp (new) |  11 ++
 graphics/opencv/pkg-plist                          | 139 ++++++++-------
 4 files changed, 248 insertions(+), 110 deletions(-)
Comment 45 Tobias C. Berner freebsd_committer freebsd_triage 2021-07-18 13:56:46 UTC
Moin moin 

Finally committed -- let me konw if you miss something.


mfg Tobias
Comment 46 Tatsuki Makino 2021-07-19 00:33:24 UTC
(In reply to Tobias C. Berner from comment #43)

Yes. It is very difficult and I am also having trouble with astro/geographiclib (which I maintain).
If the part about python is inseparable, PKGNAME will be dominated by python if flavor is supported.
However, I don't know how "libraries" like these behave when they are sandwiched between dependencies (required by and dependent on other FLAVOR-supported python ports) :)
Comment 47 Vladimir Druzenko freebsd_committer freebsd_triage 2023-08-20 17:29:11 UTC
Close?