Created attachment 220313 [details] Patch for pcre2 * Update pcre2 to 10.36 * Add main site as fallback in case sourceforge mirrors are unavailable * Use DISTVERSION instead of PORTVERSION as per "Table 5.2. Package Naming Examples" in porters handbook * Switch to tar.bz2 archive, ~25% reduction in size * Switch to CMake, significantly reduces compilation time [1] * Add "hidden" option to set MATCH_LIMIT_DEPTH * Define _USES before ARGS for consistency References: [1] Tested on 13-CURRENT (amd64), Intel i7-3770k Autotools: ====> Compressing man pages (compress-man) 1m0.95s real 2m31.33s user 6.26s sys CMake: ====> Compressing man pages (compress-man) 31.00s real 1m7.34s user 1.95s sys Compile tested on FreeBSD 13.0-CURRENT #0 r367711 (amd64) (make, make check-plist, make test) Poudriere testport OK 12.2-RELEASE (amd64) (with and without (lib)readline Poudriere testport OK 11.4-RELEASE (amd64) (with and without (lib)readline Additional notes: This change doesn't install static lib(s) by default, I can't find any port that makes use of it in the tree.
Created attachment 220468 [details] Patch for pcre2 v2 Refresh patch, make use of CMAKE_ON and _OFF macros
Created attachment 220611 [details] Patch for pcre2 v3 Cosmetic fix to Makefile, reviewed by tcb@
What is the rationale to use cmake in this case, improving compiling time by some seconds looks like a microoptimization for me, don't forget cmake itself takes almost 1h to build, so I'd rather stay with current pcre2 dependencies
Hi, I first want to apologize for the late reply, I simply forgot sorry :/ I agree that CMake on its own for pcre2 might seem like an odd move however many projects are transitioning to either CMake or Meson so moving the dependency one level further down shouldn't be much of a deal looking at the big picture. We currently have roughly 3k ports using CMake and more are transitioning from GNU Autotools. CMake is usually also considered easier to use, more cross-compiler friendly and easier to debug overall. If we also can cut compilation time that's a nice bonus. While it's not an official statement there also seems to be a general desire to move away from GNU Autotools in general for ports. As for compiling CMake I did a few tests, it compiles much faster than 1h unless you're possibly using an ARM-based SBC. Intel Pentium G3220 (3.0Ghz, 1 job) 18m5.15s real 17m23.35s user 37.21s sys RockPro64 (4x A53 @ 1.4GHz utilizing all cores) 46m19.51s real 2h49m3.86s user 5m58.92s sys Best regards, Daniel
although the time was effectively halved, this isn't about "a few seconds" the thing that often takes the longest in VM autotools based build is running configure, a task that cannot be parallelised, and that shells out a million times, often asking the same questions over and over again. moving to cmake isn't about shaving off a few seconds of build time, it's about saving hours of developer and maintainer time, because while writing and maintaining and debugging a single ten line sh script is easy, debugging thousands of lines of shell is - not impossible, but very, very tedious.
Kirill, is this still a no go if I fresh patch to the current release (10.39)?
Ping @krion In the meantime the port has been updated to latest (10.39) release, but a number of the improvements -- I do consider them improvements, even if some are rather edge-casey -- still apply (viz. CMake, sites, source format). There's now also I strange "PhilipHazel" GH_ACCOUNT in the port -- that is not the canonical account for upstream pcre2. I can't tell if GH is the official upstream source -- the GH repo says so, so does pcre.org. The GH repo has a 10.40 tag, the site still says 10.39 is latest.
@Adriaan de Groot 10.40 is currently under releases/ so I guess it's officially released, site might still provide old info. I'll update the port accordingly.
The current repo is https://github.com/PCRE2Project/pcre2 and release archive is here: https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.40/pcre2-10.40.tar.bz2 so the last commit by Kirill is the same as my conclusion. As far as CMake goes if anyone is interested I have a port fully working with the current release with exception of https://github.com/PCRE2Project/pcre2/issues/104 which I haven't had time to look into and why I opened a bug report about it.
There is an update to 10.42 now with minor bugfixes. The website (pcre.org) still says PhilipHazel, but that redirects to https://github.com/PCRE2Project/pcre2 (which is what MASTER_SITES now is, so that's fine). So scratch my previous comments about that.
Created attachment 247356 [details] Patch for pcre2 v4 Rework previous to make it more tidy and easier to follow * Move to CMake as it improves build times quite a and bit and integrates better with out framework * Make static library optional * Use upstream's unit testing
Some quick numbers, No STAITC ====> Running Q/A tests (stage-qa) 18.76s real 1m18.84s user 1.93s sys STATIC ====> Running Q/A tests (stage-qa) 33.74s real 2m49.49s user 3.03s sys Autotools ====> Running Q/A tests (stage-qa) 45.90s real 2m52.60s user 7.86s sys We can of course have STATIC as default if its of preferrence This is in general common theme and desired as we can improve build times "for free". See https://lists.freebsd.org/archives/freebsd-ports/2023-December/005056.html and the history where people want to improve build times.
Created attachment 247370 [details] patch for pcre2 v5 Refresh patch
Just wanted to add that a "mini" exp-run looks fine on 13.2-RELEASE (complete dependency list on freshports.org)
Created attachment 248514 [details] Patch for pcre2 v6 Backport upstream PR 366 and 382 https://github.com/PCRE2Project/pcre2/pull/366 https://github.com/PCRE2Project/pcre2/pull/382
Poudriere testport OK 13.2-RELEASE (amd64) Poudriere testport OK 14.0-RELEASE (amd64) Tested with following consumers in 13.2-RELEASE (amd64) using Poudriere: audio/mumble audio/ncmpc audio/zrythm cad/libredwg chinese/tin (fails, broken patch (unrelated)) comms/xastir databases/mariadb1011-client databases/mariadb1011-server databases/mariadb105-client databases/mariadb105-server databases/mariadb106-client databases/mariadb106-server databases/mongodb70 databases/postgis33 databases/postgis34 deskutils/gucharmap devel/R-cran-rJava devel/android-tools devel/anjuta devel/git devel/gitaly devel/glib20 devel/godot devel/godot-tools devel/godot35 devel/godot35-tools devel/libgit2 devel/libnjs devel/poco devel/qt5-core devel/qt6-base devel/shards devel/sord devel/swig devel/universal-ctags devel/util-linux editors/editorconfig-core-c editors/lite-xl editors/mle editors/vscode editors/xmlcopyeditor emulators/vice ftp/proftpd games/naev graphics/epsonscan2 graphics/gdal graphics/gmt irc/unreal lang/crystal lang/julia lang/njs lang/php81 lang/php82 lang/php83 mail/astroid mail/courier (fails, pkg building stage (unrelated)) mail/courier-imap mail/exim mail/exim-ldap2 mail/exim-monitor mail/exim-mysql mail/exim-postgresql mail/exim-sa-exim mail/exim-sqlite mail/fdm mail/imapfilter mail/maildrop mail/mboxgrep mail/mfmod_pcre mail/neomutt mail/postfix mail/postfix-current mail/rspamd mail/rspamd-devel math/R math/libRmath math/octave-forge-strings math/py-rpy2 math/sage (fails, port marked BROKEN) misc/boxes net-mgmt/zabbix6-agent net-mgmt/zabbix6-proxy net-mgmt/zabbix6-server net-mgmt/zabbix64-agent net-mgmt/zabbix64-proxy net-mgmt/zabbix64-server net/frr8 net/frr9 net/haproxy net/haproxy-devel net/haproxy25 net/haproxy26 net/haproxy27 net/haproxy28 net/libyang2 net/remmina net/sslh net/vinagre news/leafnode news/tin science/afni security/aide security/clamav security/clamav-lts security/ossec-hids-agent security/ossec-hids-local security/ossec-hids-server security/snuffleupagus security/sssd2 security/suricata shells/fish sysutils/cfengine-devel sysutils/fdupes sysutils/fsearch sysutils/lnav sysutils/logstalgia sysutils/syslog-ng textproc/confget textproc/gnugrep textproc/link-grammar textproc/ripgrep textproc/sigil textproc/ugrep www/angie www/angie-module-auth-jwt www/angie-module-auth-spnego www/angie-module-brotli www/angie-module-cache-purge www/angie-module-dav-ext www/angie-module-echo www/angie-module-enhanced-memcached www/angie-module-eval www/angie-module-geoip2 www/angie-module-headers-more www/angie-module-image-filter www/angie-module-jwt www/angie-module-keyval www/angie-module-lua (fails, due to dependency (unrelated)) www/angie-module-ndk www/angie-module-njs www/angie-module-perl www/angie-module-postgres www/angie-module-redis2 www/angie-module-rtmp www/angie-module-set-misc www/angie-module-subs www/angie-module-upload www/angie-module-vod www/angie-module-xslt www/apache24 www/bluefish www/edbrowse www/lagrange www/lighttpd www/mod_php81 www/mod_php82 www/mod_php83 www/mod_qos www/nginx www/nginx-devel www/nginx-full www/nginx-lite www/nginx-naxsi www/pound www/restinio www/unit www/unit-java www/unit-perl www/unit-php www/unit-python www/unit-ruby www/unit-wasm www/varnish7 www/xapian-omega x11-fm/thunar x11-toolkits/gtksourceview5 x11-toolkits/termit x11-toolkits/vte3 x11-toolkits/wxgtk32 x11-wm/i3 x11-wm/picom x11-wm/sway x11-wm/sway-devel x11-wm/swayfx x11/elementary-terminal x11/gnome-console x11/gnome-terminal x11/mate-terminal x11/roxterm x11/sakura x11/xfce4-terminal x11/xfce4-verve-plugin
Grr, hit send by accident. Tests of users (consumers) was done on 14.0, there's a testing running on 13.2 here: https://pdr2.bofh.network/build.html?mastername=132-diizzy&build=2024-02-17_11h55m17s
Checks out fine on both 13.2 and 14.0
As I mentioned on IRC yesterday, I'd be good with it, if you could remove cmake for now from your patch
Hi, I submitted issues upstream and patches because I wanted to improve and move forward. Looking at upstream the only BSD-ish like system they're doing testing on is Mac OS using CMake so that also seemed like a step in the right direction ( https://github.com/PCRE2Project/pcre2/blob/master/.github/workflows/build.yml#L66 ). Unfortunately I don't have the time or am interested in re-doing all testing again just for the same of it so I'm going to leave it as-is then. Best regards, Daniel
Created attachment 248623 [details] Patch for pcre v7 No code change, just use upstream commit as PR 382 is now merged
I updated the port to 10.43 but removed cmake part for now
Can you at least fix so we don't need to spend time on regenerating configure scripts etc? ( USES= autoreconf )
Created attachment 251283 [details] Patch for pcre v8 Compile tested on FreeBSD 14.0-RELEASE (amd64) (make, make check-plist, make test) Compile tested on FreeBSD 14.1-RELEASE (aarch64) (make, make check-plist, make test) Poudriere testport OK 13.2-RELEASE (amd64) Poudriere testport OK 14.0-RELEASE (amd64) https://pdr2.bofh.network/build.html?mastername=140-diizzy&build=2024-06-07_11h41m04s Tested with all listed consumers on freshports.org on 13.2-RELEASE (amd64) using Poudriere (looks fine): https://www.freshports.org/devel/pcre2/ https://pdr2.bofh.network/build.html?mastername=132-diizzy&build=2024-06-07_11h40m55s Please consider this as it avoids pullling in deps such as Perl during build
Friendly ping
(In reply to Daniel Engberg from comment #25) could you remove cmake part from your patch?
Created attachment 251657 [details] Patch for pcre v9 Backport upstream commmits: 57906628d7babd27c01eb1c085d3e0cdd512189a a80920509e49865405477244c294fb3d8864375e 3b90149f3c9aca9fe8c2aa623a18e18c089dc449 Reference: https://github.com/PCRE2Project/pcre2/commit/57906628d7babd27c01eb1c085d3e0cdd512189a https://github.com/PCRE2Project/pcre2/commit/a80920509e49865405477244c294fb3d8864375e https://github.com/PCRE2Project/pcre2/commit/3b90149f3c9aca9fe8c2aa623a18e18c089dc449
Are there any concerns I can try to address? If you're concerned is about dependencies, with recent work in ports tree we're now down to about one minute between GNU Autotools or CMake building all dependencies on my Tigerlake laptop.
(In reply to Daniel Engberg from comment #28) ok, let me run some tests locally and I'll come back to it this week, thanks
Created attachment 251694 [details] Patch for pcre2 v10 Backport upstream commits: c63d7c992ef1bbf64ef93e0e8e551ed29fd988e7 7e141969b538917c9f4a85839031d066eec67ad2 9a51f31da1c2d45e1f31d6a0d6b2f62975fed373 References: https://github.com/PCRE2Project/pcre2/commit/c63d7c992ef1bbf64ef93e0e8e551ed29fd988e7 https://github.com/PCRE2Project/pcre2/commit/7e141969b538917c9f4a85839031d066eec67ad2 https://github.com/PCRE2Project/pcre2/commit/9a51f31da1c2d45e1f31d6a0d6b2f62975fed373
I'd like to get rid of patches in Makefile, and rather to use the plain release version.
In that case we need to wait on next release as current is broken on 32-bit
Created attachment 256227 [details] Patch for pcre2 v11
I'd prefer to commit the final release and not release candidates. Could you please stop adding cmake dep for this port, I think we discussed it already a couple of times, thanks.
(In reply to Kirill Ponomarev from comment #11) Yes, we should wait until final release however since one port fails (I did an mini-exp run with direct consumers of pcre2) I needed to provide a patch somehow. I apologize that I didn't add that information sooner. Much have changed since 2020, CMake builds faster now that we've migrated to devel/cmake-core and takes far from 1h to build (autoreconf pulls in its fair share of deps too). Many projects (ports) either are or have migrated to either Meson (which partially pulls in the same dependencies as CMake fwiw and recently also gained support for .cmake project files) or CMake which projects also utilizes .cmake files (provided). Pulling some numbers from freshports shows that we have about about 1000 more utilizing either Meson and CMake since late 2020 and ~450 ports fewer using Autotools. Add that to what Mina and also Adriaan commented on. We can of course provide some hack to configuration files do this with Autotools I guess but it seems counter-productive or if you prefer to work with upstream on that. If you see anything that's broken I can have a look at it if you want but honestly shouldn't we try to move forward and try to cater as many as possible?