Created attachment 242926 [details] Patch to add 22.11 port and deprecate old 19.11 one. DPDK LTS releases (the November release each year) have ABI changes and API changes from previous releases. The 19.11 LTS release of DPDK is no longer actively supported so should be deprecated. The latest 22.11 version can be added as a port to replace this. Patch file attached to add new net/dpdk-22.11 port, and mark net/dpdk port as deprecated.
Created attachment 243112 [details] Patch for dpdk-22.11 * Define BUILD and RUNTIME depdency of pyelftools * Add USES= ssl * Change PORTVERSION to DISTVERSION in WRKSRC variable definition * Remove DEBUG option, handled by framework Reference: https://cgit.freebsd.org/ports/tree/Mk/Uses/meson.mk#n51 * Move patch files to files directory
Are current build and runtime dependencies correct? It seems to link with OpenSSL 1.1.X while dpdk claims to require 3.0 during configure stage, upstream bug? During configure stange it looks for a bunch of libraries we have ported, are these needed? For example: Run-time dependency libarchive found: NO (tried pkgconfig) Run-time dependency libpcap found: NO (tried pkgconfig) This one is also in base Run-time dependency libelf found: NO (tried pkgconfig) (not properly detected) Library IPSec_MB found: NO It also defines march on its own which is a no go, -march=corei7 on my Tigerlake box for example. See https://cgit.freebsd.org/ports/commit/emulators/pcsx2?id=10076c84866711af04e6c1425cdc6085ac936bfe for more information Does -O2 vs -O3 make an impact on performance (actually tested)? Best regards, Daniel
[Sorry for delayed reply, been offline for holidays] The libraries such as libarchive, libpcap etc. are all optional in DPDK. I've only included in the build for now the mandatory ones. However, I'll review the list again and see if adding more as dependencies would make sense. With regard to the -march flag, DPDK always requires some architecture setting or instruction set level specified. The default is "native" as it assumes that the user is building to run locally, however, for packaging we need to use a lowest-common-denominator value, which would be "corei7". DPDK requires a CPU with SSE4.2 support. w.r.t. O2 vs O3, I believe we have measured a difference in the past, but I don't have more recent figures. DPDK always errs on side of max performance.
(In reply to Daniel Engberg from comment #2) "It seems to link with OpenSSL 1.1.X while dpdk claims to require 3.0 during configure stage, upstream bug?" Different parts of DPDK have different requirements. Most of DPDK seems to only require 1.1, but the use of IPSec multibuffer library inside the QAT driver in DPDK looks to require >=3.0.0 for that component only.
Created attachment 243462 [details] New revision of patch for DPDK 22.11 Adding new version of patch. This version (vs original): * Takes suggested changes from WIP version submitted by reviewer * Patches DPDK config/meson.build file to detect presence of libarchive on freebsd without a pkg-config file * Added some extra dependencies to the list to get more components building and installing. (Requesting libpcap from ports rather than build-in since DPDK uses pkg-config file from v1.9 to detect pcap) * Fixed use of the CPUTYPE based on suggested approach: use default of corei7 for SSE4.2 support if no explicit CPUTYPE given, otherwise use explicitly-provided CPUTYPE.
Looks good, we usually try to use libarchive from ports which provides a pkgconfig pc file and is also more up to date with upstream which might be an alternative?
We don't make massive use of libarchive, so working with the version in base is probably fine. However, I'm ok to put a dependency on the libarchive port too. Is there an easy/standard way to specify that a port depends unconditionally up on another port? In current makefile I'm using LIB_DEPENDS and specifying library names, but that won't work if a package is in base, unless - like with pcap - I start specifying particular .so versions (which I'd really rather not do).
The libarchive question I consider not an issue for this port, since it should work fine without a ports dependency as I state in my earlier reply. Can this patch now be merged to get us an updated DPDK release in ports, please? If not, please let me know what the outstanding issues are. Thanks.
Sorry for the late reply, thanks for pinging! plist fails and also seems to install static libs despite only shared being set? It's not the end of the world but perhaps worth taking note of at least. ====> Checking for pkg-plist issues (check-plist) ===> Parsing plist ===> Checking for items in STAGEDIR missing from pkg-plist Error: Orphaned: lib/dpdk/pmds-23.0/librte_compress_isal.so Error: Orphaned: lib/dpdk/pmds-23.0/librte_compress_isal.so.23 Error: Orphaned: lib/dpdk/pmds-23.0/librte_compress_isal.so.23.0 Error: Orphaned: lib/librte_compress_isal.a Error: Orphaned: lib/librte_compress_isal.so Error: Orphaned: lib/librte_compress_isal.so.23 Error: Orphaned: lib/librte_compress_isal.so.23.0 ===> Checking for items in pkg-plist which are not in STAGEDIR Error: Missing: lib/dpdk/pmds-23.0/librte_crypto_openssl.so Error: Missing: lib/dpdk/pmds-23.0/librte_crypto_openssl.so.23 Error: Missing: lib/dpdk/pmds-23.0/librte_crypto_openssl.so.23.0 Error: Missing: lib/librte_crypto_openssl.a Error: Missing: lib/librte_crypto_openssl.so Error: Missing: lib/librte_crypto_openssl.so.23 Error: Missing: lib/librte_crypto_openssl.so.23.0 ===> Error: Plist issues found. *** Error code 1 Compiles file on 12.4, 13.2 and 14.0-RC and looks good otherwise from what I can tell.
For static libs, yes, those are always built and installed by DPDK, irrespective of the meson build type. The meson build type only controls the linkage of the applications, whether e.g. dpdk-testpmd is statically linked or linked against the shared libs. This is partially for historical reasons, which I won't get into, at this point. For the plist issues, I will investigate. I didn't see them in my checks - some of them may be due to libraries I have installed on my system that I missed including as dependencies of the dpdk package.
I can't seem to replicate the issue with the missing "librte_crypto_openssl" files in the build. Can you perhaps send me on the build log from that run? From checking the build files, that openssl crypto driver should be built so long as libcrypto is found by pkg-config, and on my systems tested, it's always found in /usr/libdata/pkgconfig. If libcrypto.pc file missing on the system you ran that plist test on? If so, is there some way to get a copy of it installed as a pre-requisite of the port? Explicitly requesting a copy of openssl with libcrypto from ports causes other (very strange) issues with the build. [The kmods stop linking for some reason!]
Hi, Here's a build log (13.2-RELEASE) https://pdr2.bofh.network/data/132-diizzy/2023-11-04_08h24m37s/logs/errors/dpdk22.11-22.11.2.log Thanks for looking into it Best regards, Daniel
..and on 12.4 https://pdr2.bofh.network/data/124-diizzy/2023-11-04_10h14m08s/logs/errors/dpdk22.11-22.11.2.log
(In reply to Daniel Engberg from comment #13) I see it's FreeBSD 12 that has the issue of the missing openssl crypto driver. On testing FreeBSD 12, this appears to be because /usr/lib/libcrypto.so exists but there is no pkg-config file for it. I'm not sure how best to fix this issue, because even putting a dependency on the lib won't help, since the lib is present just the .pc file is absent. One solution that would work, but is definitely sub-optimal is just disabling that driver in the build and omitting it from the plist file. However, that seems overly severe, depriving those using later BSDs of a working driver. Is there any way to specify in the plist file that certain objects are to be only installed for certain BSD versions or when certain conditions are met? [For the other plist issue of unlisted compress_isal driver, that seems just to be an omission on my part, though I'm not sure how/why I didn't see it before. It's an easy fix to just add the entries]
Given that FreeBSD 12 is soon to be eol I guess we could mark it as "BROKEN" on 12 is that's ok with you.
(In reply to Daniel Engberg from comment #15) not really optimal, and seems overkill to mark the whole port as broken because one driver out of dozens fails to build on BSD 12. However, given the EOL is indeed close, i suppose this will be acceptable. Anyone really wanting DPDK on BSD 12 can always compile from source directly anyway.
Do you need a new patch revision with the additional plist entries for these files added to it? Error: Orphaned: lib/dpdk/pmds-23.0/librte_compress_isal.so Error: Orphaned: lib/dpdk/pmds-23.0/librte_compress_isal.so.23 Error: Orphaned: lib/dpdk/pmds-23.0/librte_compress_isal.so.23.0 Error: Orphaned: lib/librte_compress_isal.a Error: Orphaned: lib/librte_compress_isal.so Error: Orphaned: lib/librte_compress_isal.so.23 Error: Orphaned: lib/librte_compress_isal.so.23.0
Hi, I'll finish it up during the week so we can finally get it in. Thanks for your patience. Best regards, Daniel
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=ab116e7b61789ba9abb7b157fd1ad41f977861e1 commit ab116e7b61789ba9abb7b157fd1ad41f977861e1 Author: Bruce Richardson <bruce.richardson@intel.com> AuthorDate: 2023-11-18 09:10:22 +0000 Commit: Daniel Engberg <diizzy@FreeBSD.org> CommitDate: 2023-11-18 09:24:50 +0000 net/dpdk-22.11: New port: Data Plane Development Kit 22.11 (LTS) DPDK is the open source Data Plane Development Kit that consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures. It provides a set of data plane libraries and network interface controller polling-mode drivers for offloading TCP packet processing from the operating system kernel to processes running in user space. This offloading achieves higher computing efficiency and higher packet throughput than is possible using the interrupt-driven processing provided in the kernel. WWW: https://www.dpdk.org/ Changelog: https://doc.dpdk.org/guides/rel_notes/release_22_11.html PR: 272133 net/Makefile | 1 + net/dpdk-22.11/Makefile (new) | 62 + net/dpdk-22.11/distinfo (new) | 3 + .../files/patch-config_meson.build (new) | 12 + ...atch-kernel_freebsd_contigmem_contigmem.c (new) | 20 + .../patch-kernel_freebsd_nic__uio_nic__uio.c (new) | 18 + net/dpdk-22.11/pkg-descr (new) | 15 + net/dpdk-22.11/pkg-message (new) | 30 + net/dpdk-22.11/pkg-plist (new) | 1356 ++++++++++++++++++++ 9 files changed, 1517 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=91828ddfe49ca5c8de4636c1d30b048f63bae405 commit 91828ddfe49ca5c8de4636c1d30b048f63bae405 Author: Daniel Engberg <diizzy@FreeBSD.org> AuthorDate: 2023-11-18 09:23:16 +0000 Commit: Daniel Engberg <diizzy@FreeBSD.org> CommitDate: 2023-11-18 09:24:50 +0000 net/dpdk: Deprecate and set expiration date to 2023-12-31 End of life since December 2022 upstream, point users to net/dpdk-22.11 Reference: https://core.dpdk.org/roadmap/ PR: 272133 Reported by: Bruce Richardson <bruce.richardson@intel.com> net/dpdk/Makefile | 3 +++ 1 file changed, 3 insertions(+)
Thanks for your patience and of course for your contribution Bruce!