Created attachment 236566 [details] git diff during the package target i got the following error: pkg-static: Unable to access file /usr/ports/security/snort3/work/stage/usr/local/share/doc/snort/config_changes.txt:No such file or directory i patched the port Makefile as attached for a functioning package.
Are you using ports, packages or poudriere? As I cannot reproduce this. These are my latest logs: https://pdr.bofh.network/data/latest-per-pkg/snort3/3.1.41.0,1/
thank you for the prompt reply! i was using plain old ports tree, at commit 767f874aeca9. i'll run it again and add attach a log later today. any other investigation needed i'll be happy to help. -pfy root@solo:/usr/ports/security/snort3 # diff <(env PORT_DBDIR=/var/empty make showconfig) <(make showconfig) | grep '^>' > ADDRESSSANITIZER=on: Enable address sanitizer > LARGEPCAP=on: Enable support for pcaps larger than 2 GB > THREADSANITIZER=on: Enable thread sanitizer > TSC=on: Use timestamp counter register clock (x86 only) root@solo:/usr/ports # cat /etc/make.conf CPUTYPE?=opteron-sse3 DEFAULT_VERSIONS= bdb=18 gcc=12 java=18 llvm=14 lua=5.4 mysql=8.0 nodejs=18 perl5=5.36 pgsql=14 php=8.1 python=3.10 python3=3.10 ruby=3.1 samba=4.13 ssl=openssl root@solo:/usr/ports # uname -a FreeBSD solo.bofh.vip 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 #0 releng/13.1-n250158-752f813d6cc-dirty: Sun Sep 11 17:58:33 UTC 2022 root@solo.bofh.vip:/usr/obj/usr/src/amd64.amd64/sys/NUOS amd64 (fyi the "-dirty" above is merely a tiny manpage patch which can be found at PR #243284) root@solo:~ # cat /usr/src/sys/amd64/conf/NUOS include GENERIC ident NUOS options VIMAGE options TCPHPTS options RATELIMIT
Created attachment 236610 [details] stdout+stderr: make clean package install (*SANITIZER turned off for this build, LARGEPCAP+TSC are still on)
Created attachment 236611 [details] stderr: make clean package install same build, stderr only, if it helps. i'm not seeing anything to go on i'll run a make now with just default options
Created attachment 236612 [details] env PORT_DBDIR=/var/empty BATCH=yes make clean package install same error. fyi, yes `make missing` outputs the empty string. i have 2041 pkgs installed. i'll run the build in a basic jail later tonight and if anything runs differently i'll let you know
I have tried to build it again with the non-default options mentioned here: > ADDRESSSANITIZER=on: Enable address sanitizer > LARGEPCAP=on: Enable support for pcaps larger than 2 GB > THREADSANITIZER=on: Enable thread sanitizer > TSC=on: Use timestamp counter register clock (x86 only) There was an error with TSC on which I have fixed and committed. But for you my advise will be to remove snort3 entirely with "pkg remove snort3" and install it from scratch. Are you using any other software like portmaster or portupgrade? I believe it's messing up with the upgrade procedure. I couldn't reproduce the error you have mentioned though.
So YES, I successfully built/installed the package made from the clean port with no error (or at least no stoppage) using a clean base jail (first using pkg add to install the 72 direct and indirect dependencies copied in from the host, which are all built from ports with the make.conf i put in prior comment, many with custom options). Meaning you're not crazy and your port must work fine for most people. This means you can relax for now. Thank you again for looking into this so quickly; i find you neither a bastard nor from hell, rather a gentleman and a scholar (your secret is safe with me). As your loyal pimply faced youth i'll continue using process of elimination to see what about my environment caused my error with the port. No, i'm not using portmaster or portupgrade, just /bin/csh and /usr/bin/make with the ports tree 'main' branch from git.FreeBSD.org/ports. When i initially experienced and reported this error it was from a system that had never before had snort3 nor snort, neither installed nor built. While making the attached logs with your clean port on my dirty system (as of ports commit #767f874aeca9, i.e. snort3 commit #540c7abd52cd) i tried doing so both with and without the snort3-3.1.41.0,1 package installed during the 'package' make target (the one i had packaged using my patch attached to this PR). Either way, it stopped in failure with the same error at the same point. I'll keep hunting the underlying issue over the weekend. Good day and God bless to you.
Okay, so far i've determined that Snort 3.1's configure/cmake detects and attempts to use libunwind generally from my dirty system and w3m while DOCS is enabled. The result is one or both of those is mucking with the build and/or package target in the snort3 port. Can you tell me if you've already considered optionalizing and registering libunwind with the port and decided against it or if it's an oversight? I will now see how removing one or the other affects the packaging, and further investigate the differences affected in building and staging by their existence. Until hearing your feedback to my question above, i'm intent on adding a working libunwind option and preventing any vestigial html (or pdf) doc producing steps in the distribution source. (i see `pkg info -l snort3 | egrep (pdf|html)` is currently empty despite the port having set MAKE_HTML_DOC and MAKE_PDF_DOC cmake bools). However i imagine anyone making use of *SANITIZER options would make use of backtraces as well.
(In reply to Chad Jacob Milios from comment #7) There is always a BOFH mode and the triggering solely depends on the person who is on the other end of the phone. 3:) (In reply to Chad Jacob Milios from comment #8) Yes I did try and I don't have the energy to try that once more as it heavily deteriorates the snort performance on FreeBSD plus netmap combination. I have absolute no clue why. But if you can share some proof/test that it is no longer the case I would be happy to add it. And I would actually need a proof of at least 2-3 gbps traffic handling on a 10G nic. I have not used any sanitizers in our use cases.
Okay so my comment #8 was in response to finding a red herring. It's actually the presence of Ruby on my dirty system, along with the requested DOCS option, which results the error originally filed for this PR. Turning off DOCS or removing Ruby causes the clean port to successfully package as plist'ed. I'll look into what this file config_changes.txt is even used for. (It sure looks a lot more like code than human sensical text.) If RUBY_EXECUTABLE is found then the install target from inside $WRKSRC omits config_changes.txt from being installed at all, at least in the way that it's being invoked from the port. Then upon further investigation i realized that config_changes.txt looks like an elaborate mapping table, seems only to be used programmatically by the snort2lua utility. At build or upon run? I'll have to look a little deeper and determine exactly how snort2lua builds and operates to be sure about that statement. At first glance i thought the omission of config_changes.txt from the installed docs dir implies that Snort developers decided that retaining doc/upgrade/config_changes.txt for human reference or enjoying access to a working snort2lua utility are mutually exclusive conditions. I for one reject that mentality but i'll accept upstream's decision at this juncture if it's indeed what they intend. Still just a weak theory shrouded in confusion. If this line of reasoning is true then it seems Ruby as a dependency ought not be conditional on DOCS, despite config_changes.txt and get_differences.rb living in a directory called "docs". ¯\_(ツ)_/¯ At this point i wonder what you'd think is the better way forward: unconditionally depending on Ruby (run/build/both, depending on how snort2lua functions) or adding a default SNORT2LUA option (or call it MIGRATION_TOOL, or whatever) which pulls in Ruby and to prepare snort2lua or skips snort2lua otherwise. However, as i write this i'm also completing more test builds by which i reach the following information which only adds to my confusion: (note: Host has Ruby, Jail does not. these are using clean port from #1e69945e6906. I added libunwind to Jail for these tests to normalize for Ruby as that was less disruptive than removing libunwind from Host; that's another issue entirely.) After `make stage`, $WRKSRC.Jail and $WRKSRC.Host are _identical_ in all form and content, $STAGEDIR.Jail and $STAGEDIR.Host differ ONLY by the one file config_changes.txt existing in $STAGEDIR.Jail but not $STAGEDIR.Host, and $STAGEDIR.Jail/$PREFIX/share/doc/snort/config_changes.txt == $WRKSRC.*/doc/upgrade/config_changes.txt exactly. So it seems currently we don't really have any buildtime requirement for Ruby despite doc/upgrade/CMakeLists.txt conditionalizing on RUBY_EXECUTABLE and its mere existence somehow influencing the port stage target to cause the inverse existence of but one lowly unmodified txt file. Futhermore, i cannot find any file named *.rb nor can i match /ruby/i among any file or symbol in $STAGEDIR.* nor $WORKSRC.* so i think that pretty much rules out the idea of a runtime dependency as well. What. The. Fork. I'm tempted to try `make package` two more times from Jail (still without Ruby) but first merely creating /usr/bin/ruby linked from /usr/bin/{true,false} to see if that matters. My hypothesis being that both attempts then would stage and log identically as Host has (i.e. failing to produce/install config_changes.txt) and thus fail packaging. However just as I begin to try that out, a Florida thunderstorm blinks power a few times and solo.bofh.vip (aka Host) remains lifeless for the moment and i'm uninspired to move my keister right now as i type this on a laptop. My WiFi returns unattended so i'll send this out and maybe try that later. At that point i figure that if it's decided we do want to install config_changes.txt then you should probably apply my patch to Makefile or alternately patch a short circuit over the check on RUBY_EXECUTABLE so that the distfile's `make install` yields the same result. (If you're still wondering, i conclude the detection of w3m only influenced the build log in the most trivial way and had no bearing whatsoever on the build process nor the result, neither $WRKSRC nor $STAGEDIR.) Contemplating further, i consider that perhaps we aren't even staging docs/snort precisely correct at present. If Snort does install config_changes.txt unmodified in the absence of Ruby by simply adding it to UNBUILT_SOURCES, yet they've gone out of their way to check for Ruby, include a lone Ruby script in their tarball, then conditionally append it to BUILT_SOURCES, maybe we're supposed to be triggering that custom_command to get a processed and meaningfully modified config_changes.txt. (There are a couple other BUILT_SOURCES missing from doc/snort that appear in doc/upgrade/CMakeLists.txt, though I can certainly see why you'd forego such unexciting files as: version.txt containing trivial output from `snort -V` and snort2lua_cmds.txt containing the output of `snort2lua --help`.) At this point i'm far beyond my depth because not only can i never see myself using the snort2lua migration utility for any reason, i've never actually even used CMake at all for anything of my own, nor do i think i've ever worked on any significant port of any distfile that does, or had i, it must've been entirely unmemorable for having no cause that i look inside; i wouldn't deign to guess the proper way to trigger such custom_command entries from the appropriate port stage. I'll study up some CMake porter's-fu unless you first share better insight from where i leave off with this message. I'm tempted to roll out and spin up an Arch, Debian or Mint just to compare how Snort 3 deploys into those systems and detect what mutation of contents this elusive file may contain. If it comes to that, perhaps you'd suggest a Linux distro or three of some repute you'd be more curious to know the situation upon; they're all equally trash as far as this pimply faced youth is concerned. In unrelated news, Snort's CMake scripts are automatically finding libunwind on the build system and successfully linking against it while exposing no boolean, obviously without registration from the port. We definitely need to kibosh such behavior especially if what you say in comment #9 still holds true. I'll go ahead and add an option for that and file that in a new PR for you very soon, or you can beat me to it.
Correction: last comment, paragraph 9: cannot find any file named *.rb __ in $STAGEDIR.* __ nor can i match /ruby/i among any file or symbol in $STAGEDIR.* nor $WORKSRC.*
So i just learned from Porter's Handbook section 6.5.4 i probably ought to have checked $WRKDIR/.build as well for the sort of artifacts i sought to find in $WRKSRC. I'll see if that turns up any new information counter to what i said previously regarding the contents of $WRKSRC between different experimental runs.
Created attachment 236654 [details] generates config_changes.txt using Ruby script from $WRKSRC It seems that get_differences.rb sure does make significant changes to config_changes.txt. What the ramifications of those changes might be, how they are decided and why the distfile version is so different but similar is beyond my depth. There surely is a more appropriately proper way to invoke CMake to trigger the custom_command in doc/upgrade/CMakeLists.txt. If you figure that out, maybe the thorough thing would then be to also generate version.txt and snort2lua_cmds.txt while you're at it. I'm not doing all that here. I simply grokked the custom_command which outputs config_changes.txt and recreated the equivalent command in the port Makefile. I also grokked the Ruby script to establish that no funny stuff that CMake tends to set has any bearing; no accessing ENV or Dir.pwd and friends.
Created attachment 236655 [details] the results of the Ruby script relative to the file already in $WRKSRC Here's how the file in question mutated once I `make package` with the latest attached patch
Keyword: patch or patch-ready – in lieu of summary line prefix: [patch] * bulk change for the keyword * summary lines may be edited manually (not in bulk). Keyword descriptions and search interface: <https://bugs.freebsd.org/bugzilla/describekeywords.cgi>
(In reply to Chad Jacob Milios from comment #14) I think I have missed the trail somehow but wanted to confirm whether if this error still persists as I have already committed your linunwind patch.