Bug 240794 - security/suricata5: update to 5.0.0-rc1 and assorted port corrections
Summary: security/suricata5: update to 5.0.0-rc1 and assorted port corrections
Status: Closed Works As Intended
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Muhammad Moinur Rahman
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2019-09-24 14:37 UTC by Franco Fichtner
Modified: 2019-10-07 16:53 UTC (History)
2 users (show)

See Also:
bofh: maintainer-feedback+


Attachments
5.0.0-rc1 (18.64 KB, patch)
2019-09-24 14:37 UTC, Franco Fichtner
bofh: maintainer-approval-
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Franco Fichtner 2019-09-24 14:37:47 UTC
Created attachment 207772 [details]
5.0.0-rc1

Hello,

security/suricata5 was created without ticket or discussion.  I just found it now and the state of the port is outdated and opportunistic at best.  The name "suricata5" is problematic because we want to update "suricata" to version 5 when that is out...

A list of necessary changes:

* Create a proper PYTHON option (align with security/suricata)
* Set CONFLICTS_INSTALL properly (coexist with security/suricata)
* GEOIP option here is broken based on outdated port option (align with security/suricata)
* Add JSON option (align with security/suricata)
* Add NETMAP option (align with security/suricata)
* Disable HYPERSCAN default (align with security/suricata)
* Fix outdated NSS option (align with security/suricata)
* Fix outdated REDIS option (align with security/suricata)

While here update to 5.0.0.rc1:

https://suricata-ids.org/2019/09/24/please-help-us-test-suricata-5-0-0-rc1/


Cheers,
Franco
Comment 1 Muhammad Moinur Rahman freebsd_committer 2019-09-24 20:03:05 UTC
Nothing to worry about the problematic name. As soon as you update to stable 5 I will giveup on this one.
1. It seemed like PYTHON is a requirement now compared to 4.X. Please confirm as I have faced problems without making PYTHON as default. Maybe we should add USES=python:build.
2. Can you please let me know what is the point of putting NETMAP as an option rather than hardcoding after 11.X where NETMAP is in kernel?
Comment 2 Franco Fichtner 2019-09-25 04:23:06 UTC
We can gladly discuss option removal/defaults for security/suricata.
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-25 11:21:15 UTC
Questions:

- Is suricata 4.x likely to be supported going forward?
- Is there value in creating a suricata4 from suricata and updating suricata to 5.x?

Let's figure out the best path forward as if two ports didn't exist at the moment.

We can then update this issue's summary to handle whatever updates once that's worked out.
Comment 4 Franco Fichtner 2019-09-25 14:52:46 UTC
I guess that 4.1.x will be supported a while longer and may be of use to those who can't use RUST since it will be a hard requirement of Suricata 5.  But if you take a look at the bug tracker there's no open ticket so we won't know until we do if there is actual need.

Still, suricata package should move to version 5 this year; the release is likely at the end of October some time during Suricon.  It would mean we have a quarterly that could potentially run two packages with the same Suricata version.  It's pretty hard not to trip users with this and decision for ports options that deviate from the historic package constraints may create conflicting expectation as to how move forward.

From a security standpoint it's not so great.  Who will push updates even if patches are provided like here?  Time is already ticking for security issues reported with the latest version.  Not to mention 4.1.5 patch waiting for an assignee.

Furthermore, effort is wasted here as I had a separate suricata-devel package in an open source tree since May 2019 and a simple question would have avoided virtually all duplicate work, which could be spent on committing security updates or option alignment as raised here after the fact.

https://github.com/opnsense/ports/commits/master/opnsense/suricata-devel

As an avid contributor, FreeBSD as its sum of people and rules and handbooks place a lot of restrictions on the patches that will go into the tree slowly but steadily. 
 Please forgive my astonishment at the easy way some decisions are handled internally and now we are here to do a post-commit review of something that wouldn't have passed apparent committer standards if it were to be contributed by an outsider.  I simply cannot wrap my head around this.

Anyway, none of this is unsolvable, but all of it requires at least some form of coordination going forward.


Cheers,
Franco
Comment 5 Muhammad Moinur Rahman freebsd_committer 2019-09-25 19:01:29 UTC
Thanks for the details. We understand your frustration and thanks for your contribution to a great project like Opnsense. suricata is do a major part of opnsense hence you are maintaining suricata in the FreeBSD ports tree too. As the base of Opnsense you are using FreeBSD/HardenedBSD and as mentioned that you had the suricata-devel version in your tree, you could have submitted a ticket and it would have been definitely added into FreeBSD tree too by now. Unfortunately it is not always possible to look into multiple projects for same alike works done. 
So far regarding the delay you are talking about committers do have to test vigorously before committing and despite that we do make mistakes. Now I would like to get back to technical points.
1. Can you please explain why do we need to align everything with suricata port? A newer version can come up with more options where we do not need to align with previous version. I understand your perspective as it makes easier for you to update on your side.
2. JSON cannot be an option. It has to be hardcoded. suricata 5.X does not build without libjansson. There are people who use pkg there are people who still use ports. And if I disable JSON it won't build. We do have to test disabling and enabling most of the important options.
3. rules are the real heart of suricata and I don't see a good reason that suricata-update should be optional and dependent on option PYTHON.
4. I can see that opnsense is right now synced with 11.2 of HardenedBSD which has NETMAP built into GENERIC kernel unlike IPFW which requires loadable kernel module; hence IPFW can be an OPTION. So why not remove NETMAP option and make it default?
5. I can see that you are using lots of --with-* for the options which are actually not required as in most of the cases this is taken care by ports Mk framework or USES=pkgconfig unlike libpcap which has two different versions. Is this required specifically for opnsense? 
6. Moving suricata5 to suricata-devel is not an issue. We can do it along with this commit. So when there is 5.X stable we can maybe create suricata4 and update suricata to 5.X. To keep sync with opnsense I would like to move it to suricata-devel now and discuss the possibilities later.
Looking forward for your positive feedback so that I can quickly commit this into the tree.
My builder will take approximately 80 minutes to test and build before I will commit.
Comment 6 commit-hook freebsd_committer 2019-10-05 21:32:31 UTC
A commit references this bug:

Author: bofh
Date: Sat Oct  5 21:31:55 UTC 2019
New revision: 513846
URL: https://svnweb.freebsd.org/changeset/ports/513846

Log:
  security/suricata5: Update version 5.0.0-beta1=>5.0.0-rc1

  - Remove HYPERSCAN as OPTIOINS_DEFAULT

  PR:		240794
  Reported by:	franco@opnsense.org
  Relnotes:	https://suricata-ids.org/2019/09/24/please-help-us-test-suricata-5-0-0-rc1/

Changes:
  head/security/suricata5/Makefile
  head/security/suricata5/distinfo
  head/security/suricata5/pkg-plist
Comment 7 Muhammad Moinur Rahman freebsd_committer 2019-10-05 21:33:43 UTC
For the time being just updated the version and keeping this open for further discussion on aligning this port with suricata.
Comment 8 Franco Fichtner 2019-10-07 16:53:57 UTC
Several insiders suggested to me that FreeBSD policy makes this split work perfectly possible so I don't feel anything of value can be done in the long term.  Thanks for your work.