Bug 237830 - security/vuxml: split vuln.xml by year, use includes to read them, test tools if all is fine
Summary: security/vuxml: split vuln.xml by year, use includes to read them, test tools...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Ports Security Team
URL:
Keywords: dogfood, needs-patch, needs-qa
Depends on:
Blocks:
 
Reported: 2019-05-10 14:06 UTC by Kurt Jaeger
Modified: 2020-07-23 15:18 UTC (History)
5 users (show)

See Also:
bugzilla: maintainer-feedback? (ports-secteam)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kurt Jaeger freebsd_committer freebsd_triage 2019-05-10 14:06:06 UTC
commits to vuln.xml cause size-problems in SVN.

And editing an ever-growing file does not sound wise, either.

To ease the handling:
- split vuln.xml by year
- use XML includes to read the yearly files
- test tools if this works

Volunteers ?
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2019-05-14 02:48:38 UTC
Freshports may parse this in a custom manner, cc Dan accordingly

What other tools / people may be affected?
Comment 2 Dan Langille freebsd_committer freebsd_triage 2019-05-14 03:18:16 UTC
FreshPorts reads the file touched in the commit.

If the individual files are self-enclosed XML, FreshPorts might not notice.

If someone can show me example files / commits, I will verify.

Sounds like a very good BSDCan project.
Comment 3 Dan Langille freebsd_committer freebsd_triage 2019-05-14 14:55:18 UTC
(In reply to Dan Langille from comment #2)
I scanned the code. There are a few constants referring to 

* /ports/head/security/vuxml/vuln.xml
* security/vuxml/vuln.xml

and one bit of code which scans the incoming email via perl for:

        while (<VUXML>) {
                next unless m#^\s+\$FreeBSD: head/security/vuxml/vuln.xml (\d+) (\d\d\d\d\-\d\d\-\d\d) (\d\d:\d\d:\d\d\S+) (\S+) .*$#;
                $ident{Revision}  = $1;
                $ident{Date}      = $2;
                $ident{Time}      = $3;
                $ident{Committer} = $4;
        }

        return %ident;

Nothing insurmountable.

FreshPorts processes the commit, and then queues a job for the vuxml processing (i.e. tagging vuln versions) which is done later. If vuxml processing breaks, it does not upset the website; only the 'skulls' next to version numbers is affected.
Comment 4 Jochen Neumeister freebsd_committer freebsd_triage 2020-07-23 15:18:13 UTC
The problem is "pkg audit foo". We are currently discussing a solution with secteam to make vuxml modern. I am closing here, but of course suggestions are welcome.