Bug 276395 - net/scamper: update to 20240229
Summary: net/scamper: update to 20240229
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Robert Clausecker
URL: https://mailman.caida.org/pipermail/s...
Keywords:
Depends on:
Blocks:
 
Reported: 2024-01-17 02:06 UTC by mjl
Modified: 2024-03-19 14:27 UTC (History)
1 user (show)

See Also:


Attachments
update net/scamper to 20240117 (3.89 KB, application/mbox)
2024-01-17 02:06 UTC, mjl
mjl: maintainer-approval+
Details
update net/scamper to 20240122 (3.84 KB, patch)
2024-01-23 21:50 UTC, mjl
mjl: maintainer-approval+
Details | Diff
update net/scamper to 20240229 (2.14 KB, patch)
2024-02-29 20:38 UTC, mjl
mjl: maintainer-approval+
Details | Diff
update net/scamper to 20240229, add net/py-scamper (3.90 KB, patch)
2024-03-17 23:21 UTC, mjl
mjl: maintainer-approval+
Details | Diff
add net/py-scamper (1.77 KB, patch)
2024-03-19 07:13 UTC, mjl
mjl: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description mjl 2024-01-17 02:06:10 UTC
Created attachment 247716 [details]
update net/scamper to 20240117

Update net/scamper to 20240117.  Add net/py-scamper.

Changelog:

scamper:
* fix crashes if a task is provided an apparent response, but the task
  has not yet sent a packet in ping, sting, tbit, trace, tracelb.
* find and fix memory leaks in measurement command parsing code via
  fuzzing.
* add udpprobe, http, dealias -m midarest, and dealias -m midardisc methods.
  udpprobe and http do not currently have json output methods.
* remove -d dport, -s sport, and -t ttl dealias parameters.  these
  were only valid for mercator, and can be specified in a probedef instead.
* record size of responses for dealias probes.
* modify all commands to be able to take human-friendly wait parameters --
  e.g. trace -W 0.5s or trace -W 500ms instead of trace -W 50000.
* support transmitting raw packets on linux ppp interfaces
* add -b payload-size parameter to ping

python interface:
* add python interface to most of scamper.  building this module requires
  cython < 3.0.  to build, use something like:
  PYTHON=/path/to/python CYTHON=/path/to/cython ./configure --with-python
* the documentation for this interface is available at
  https://www.caida.org/catalog/software/scamper/python/
* the ubuntu PPA provides python3-scamper, and the FreeBSD port should
  hopefully provide py-scamper shortly.  these sources may be useful
  for getting the module packaged for other operating systems.
  https://launchpad.net/~matthewluckie/+archive/ubuntu/scamper

libscamperfile and libscamperctrl:
* improvements, bump versions

tests:
* many new unit tests and fuzzing cradles for scamper components

sc_pinger:
* add -l limit and -M move-dir options

sc_remoted:
* no need to link against libscamperfile, remove dependency

sc_attach, sc_filterpolicy, sc_prefixscan, sc_wartsfix:
* compile / link on windows.
Comment 1 mjl 2024-01-23 21:50:01 UTC
Created attachment 247904 [details]
update net/scamper to 20240122

compared to the 20240117 patch, this patch does not require cython to be installed to build the python module, and has a more recent version of scamper.
Comment 2 mjl 2024-02-29 20:38:54 UTC
Created attachment 248831 [details]
update net/scamper to 20240229

Compared to the previous patch a month ago, I've left the py-scamper port out, so that this update is simple, in the hope of moving this forward.

Changelog from 20240122:

https://mailman.caida.org/pipermail/scamper-announce/2024-February/000040.html

scamper:
* bugfix: correct SCAMPER_DL_IS_ICMP_TIME_REPLY macro so that scamper
  does not ignore ICMP replies quoting TCP/UDP packets for 1/256
  source port values.
* bugfixes for -O icmp-rxerr: handle EPOLLERR on linux, try using
  IPV6_RECVHOPLIMIT before IPV6_HOPLIMIT for IPv6 udp probing.
* bugfix: fix possible assertions in dealias, trace, and tracelb code.
* ping: count UDP replies as from the target
* trace: correct datalink timestamp adjustments
* add more specific error messages when a scamper command is not
  accepted, return them over control socket or terminal, as appropriate.
* scamper_fd_t: refactor fd_addr, fd_sport, fd_dl, fd_ifindex
* scamper_fd_t: decouple udp4raw/udp4dg from each other.  use udp4dg
  and udp6 sockets in udpprobe primitive, rather than have each
  udpprobe task create its own socket.
* scamper_fd_t: add methods that return an available open socket for
  probing a given destination without blocking if another task is
  probing the same destination.  use this with ping -F 0 and trace -s 0,
  instead of generating random port values, so that scamper opens
  the minimum number of file descriptors.
* ping and trace: have -O dl mean to capture transmit and receive
  timestamps from a datalink interface.  add -O raw option so that
  IPv4 TCP probes can be transmitted over a raw socket, to allow transmitted
  TCP packets to be observed on the datalink interface.
* add memory-mapped ring buffer support (PACKET_RX_RING) for receiving
  packets on datalink sockets on Linux.  enable at scamper runtime
  with -O ring.  contributed by Alistair King.

python module:
* generate module .c with Cython 0.29.37, to enable module to compile
  with Python 3.11.
* link against libpython
* have all time parameters take either a timedelta or float/int.  float/int
  parameters are all seconds.
* add ScamperInstError exception which identifies the scamper instance
  and includes the error string returned by the instance.
* reduce copying in methods that return bytes objects
* add str method for ScamperInst

libscamperctrl:
* pass error strings through, bump library version

libscamperfile:
* add new functions, bump library version

tests:
* add unit tests for scamper_fd
* add more timeval unit tests
* add more cmd_dealias and cmd_ping tests
Comment 3 Robert Clausecker freebsd_committer freebsd_triage 2024-03-14 09:50:04 UTC
I apologise for the slow processing.  I'm currently busy with other stuff and it seems nobody picked up the slack.

Will commit this one with my next batch.
Comment 4 mjl 2024-03-15 00:23:05 UTC
Thanks.  While the current patch doesn't have the py-scamper port, if you were able to provide feedback on the prior patch which did have the py-scamper port, then I can act on that feedback for the next update.
Comment 5 Robert Clausecker freebsd_committer freebsd_triage 2024-03-15 07:24:27 UTC
(In reply to mjl from comment #4)

Hi mjl,

The py-scamper patch looks like it should be ok, but I didn't have time to build-test it yet.  Feel free to add a new patch with the new port.  If you can do it this weekend, I should be able to include it in my next batch of commits.
Comment 6 mjl 2024-03-17 23:21:09 UTC
Created attachment 249247 [details]
update net/scamper to 20240229, add net/py-scamper

This version of the patch includes the py-scamper port.
Comment 7 Robert Clausecker freebsd_committer freebsd_triage 2024-03-18 12:29:51 UTC
(In reply to mjl from comment #6)

Hi mjl,

Could you please make the new port a separate patch that applies on top of attachment #248831 [details]?  I already have the first patch tested and ready to be committed.  It would also be better to separate the two actions of “update port” and “add new port” so they can be individually reverted if needed.
Comment 8 mjl 2024-03-19 07:13:51 UTC
Created attachment 249296 [details]
add net/py-scamper

This is a git diff between the two branches I created for the two patches that I have not made obsolete (248831, 249247).  This patch adds net/py-scamper, and modifies net/scamper so that py-scamper can be a slave port from net/scamper.
Comment 9 Robert Clausecker freebsd_committer freebsd_triage 2024-03-19 07:16:32 UTC
(In reply to mjl from comment #8)

Thank you.  I think I may just be able to get this one in.
Comment 10 commit-hook freebsd_committer freebsd_triage 2024-03-19 14:21:25 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=499be1d04c11c07486e50b9befb72c624ad2e7e2

commit 499be1d04c11c07486e50b9befb72c624ad2e7e2
Author:     Matthew Luckie <mjl@luckie.org.nz>
AuthorDate: 2024-03-19 07:27:34 +0000
Commit:     Robert Clausecker <fuz@FreeBSD.org>
CommitDate: 2024-03-19 14:18:52 +0000

    net/py-scamper: new port

    This port has the Python bits of net/scamper.

    PR:             276395

 net/Makefile                  |  1 +
 net/py-scamper/Makefile (new) | 36 ++++++++++++++++++++++++++++++++++++
 net/scamper/Makefile          |  8 +++++++-
 3 files changed, 44 insertions(+), 1 deletion(-)
Comment 11 commit-hook freebsd_committer freebsd_triage 2024-03-19 14:21:39 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=d9345dee0c7ea27905c9a3bbad9925a4bffd9d18

commit d9345dee0c7ea27905c9a3bbad9925a4bffd9d18
Author:     Matthew Luckie <mjl@luckie.org.nz>
AuthorDate: 2024-02-29 20:33:26 +0000
Commit:     Robert Clausecker <fuz@FreeBSD.org>
CommitDate: 2024-03-19 14:18:23 +0000

    net/scamper: update to 20240229

    PR:             276395

 net/scamper/Makefile  | 5 ++---
 net/scamper/distinfo  | 6 +++---
 net/scamper/pkg-plist | 8 +++++---
 3 files changed, 10 insertions(+), 9 deletions(-)
Comment 12 Robert Clausecker freebsd_committer freebsd_triage 2024-03-19 14:27:18 UTC
Thank you for your contribution.