Bug 227502 - Unable to add pfsense as monitored target in ntopng
Summary: Unable to add pfsense as monitored target in ntopng
Status: Closed Works As Intended
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Hartmut Brandt
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-04-13 16:10 UTC by Florin Samareanu
Modified: 2018-05-14 10:58 UTC (History)
3 users (show)

See Also:


Attachments
Pcap capture of successful and failed requests (705 bytes, application/gzip)
2018-04-13 16:10 UTC, Florin Samareanu
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Florin Samareanu 2018-04-13 16:10:58 UTC
Created attachment 192498 [details]
Pcap capture of successful and failed requests

While trying to add a host running pfsense in ntopng using SNMP for traffic graphing I get an error. Troubleshooted this with ntopng team in https://github.com/ntop/ntopng/issues/1734, reported to pfsense team and was directed to FreeBSD bug tracker. Attached is a pcap capture of a successful snmpwalk and a failed try in ntopng.
I am running latest pfsense stable with bsdsnmp. If I switch to net-SNMP the target is added successfully.
Comment 1 Eugene Grosbein freebsd_committer freebsd_triage 2018-04-14 21:38:15 UTC
I've reproduced the problem building and running third-party/snmp/test.c from net/ntopng source tree. I run bsnmpd in debug mode:

/usr/sbin/bsnmpd -p /var/run/snmpd.pid -d -D dump,trace=0x30000000

Incoming SNMPv1 GetRequest as captured and decoded by tcpdump:

04:15:32.993260 IP (tos 0x0, ttl 62, id 21558, offset 0, flags [none], proto UDP (17), length 81)
    X.X.X.X.46351 > X.X.X.X.Y: [udp sum ok]  { SNMPv1 C="xxxxxxx" { GetRequest(34) R=1  .1.3.6.1.2.1.1.5.0 } }

bsnmpd fails to parse it producing errors:

snmpd[45132]: ASN.1: non-minimal integer at 00 00 00 00 04 07 72 65 77 6f 72 74 68 a0 22 02 04 00 00 00 01 02 04 00 00 00 00 02 04 00 00 00 00 30 0e 30 0c 06 08 2b 06 01 02 01 01 05 00 05 00
snmpd[45132]: SNMP: cannot decode version

ntopng uses bundled copy of library https://github.com/ejrh/snmp to encode SNMP data into packets and this library seems to produce incorrect DER/ASN.1 packets always encoding integers with 4 bytes per value. The library itself is pretty old, it was not updated for 6 years.

snmpwalk, on the other hand, produces correct requests and bsnmpd answers just fine.

It seems, net-snmpd tolerates such standard violation but bsnmpd does not. Please note that other modern software tend to stick to strict validation too. 
For example, golang's library encoding/asn1 rejects such invalid "non-minimal integer encodings" since version 1.7: https://golang.org/doc/go1.7
Comment 2 Florin Samareanu 2018-04-17 11:33:01 UTC
Hello,

I got this in the ntopng bug I raised ( https://github.com/ntop/ntopng/issues/1734 ):

@fsamareanu sure. In SNMP some fields are encoded with TLV (Type Length Value), where the length specifies how many bytes to read on the next field. The daemon you are using seems to ignore the "Length" value from some SNMP fields (at least in version, error-status, error-index fields) and assume length 1. This works well with snmpwalk which uses a 1 byte values but it does not with ntopng which uses 4 bytes values.
Comment 3 Eugene Grosbein freebsd_committer freebsd_triage 2018-04-17 12:10:24 UTC
(In reply to Florin Samareanu from comment #2)

The problem is more complex. Indeed, "In SNMP some fields are encoded with TLV (Type Length Value), where the length specifies how many bytes to read on the next field" - that's true. But ASN.1/DER encoding standard states that if integer value is small enought to be fit in single byte, it MUST be encoded with single byte and using 4 bytes is not allowed for such case by this standard.

snmpwalk can send small or large integers and it encodes them just right using noted "minimal integer" encoding: 1 or more bytes correspongingly. bsnmpd parses both of small and large TLV fileds just fine.

On the other hand, ejrh/snmp library used by ntopng ALWAYS encodes integers using 4 bytes, even for small values (R=1 in the test above). That is standard violation that does not pass bsnmpd's strict validation, but pass relaxed validation of net-snmpd.
Comment 4 Florin Samareanu 2018-04-19 06:58:37 UTC
Thank you. I've passed the message back to ntopng devs.
Comment 5 Eugene Grosbein freebsd_committer freebsd_triage 2018-05-14 10:58:39 UTC
Feedback timeout. Also, the problem believed in the ntopng itself. This PR can be re-opened if new data arrives.