Bug 221676
| Summary: | libxo fails to format certain fixed values correctly (eg: arp: invalid json output from libxo when bridge is present) | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | Jimmy Olgeni <olgeni> | ||||
| Component: | bin | Assignee: | Phil Shafer <phil> | ||||
| Status: | Closed FIXED | ||||||
| Severity: | Affects Some People | CC: | duncan, garga, koobs, marcel, net, phil, re, rgrimes, wes | ||||
| Priority: | --- | Keywords: | easy | ||||
| Version: | CURRENT | Flags: | koobs:
mfc-stable11+
|
||||
| Hardware: | Any | ||||||
| OS: | Any | ||||||
| Attachments: |
|
||||||
Created attachment 185787 [details] Adds the quote directive to xo fields to make sure that strings are quoted in json output I can reproduce this in general for libxo. The number detecting heuristic it uses allows for C style numeric suffices (for example 34f, 12d). This allows any strings ending the characters in the set [diouDOUeEfFgG] to be treated as numbers (left unquoted in json). This has two drawbacks firstly leaving strings such as bridge unquoted because it ends in e, secondly it allows numbers with C style suffices to be output unqoted these are not valid json. for example: { "key": 35f } is invalid. I've added the quote directive to all of the relevant xo_emit calls as per the documentation on libxo at http://juniper.github.io/libxo/libxo-manual.html. I've added a patch which should fix this problem for the arp application. I think this may apply to other applications using libxo as well. I just hit the same issue. Looking at the current code in head[1] it looks like the patch still needs to be applied. Anything I can do to help make that happen? [1]: https://svnweb.freebsd.org/base/head/usr.sbin/arp/arp.c?revision=331714&view=markup#l670 CC committer of libxo support (base r316592) for arp, re@ for 11.2-STABLE (and subsequent RELEASE) merge approval @Jimmy Now that there is a patch, and if you have the cycles, feel free to assign yourself and shepherd the change in with a phabricator review and finding someone to review it. While I'm here cc Phil and Marcel, who have updated libxo in base in the past, and may be able to help review (along with Renato) No, this is very much a libxo bug.
I'm looking at the "fmt" to see if it's a numeric format, for example, "%32d" or "%f", but I don't handle the case where it's a fixed value correctly, so the "{:type/ethernet}" works but "{:type/bridge}" fails, since "e" is a numeric format type. I'll add a check to handle this (in xo_format_value()).
Apologies for the issue.
Thanks,
Phil
Comment on attachment 185787 [details]
Adds the quote directive to xo fields to make sure that strings are quoted in json output
Bug is in libxo, not specific to arp.
Have fix in libxo github repo: https://github.com/Juniper/libxo/commit/27d3021cc3cc8cfbe9ddee5930cd7a9afea8f68f#diff-5a0d468963477f7daedb8308c219dd80 I'll spin a release and import it into HEAD this afternoon. Thanks, Phil A commit references this bug: Author: phil Date: Wed May 23 01:20:32 UTC 2018 New revision: 334068 URL: https://svnweb.freebsd.org/changeset/base/334068 Log: Import libxo-0.9.0: - Add xo_format_is_numeric() with improved logic to decide if format strings are numeric, so json output quotes them - Convert docs to sphinx/rst - update tests Includes fix for PR 221676: https://github.com/Juniper/libxo/commit/27d3021cc3cc8cfbe9ddee5930cd7a9afea8f68f#diff-5a0d468963477f7daedb8308c219dd80 PR: 221676 MFC after: 5 days Changes: _U head/contrib/libxo/ head/contrib/libxo/configure.ac head/contrib/libxo/doc/Makefile.am head/contrib/libxo/doc/_static/ head/contrib/libxo/doc/_templates/ head/contrib/libxo/doc/api.rst head/contrib/libxo/doc/conf.py head/contrib/libxo/doc/example.rst head/contrib/libxo/doc/faq.rst head/contrib/libxo/doc/field-formatting.rst head/contrib/libxo/doc/field-modifiers.rst head/contrib/libxo/doc/field-roles.rst head/contrib/libxo/doc/format-strings.rst head/contrib/libxo/doc/formatting.rst head/contrib/libxo/doc/getting.rst head/contrib/libxo/doc/howto.rst head/contrib/libxo/doc/index.rst head/contrib/libxo/doc/intro.rst head/contrib/libxo/doc/libxo-manual.html head/contrib/libxo/doc/options.rst head/contrib/libxo/doc/xo.rst head/contrib/libxo/doc/xohtml.rst head/contrib/libxo/doc/xolint.rst head/contrib/libxo/doc/xopo.rst head/contrib/libxo/libxo/libxo.c head/contrib/libxo/tests/core/saved/test_01.E.out head/contrib/libxo/tests/core/saved/test_01.H.out head/contrib/libxo/tests/core/saved/test_01.HIPx.out head/contrib/libxo/tests/core/saved/test_01.HP.out head/contrib/libxo/tests/core/saved/test_01.J.out head/contrib/libxo/tests/core/saved/test_01.JP.out head/contrib/libxo/tests/core/saved/test_01.T.out head/contrib/libxo/tests/core/saved/test_01.X.out head/contrib/libxo/tests/core/saved/test_01.XP.out head/contrib/libxo/tests/core/saved/test_12.J.out head/contrib/libxo/tests/core/saved/test_12.JP.out head/contrib/libxo/tests/core/test_01.c head/contrib/libxo/tests/gettext/saved/gt_01.J.out head/contrib/libxo/tests/gettext/saved/gt_01.JP.out head/lib/libxo/add.man head/lib/libxo/xo_config.h head/usr.bin/xohtml/xohtml.sh A commit references this bug: Author: gjb Date: Thu May 31 23:56:00 UTC 2018 New revision: 334458 URL: https://svnweb.freebsd.org/changeset/base/334458 Log: MFC r334068 (phil): Import libxo-0.9.0: - Add xo_format_is_numeric() with improved logic to decide if format strings are numeric, so json output quotes them - Convert docs to sphinx/rst - update tests PR: 221676 Approved by: re (marius) Sponsored by: The FreeBSD Foundation Changes: _U stable/11/ stable/11/contrib/libxo/configure.ac stable/11/contrib/libxo/doc/Makefile.am stable/11/contrib/libxo/doc/_static/ stable/11/contrib/libxo/doc/_templates/ stable/11/contrib/libxo/doc/api.rst stable/11/contrib/libxo/doc/conf.py stable/11/contrib/libxo/doc/example.rst stable/11/contrib/libxo/doc/faq.rst stable/11/contrib/libxo/doc/field-formatting.rst stable/11/contrib/libxo/doc/field-modifiers.rst stable/11/contrib/libxo/doc/field-roles.rst stable/11/contrib/libxo/doc/format-strings.rst stable/11/contrib/libxo/doc/formatting.rst stable/11/contrib/libxo/doc/getting.rst stable/11/contrib/libxo/doc/howto.rst stable/11/contrib/libxo/doc/index.rst stable/11/contrib/libxo/doc/intro.rst stable/11/contrib/libxo/doc/libxo-manual.html stable/11/contrib/libxo/doc/options.rst stable/11/contrib/libxo/doc/xo.rst stable/11/contrib/libxo/doc/xohtml.rst stable/11/contrib/libxo/doc/xolint.rst stable/11/contrib/libxo/doc/xopo.rst stable/11/contrib/libxo/libxo/libxo.c stable/11/contrib/libxo/tests/core/saved/test_01.E.out stable/11/contrib/libxo/tests/core/saved/test_01.H.out stable/11/contrib/libxo/tests/core/saved/test_01.HIPx.out stable/11/contrib/libxo/tests/core/saved/test_01.HP.out stable/11/contrib/libxo/tests/core/saved/test_01.J.out stable/11/contrib/libxo/tests/core/saved/test_01.JP.out stable/11/contrib/libxo/tests/core/saved/test_01.T.out stable/11/contrib/libxo/tests/core/saved/test_01.X.out stable/11/contrib/libxo/tests/core/saved/test_01.XP.out stable/11/contrib/libxo/tests/core/saved/test_12.J.out stable/11/contrib/libxo/tests/core/saved/test_12.JP.out stable/11/contrib/libxo/tests/core/test_01.c stable/11/contrib/libxo/tests/gettext/saved/gt_01.J.out stable/11/contrib/libxo/tests/gettext/saved/gt_01.JP.out stable/11/lib/libxo/add.man stable/11/lib/libxo/xo_config.h |
Trying to get json output from arp(8): # arp -a --libxo json | jq . parse error: Invalid numeric literal at line 1, column 170 Indeed, the output document was this: {"__version": "1", "arp": {"arp-cache": [{"hostname":"?","ip-address":"172.16.1.1","mac-address":"02:c9:3a:59:26:00","interface":"bridge0","permanent":true,"type":bridge}, {"hostname":"olgeni.home.olgeni.com","ip-address":"192.168.0.1","mac-address":"bc:ae:c5:5b:e5:e5","interface":"re0","permanent":true,"type":"ethernet"}, {"hostname":"?","ip-address":"192.168.0.254","mac-address":"4c:60:de:f9:44:46","interface":"re0","expires":1173,"type":"ethernet"}]} } Everything looks ok, but the 'bridge' string is not quoted.