Bug 197499 - wc libxo emits invalid JSON when used on multiple files
Summary: wc libxo emits invalid JSON when used on multiple files
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Marcel Moolenaar
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-09 20:31 UTC by Allan Jude
Modified: 2015-02-11 17:57 UTC (History)
1 user (show)

See Also:


Attachments
Patch for wc that puts the total object outside the file array (481 bytes, patch)
2015-02-09 21:01 UTC, Allan Jude
no flags Details | Diff
removes the 'total' key, making the total just a regular array element (389 bytes, patch)
2015-02-09 21:02 UTC, Allan Jude
no flags Details | Diff
make 'total' the last element of the file array [updated patch] (450 bytes, patch)
2015-02-10 00:06 UTC, Allan Jude
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Allan Jude freebsd_committer 2015-02-09 20:31:10 UTC
When using wc's libxo JSON output method, it emits invalid JSON.

Steps to recreate:

wc --libxo json /etc/defaults/*



The output is (pretty-printed for readability):



{
  "wc": {
    "file": [
      {
        "lines": 111,
        "words": 644,
        "characters": 3999,
        "filename": "/etc/defaults/bluetooth.device.conf"
      },
      {
        "lines": 87,
        "words": 403,
        "characters": 2439,
        "filename": "/etc/defaults/devfs.rules"
      },
      {
        "lines": 385,
        "words": 1228,
        "characters": 11700,
        "filename": "/etc/defaults/periodic.conf"
      },
      {
        "lines": 692,
        "words": 4576,
        "characters": 36171,
        "filename": "/etc/defaults/rc.conf"
      },
      "total": {
        "lines": 1275,
        "words": 6851,
        "characters": 54309,
        "filename": "total"
      }
    ]
  }
}


The problem is that the 'total' segment, is a key/value pair, but that is not allowed inside an array.

The output would either need to be changed to remove the key (the other elements in the array use the filename for identification), or be made a sibling of the 'file' 2nd level element, like so:

{
  "wc": {
    "file": [
      {
        "lines": 111,
        "words": 644,
        "characters": 3999,
        "filename": "/etc/defaults/bluetooth.device.conf"
      },
      {
        "lines": 87,
        "words": 403,
        "characters": 2439,
        "filename": "/etc/defaults/devfs.rules"
      },
      {
        "lines": 385,
        "words": 1228,
        "characters": 11700,
        "filename": "/etc/defaults/periodic.conf"
      },
      {
        "lines": 692,
        "words": 4576,
        "characters": 36171,
        "filename": "/etc/defaults/rc.conf"
      }
    ],
    "total": {
      "lines": 1275,
      "words": 6851,
      "characters": 54309,
      "filename": "total"
    }
  }
}
Comment 1 Allan Jude freebsd_committer 2015-02-09 20:59:00 UTC
I am not clear which was the intended output, so I have attached a patch for each way:

'inside' (total is the last element on the file array)

and 'outside' (total is a sibling key, at the same level as the file array)
Comment 2 Allan Jude freebsd_committer 2015-02-09 21:01:54 UTC
Created attachment 152820 [details]
Patch for wc that puts the total object outside the file array
Comment 3 Allan Jude freebsd_committer 2015-02-09 21:02:22 UTC
Created attachment 152821 [details]
removes the 'total' key, making the total just a regular array element
Comment 4 Allan Jude freebsd_committer 2015-02-10 00:06:18 UTC
Created attachment 152825 [details]
make 'total' the last element of the file array [updated patch]
Comment 5 Marcel Moolenaar freebsd_committer 2015-02-10 20:33:50 UTC
I don't seem to be able to reproduce your problem:

fbsdvm64% wc --libxo json,pretty /etc/defaults/*
{
  "wc": {
    "file": [
      {
        "lines": 111,
        "words": 644,
        "characters": 3999,
        "filename": "/etc/defaults/bluetooth.device.conf"
      },
      {
        "lines": 87,
        "words": 403,
        "characters": 2439,
        "filename": "/etc/defaults/devfs.rules"
      },
      {
        "lines": 381,
        "words": 1221,
        "characters": 11568,
        "filename": "/etc/defaults/periodic.conf"
      },
      {
        "lines": 699,
        "words": 4593,
        "characters": 36522,
        "filename": "/etc/defaults/rc.conf"
      }
    ],
    "total": {
      "lines": 1278,
      "words": 6861,
      "characters": 54528,
      "filename": "total"
    }
  }
}
fbsdvm64% 

"file" is a proper array and "total" falls outside of the array. jsonlint.com flags this as valid JSON.

I'm running sources as of today.

Do you have older sources or modified sources?
Comment 6 Allan Jude freebsd_committer 2015-02-11 05:17:04 UTC
(In reply to Marcel Moolenaar from comment #5)

My results are different:
# uname -a
FreeBSD Nexus.HML3.ScaleEngine.net 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r276776: Wed Jan  7 16:16:14 UTC 2015     root@Nexus.HML3.ScaleEngine.net:/usr/obj/usr/src/sys/NEXUS  amd64


# wc --libxo json,pretty /etc/defaults/*
{
  "wc": {
    "file": [
      {
        "lines": 111,
        "words": 644,
        "characters": 3999,
        "filename": "/etc/defaults/bluetooth.device.conf"
      },
      {
        "lines": 87,
        "words": 403,
        "characters": 2439,
        "filename": "/etc/defaults/devfs.rules"
      },
      {
        "lines": 385,
        "words": 1228,
        "characters": 11700,
        "filename": "/etc/defaults/periodic.conf"
      },
      {
        "lines": 692,
        "words": 4576,
        "characters": 36171,
        "filename": "/etc/defaults/rc.conf"
      },
      "total": {
        "lines": 1275,
        "words": 6851,
        "characters": 54309,
        "filename": "total"
      }
    ]
  }
}


Let me try a newer build
Comment 7 Marcel Moolenaar freebsd_committer 2015-02-11 17:57:06 UTC
Committed revision 278590.
Comment 8 commit-hook freebsd_committer 2015-02-11 17:57:13 UTC
A commit references this bug:

Author: marcel
Date: Wed Feb 11 17:56:25 UTC 2015
New revision: 278590
URL: https://svnweb.freebsd.org/changeset/base/278590

Log:
  Close the file list before opening the container that holds the
  totals, otherwise we end up emitting invalid JSON -- provided
  libxo does not prevent us from doing that.

  PR:		197499
  Submitted by:	allanjude@

Changes:
  head/usr.bin/wc/wc.c