Bug 206785 - [new port] textproc/miller: processing files with name-indexed data such as CSV
Summary: [new port] textproc/miller: processing files with name-indexed data such as CSV
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Kurt Jaeger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-01-31 12:14 UTC by Kurt Jaeger
Modified: 2016-02-08 06:23 UTC (History)
2 users (show)

See Also:


Attachments
shar (3.73 KB, text/plain)
2016-01-31 12:14 UTC, Kurt Jaeger
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kurt Jaeger freebsd_committer freebsd_triage 2016-01-31 12:14:12 UTC
Created attachment 166339 [details]
shar

Builds on 11a, 10.2a+i, BROKEN on 9.

portlint -AC is OK.
Comment 2 Kurt Jaeger freebsd_committer freebsd_triage 2016-01-31 12:30:44 UTC
One small fix:

LICENSE_FILE=   ${WRKSRC}/LICENSE.txt
Comment 3 John Kerl 2016-02-02 15:55:12 UTC
Hi -- I'm the author of Miller. Just found this thread inadvertently.

I'm happy to help out with getting this into FreeBSD. (Miller is already in NetBSD, Debian, Ubuntu, and Brew using GNU autoconfig -- so I hope it will be quick to resolve specific issues here.)


-- John Kerl
kerl.john.r@gmail.com
http://johnkerl.org
https://github.com/johnkerl/miller
Comment 4 John Kerl 2016-02-02 15:57:09 UTC
Also would love to discuss with you the mechanism for keeping FreeBSD's package tracking up to date with Miller package versions as I release them.
Comment 5 sf(jungleboogie) 2016-02-02 16:12:51 UTC
(In reply to John Kerl from comment #3)

Hi John,

Happy to hear you found this thread. Thanks for mlr and all the work you've done to get it working on FreeBSD 10.x.

As you see above, the build fails on 9.x due to that version using an older version of Flex. 

Unfortunately, I don't know how to patch flex to support mlr on freeBSD 9.x
Do you have any suggestions?

RE: (In reply to John Kerl from comment #4)
Once a new version is released, the makefile is modified to support the new release (i.e. changing from 3.3.2 to 3.4.0) and any other changes that are necessary. 
Updating a port manual is here:
https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/port-upgrading.html

You can either make the patch and submit here on bugzilla or I can. Either way, it will need to be reviewed and updated by someone with ports subversion access.
Comment 6 John Kerl 2016-02-02 16:16:23 UTC
I don't know about patching flex, but, Miller is very simple with its lex/parse demands. So it might be simpler for me to make sure it's using syntax that an older version of flex can handle.
Comment 7 sf(jungleboogie) 2016-02-02 16:21:38 UTC
(In reply to John Kerl from comment #6)

That would probably be best.

BTW, here's the build log on 9.x:
http://people.freebsd.org/~pi/logs/textproc__miller-93a-1454241832.txt
Comment 8 Kurt Jaeger freebsd_committer freebsd_triage 2016-02-02 17:22:48 UTC
(In reply to John Kerl from comment #3)
In configure on 9.3, /usr/bin/flex is found, but is too old to work.

If you can provide a configure option that allows us to suggest a different
flex to execute, that would be sufficient. Can you do that ?
Comment 9 Kurt Jaeger freebsd_committer freebsd_triage 2016-02-02 17:27:59 UTC
(In reply to John Kerl from comment #4)
FreeBSD has portscout:

https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/book.html#distfile-survey

http://portscout.freebsd.org/

which alerts the maintainer if one of the ports has new distfiles.
So this is mostly covered already. If you have new versions, the maintainer
should testbuild them and submit PRs to commit any updates.
Comment 10 John Kerl 2016-02-02 18:20:50 UTC
(In reply to John Kerl from comment #6)

P.S. What's the flex --version in FreeBSD 9?
Comment 11 Kurt Jaeger freebsd_committer freebsd_triage 2016-02-02 18:52:38 UTC
(In reply to John Kerl from comment #10)
9.3a:
flex 2.5.4
10.2:
flex 2.5.37
11a:
flex 2.6.0
Comment 12 John Kerl 2016-02-02 20:19:25 UTC
(In reply to Kurt Jaeger from comment #11)
OK. 2.5.4 is really old. My guess is it doesn't like `--prefix`. I can implement that functionality outside of flex itself with some sedding in `dsls/Makefile`.
Comment 13 John Kerl 2016-02-03 02:33:07 UTC
(In reply to John Kerl from comment #12)

Ouch, 2.5.4 is *really* old & lacks several features I rely on. Here are some options:

* I re-work Miller's use of flex to be ancient-compatible. A bit more of a project than I thought. But this is doable since people used to use flex for things in those days; a solution exists.

* I include a copy of modern flex directly within the Miller distribution. Yuck.

* I run flex & lemon only at package-build time & from then on treat them as .c/.h files which are checked into git.

The third isn't particularly elegant but might not be too bad. After all the grammar doesn't change *too* often ... as long as the miller-dev readme material is quite clear about who needs to run flex and when ... maybe that will suffice.
Comment 14 John Kerl 2016-02-03 02:34:26 UTC
(In reply to John Kerl from comment #13)

P.S. I don't have a FreeBSD system but was able to grab flex 2.5.4 from sourceforge & get a repro on my dev box. So whatever I submit back to you will have been well-tested. :)
Comment 15 sf(jungleboogie) 2016-02-06 00:24:06 UTC
(In reply to John Kerl from comment #14)

Hi John,

Any luck in determining how to package for FreeBSD 9.3?
Comment 16 John Kerl 2016-02-06 04:25:21 UTC
(In reply to sf(jungleboogie) from comment #15)
Hi there -- I think the easiest thing will be to have the lexer/parser .c and .h files in source control, and not to run flex (or lemon) on every build. Rather, to run them only when specifically invoked to do so, at package-release time. Along with copious comments for maintainers someday.

This sounds pretty simple, but it's going to be a very busy weekend in RL. So, probably next week sometime. :)
Comment 17 Kurt Jaeger freebsd_committer freebsd_triage 2016-02-06 18:06:47 UTC
I found a way to conditionaly BUILD_DEPEND on flex for 9 and replace the c/dsls/Makefile flex call with the proper path. This builds fine. Final round of testbuilds is @work, will commit if all is peachy.
Comment 18 commit-hook freebsd_committer freebsd_triage 2016-02-06 18:09:59 UTC
A commit references this bug:

Author: pi
Date: Sat Feb  6 18:09:51 UTC 2016
New revision: 408308
URL: https://svnweb.freebsd.org/changeset/ports/408308

Log:
  New port: textproc/miller

  Miller is like sed, awk, cut, join, and sort for name-indexed data such
  as CSV.

  With Miller you get to use named fields without needing to count
  positional indices.

  WWW: http://johnkerl.org/miller/doc/

  PR:		206785
  Requested by:	jungleboogie0@gmail.com

Changes:
  head/textproc/Makefile
  head/textproc/miller/
  head/textproc/miller/Makefile
  head/textproc/miller/distinfo
  head/textproc/miller/files/
  head/textproc/miller/files/extra-patch-c_Makefile
  head/textproc/miller/files/extra-patch-libtool
  head/textproc/miller/pkg-descr
Comment 19 Kurt Jaeger freebsd_committer freebsd_triage 2016-02-06 18:10:56 UTC
Committed, thanks for the suggestion and thanks to John for the software!
Comment 20 John Kerl 2016-02-07 22:45:48 UTC
(In reply to Kurt Jaeger from comment #19)
Awesome!!

Two questions for you: (1) Do I need to pull any of these changes upstream into the master Miller github repo? Or do they remain FreeBSD-only files, to be merged by the FreeBSD package owner (as triggered by portscout) whenever I push a new usptream Miller release?

And (2) Is https://www.freshports.org/textproc/miller/ the final link for me to direct FreeBSD users to from Miller's homepage? Or wil a freebsd.org/something/something begin to exist at some point?
Comment 21 sf(jungleboogie) 2016-02-07 22:57:23 UTC
(In reply to John Kerl from comment #20)

2) It doesn't really make sense to direct people to this bug instead of freshports.
This is just the bug submitting for a new port.
If you don't want to link to freshports, you can link here:
https://svnweb.freebsd.org/ports/head/textproc/miller/

Although freshports is fetching info from the page above and displaying it in a nice format.
Comment 22 John Kerl 2016-02-07 23:01:51 UTC
(In reply to sf(jungleboogie) from comment #21)
RE #2: definitely, that bug was/is just a temp link while awaiting the real link.

OK I got it. Your previous comment made me think freshports was separate and unaffiliated, but if freshports' textproc/miller is a prettification of freebsd's textproc/miller, and if those two are in sync ... then I'll just point to freshports.

Thanks for this important clarification!
Comment 23 Kurt Jaeger freebsd_committer freebsd_triage 2016-02-08 06:23:03 UTC
John, for (1) please look at the files in files/ -- if you can integrate those changes upstream in a future release, so that we can remove them from the port, that would help. For (2), the freshports link is as good as it gets.