Bug 224455 - [NEW PORT] sysutils/grok_exporter: Export Prometheus metrics from arbitrary unstructured log data
Summary: [NEW PORT] sysutils/grok_exporter: Export Prometheus metrics from arbitrary u...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Li-Wen Hsu
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-19 14:14 UTC by Gasol Wu
Modified: 2018-03-21 05:38 UTC (History)
1 user (show)

See Also:


Attachments
shar file (5.83 KB, text/plain)
2017-12-19 14:14 UTC, Gasol Wu
no flags Details
poudriere test log on freebsd 10.4 amd64 (13.28 KB, text/plain)
2017-12-19 14:15 UTC, Gasol Wu
no flags Details
poudriere testport log on freebsd 11.1 amd64 (13.51 KB, text/plain)
2017-12-19 14:15 UTC, Gasol Wu
no flags Details
shar file (5.86 KB, text/plain)
2018-02-26 09:01 UTC, Gasol Wu
no flags Details
shar file (18.03 KB, text/plain)
2018-03-01 07:31 UTC, Gasol Wu
no flags Details
shar file (5.86 KB, text/plain)
2018-03-09 09:15 UTC, Gasol Wu
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gasol Wu 2017-12-19 14:14:45 UTC
Created attachment 188960 [details]
shar file

Grok is a tool to parse crappy unstructured log data into something structured
and queryable. Grok is heavily used in Logstash to provide log data as input for
ElasticSearch.

Grok ships with about 120 predefined patterns for syslog logs, apache and other
webserver logs, mysql logs, etc. It is easy to extend Grok with custom patterns.

The grok_exporter aims at porting Grok from the ELK stack to Prometheus
monitoring. The goal is to use Grok patterns for extracting Prometheus metrics
from arbitrary log files.

WWW: https://github.com/fstab/grok_exporter
Comment 1 Gasol Wu 2017-12-19 14:15:23 UTC
Created attachment 188961 [details]
poudriere test log on freebsd 10.4 amd64
Comment 2 Gasol Wu 2017-12-19 14:15:53 UTC
Created attachment 188962 [details]
poudriere testport log on freebsd 11.1 amd64
Comment 3 Li-Wen Hsu freebsd_committer freebsd_triage 2018-02-21 07:30:06 UTC
Doesn't build on i386: https://gist.github.com/1f3de37d3dcba1f4e1a7a6730c670c70
Comment 4 Gasol Wu 2018-02-26 09:01:51 UTC
Created attachment 191013 [details]
shar file

Add BROKEN_i386 line
Comment 5 Li-Wen Hsu freebsd_committer freebsd_triage 2018-02-27 19:36:43 UTC
(In reply to Gasol Wu from comment #4)

We don't usually add a port with `BROKEN` mark, if you are very sure it cannot work on i386 (and/or other architectures), use `ONLY_FOR_ARCHS` or `NOT_FOR_ARCHS`.
Comment 6 Gasol Wu 2018-03-01 07:26:37 UTC
(In reply to Li-Wen Hsu from comment #5)

Okay, I got the point. It says only works on 64bits on Linux, Windows and Darwin in README.md. I just copy (patched) darwin related code for FreeBSD support. And run it on production without problem.

https://github.com/fstab/grok_exporter#status

I will revise Makefile to using `ONLY_FOR_ARCHS`
Comment 7 Gasol Wu 2018-03-01 07:31:21 UTC
Created attachment 191091 [details]
shar file

use ONLY_FOR_ARCHS instead of BROKEN
Comment 8 Li-Wen Hsu freebsd_committer freebsd_triage 2018-03-06 11:20:28 UTC
I'm sorry that I forgot this, cloud you install devel/rclint and use it to verify the rc script?  Thanks very much!
Comment 9 Gasol Wu 2018-03-09 09:15:00 UTC
Created attachment 191334 [details]
shar file

Here is output result of rclint:

Before:
Checking grok_exporter.in
ERROR:root:[0]: Missing FreeBSD RCSId keyword
==> All rc scripts must contain a line beginning # $FreeBSD$.  Do not
    include blank lines without comment markers at the beginning (#)
    until the script begins
ERROR:root:[45]: Order of rc file incorrect
==> Order of the rc file should be shebang/header/$FreeBSD$/sourcing
    rc_subr/name/rcvar/load_rc_config/setting defaults/setting other
    definitions/defining functions.  Do not include unassociated shell
    commands, and blocks must be separated by single blank lines.
    Single blank lines may appear inside the defaults, definitions and
    functions blocks
ERROR:root:[26]: Do not quote values unless necessary
==> Unless there are spaces in the value, quotes are unnecessary.
    With the syntax ${variable=value}, value can even contain spaces
    and does not need quoting
ERROR:root:[28]: Do not quote values unless necessary
==> Unless there are spaces in the value, quotes are unnecessary.
    With the syntax ${variable=value}, value can even contain spaces
    and does not need quoting
ERROR:root:[29]: Do not quote values unless necessary
==> Unless there are spaces in the value, quotes are unnecessary.
    With the syntax ${variable=value}, value can even contain spaces
    and does not need quoting
ERROR:root:[32]: Do not quote values unless necessary
==> Unless there are spaces in the value, quotes are unnecessary.
    With the syntax ${variable=value}, value can even contain spaces
    and does not need quoting
ERROR:root:[0]: No description included
==> All rc files should include a variable desc="Description of the
    service"
ERROR:root:[27]: Do not clobber blank values for non-mandatory variables
==> Syntax for variables that are not mandatory is ${var=value};
    including := will override var="" set in rc.conf (man sh).  If
    this variable is mandatory just for this script, then ignore this
    error

After revised:
Checking grok_exporter.in
ERROR:root:[29]: Do not clobber blank values for non-mandatory variables
==> Syntax for variables that are not mandatory is ${var=value};
    including := will override var="" set in rc.conf (man sh).  If
    this variable is mandatory just for this script, then ignore this
    error
Comment 10 Gasol Wu 2018-03-19 02:51:24 UTC
hello, still having problem?
Comment 11 Li-Wen Hsu freebsd_committer freebsd_triage 2018-03-19 07:08:31 UTC
Hi, looks good to me, sorry that I've been busy recently but I will get this in soon.  BTW, do you mind I change this to net-mgmt category to sit with other prometheus plugins?
Comment 12 Gasol Wu 2018-03-19 13:56:30 UTC
I'm ok for changing category :)
Comment 13 commit-hook freebsd_committer freebsd_triage 2018-03-21 05:38:03 UTC
A commit references this bug:

Author: lwhsu
Date: Wed Mar 21 05:37:37 UTC 2018
New revision: 465147
URL: https://svnweb.freebsd.org/changeset/ports/465147

Log:
  Add net-mgmt/grok_exporter, export Prometheus metrics from arbitrary
  unstructured log data

  PR:		224455
  Submitted by:	Gasol Wu <gasol.wu@gmail.com>

Changes:
  head/net-mgmt/Makefile
  head/net-mgmt/grok_exporter/
  head/net-mgmt/grok_exporter/Makefile
  head/net-mgmt/grok_exporter/distinfo
  head/net-mgmt/grok_exporter/files/
  head/net-mgmt/grok_exporter/files/config.yml.in
  head/net-mgmt/grok_exporter/files/grok_exporter.in
  head/net-mgmt/grok_exporter/pkg-descr
  head/net-mgmt/grok_exporter/pkg-plist