Bug 222883 - [NEW PORT] www/webhook: Easily create HTTP endpoints (hooks) to execute shell commands
Summary: [NEW PORT] www/webhook: Easily create HTTP endpoints (hooks) to execute shell...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Dave Cottlehuber
URL:
Keywords: feature, needs-qa
Depends on:
Blocks:
 
Reported: 2017-10-09 21:32 UTC by Stefan Bethke
Modified: 2018-03-26 12:26 UTC (History)
3 users (show)

See Also:
stb: maintainer-feedback+


Attachments
New port www/webhook as shar (5.84 KB, application/x-shar)
2017-10-09 21:32 UTC, Stefan Bethke
no flags Details
Port for Version 2.6.6 (5.64 KB, application/x-shar)
2017-11-05 12:33 UTC, Stefan Bethke
no flags Details
Updated version as per suggestions in this issue. (5.53 KB, application/x-shar)
2017-12-11 21:08 UTC, Stefan Bethke
no flags Details
Port for webhook 2.6.7 (5.53 KB, application/x-shar)
2017-12-13 20:57 UTC, Stefan Bethke
stb: maintainer-approval+
Details
Updated port, version 2.6.8 (5.53 KB, application/x-shar)
2017-12-22 18:36 UTC, Stefan Bethke
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Bethke 2017-10-09 21:32:44 UTC
Created attachment 187031 [details]
New port www/webhook as shar

webhook is a lightweight configurable tool written in Go, that allows you to
easily create HTTP endpoints (hooks) on your server, which you can use to
execute configured commands.  You can also pass data from the HTTP request
(such as headers, payload or query variables) to your commands.  webhook
also allows you to specify rules which have to be satisfied in order for the
hook to be triggered.

WWW: https://github.com/adnanh/webhook
Comment 1 Stefan Bethke 2017-10-09 21:37:43 UTC
Note the ugly use of GH_TUPLE to supply more-or-less well matched dependencies to the project.  I've opened a pull request upstream to have all dependencies properly vendored: https://github.com/adnanh/webhook/pull/180
Comment 2 Stefan Bethke 2017-11-05 12:33:49 UTC
Created attachment 187755 [details]
Port for Version 2.6.6

Port for version 2.6.6.

2.6.6 includes properly vendors dependencies, making the previous GH_TUPLE hack superfluous.
Comment 3 Stefan Bethke 2017-11-08 21:02:10 UTC
Any way I can help this along?
Comment 4 Dave Cottlehuber freebsd_committer freebsd_triage 2017-11-09 00:46:29 UTC
thanks stb! sorry this took so long to get underway.

in general LGTM, a few typos and traces of gitea in there ;-).

## general

All the upstream docs mention a JSON file, not yaml. Evidently it works with
both, but surely it would be better to match the docs, and then use YAML
yourself? POLA for users. your call!

Why the LICENSE file within the port? This would normally point towards a
master licence file from the port's own source, and not for the port makefiles
itself.

You'll need a multi LICENSE section as a number of the dependencies are not MIT
based - there's ALv2, BSD3CLAUSE and more in here
https://github.com/adnanh/webhook/blob/master/Godeps/Godeps.json please check
and make sure these are correct.

LICENSE=	APACHE20 BSD3CLAUSE MIT 
LICENSE_COMB=	multi

DISTVERSION is the new PORTVERSION now

You can simply use files/webhook.yaml.sample as you are not doing any
substitutions which is the normal reason for files/*.in then drop SUB_FILES
from the Makefile, and update the do-install section.


## rc.d script

s/gitea/webhook/
s/webook/webhook/

drop /usr/bin/env as we are not passing env options here

this could reasonably use nobody as the default user and be a lot safer


## daemon

normally we'd use the following construct for daemon(8) wrappers:

```
pidfile=/var/run/${name}.pid
extra_commands=reload
sig_reload=USR1

procname=%%PREFIX%%/sbin/${name}
command=/usr/sbin/daemon
command_args=" -f \
	-t ${name} \
	-p ${pidfile} \
	-u ${webhook_user} \
	-r \
	${procname} \
	-hooks ${webhook_conf} \
	${webhook_options}"

start_precmd="install -o ${webhook_user} /dev/null ${pidfile}"
...
```

using -t ... the daemon(8) process clearly shows the webhook name

using command=/usr/sbin/daemon keeps the config file clean and tidy

however this *also* means that the reload command sends its signals
to the parent daemon and not the child, which requires using something
like:

reload_cmd="pkill -SIGUSR1-U ${webhook_user} -F {pidfile} ${procname}"
Comment 5 Joseph Mingrone freebsd_committer freebsd_triage 2017-11-09 03:17:19 UTC
You could also remove pkg-plist and specify `PLIST_FILES="@sample etc/webhook.yaml.sample" sbin/webhook` in Makefile.  See 3.2.2. of the Porter's Handbook.

A space should be added before `.include <bsd.port.mk>`.
Comment 6 Stefan Bethke 2017-11-09 11:03:23 UTC
(In reply to Dave Cottlehuber from comment #4)

Thank you for that in-depth feedback!

Many of the points are indeed sloppyness on my part, I'll try to address them asap.

One of the use cases of webhook is running commands as a privileged user, something that's a lot harder to accomplish with say Apache or NGINX.  But I'm happy to change the default to nobody.

One issue with daemon(8) and -t is that the version in 10-stable doesn't have that yet. tobik suggested (and implemented) a build-time switch for www/gitea.  I'm happy to adopt that here, too.

For the example config, I picked YAML because it allows comments (unlike JSON).  I opted for a custom example config because I'd really like ports to have an actually working out of the box config when I first try out a port.  If you feel the gap between this sample config and the README and docs on adnanh/webhook is too large, I'd rather submit a pull request against the docs in the upstream repo.
Comment 7 Dave Cottlehuber freebsd_committer freebsd_triage 2017-11-09 12:47:31 UTC
+1 for yaml with comments vs JSON without then, maybe just mention in there that
JSON also works. I agree, working OOTB is important.

wrt -t in daemon, sorry I should have checked that. This is useful, thanks! Going to add it to a few other ports right now.

.if ${OPSYS} == FreeBSD && ${OSVERSION} >= 1100513
DAEMONARGS=     -S -l \$${gitea_facility} -s \$${gitea_priority} -t \$${name}
.else
DAEMONARGS=     -f
.endif

regarding whether to use command/command_args, I am not sure which is "better".

If you want to add a github webhook yaml example here's one. I've been using 
this for a couple of weeks now to kick off ansible.

---
# github post-commit hook
- id: ansible
  execute-command: "/home/ansible/src/ansible/deploy.sh"
  command-working-directory: "/home/ansible/src/ansible"
  pass-arguments-to-command:
  - source: payload
    name: head_commit.id
  - source: payload
    name: pusher.name
  - source: payload
    name: pusher.email
  trigger-rule:
    and:
    - match:
        type: payload-hash-sha1
        secret: .....
        parameter:
          source: header
          name: X-Hub-Signature
    - match:
        type: value
        value: refs/heads/master
        parameter:
          source: payload
          name: ref
Comment 8 Dave Cottlehuber freebsd_committer freebsd_triage 2017-12-01 13:01:41 UTC
bumping - lets see if we can land this before 2018. Can I help out anywhere?
Comment 9 Stefan Bethke 2017-12-01 13:38:28 UTC
Sorry, ENOTIME because of $work. I am planning to work on it this weekend, and am confident to have an acceptable solution on Sunday.
Comment 10 Stefan Bethke 2017-12-11 20:49:56 UTC
(In reply to Dave Cottlehuber from comment #8)

I've tried to apply all the suggestions; the current version is at https://github.com/stblassitude/webhook-port

The start script however, is not working as intended; daemon is trying to start webhook, but failing. Running the script with -vx:

+ echo 'Starting webhook.'
Starting webhook.
+ [ -n '' ]
+ _cd=''
+ _doit=$'/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t'
+ [ -n nobody ]
+ _doit=$'su -m nobody -c \'sh -c "/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t"\''
+ [ -n '' ]
+ [ -n '' ]
+ _doit=$' limits -C daemon su -m nobody -c \'sh -c "/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t"\''
+ _run_rc_doit $' limits -C daemon su -m nobody -c \'sh -c "/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t"\''
+ debug $'run_rc_command: doit:  limits -C daemon su -m nobody -c \'sh -c "/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t"\''
+ eval $' limits -C daemon su -m nobody -c \'sh -c "/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t"\''
+ limits -C daemon su -m nobody -c $'sh -c "/usr/sbin/daemon   -S -l daemon -s debug -t webhook \t-p /var/run/webhook.pid \t-u nobody \t/usr/local/sbin/webhook \t-hooks /usr/local/etc/webhook.yaml \t"'
+ _return=0
+ [ 0 -ne 0 ]
+ return 0
+ _run_rc_postcmd
+ [ -n '' ]
+ return 0
+ [ -n '' ]
+ return 0
daemon: failed to set user environment

I believe the issue is that the default start_cmd is using limits and su, which is creating an environment too restrictive for daemon.

Do you have a sample port with a working start script I can peek at?  I did look around a bit, but didn't immediately find one using daemon in this way.
Comment 11 Stefan Bethke 2017-12-11 21:08:35 UTC
Created attachment 188729 [details]
Updated version as per suggestions in this issue.
Comment 12 Dave Cottlehuber freebsd_committer freebsd_triage 2017-12-11 21:59:28 UTC
I can't test right now but does this help?

command_args="-f -t ${name} -p ${pidfile} /usr/bin/env ${webhook_env} ${procname} server -config=${webhook_conf}"

It's what I was using in my half-finished webhook.
Comment 13 Stefan Bethke 2017-12-11 22:37:38 UTC
(In reply to Dave Cottlehuber from comment #12)

I should have mentioned that I found grafana2/files/grafana2.in in the meantime, and used it to run daemon correctly.

The version on Github and the SHAR attached should be fully working.  I've tested on 10-stable and 11-stable.
Comment 14 Stefan Bethke 2017-12-13 20:57:18 UTC
Created attachment 188812 [details]
Port for webhook 2.6.7

webhook 2.6.7 was just released.  The attached shar has been updated to use the new version.

https://github.com/adnanh/webhook/releases/tag/2.6.7
Comment 15 Stefan Bethke 2017-12-22 18:36:35 UTC
Created attachment 189041 [details]
Updated port, version 2.6.8

The attachment contains the port with all requested improvements, updated to the current version 2.6.8

Release notes: https://github.com/adnanh/webhook/releases/tag/2.6.8
Comment 16 Stefan Bethke 2018-03-26 11:01:46 UTC
Ping. I would be very happy if someone could commit this, or tell me what changes are needed to get it committed.
Comment 17 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-26 12:11:21 UTC
Looks good.  I propose a few minor adjustments.

- Remove deprecated "Created by" line
- Conform to https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/book.html#porting-order
- Remove rclint warnings

https://reviews.freebsd.org/D14853

If you approve, I will commit with these changes.
Comment 18 Stefan Bethke 2018-03-26 12:13:20 UTC
Thank you, please go ahead!
Comment 19 commit-hook freebsd_committer freebsd_triage 2018-03-26 12:24:25 UTC
A commit references this bug:

Author: jrm
Date: Mon Mar 26 12:24:19 UTC 2018
New revision: 465583
URL: https://svnweb.freebsd.org/changeset/ports/465583

Log:
  New port, www/webhook: lightweight incoming webhook server

  WWW: https://github.com/adnanh/webhook

  PR:		222883
  Submitted by:	stb@lassitu.de (maintainer)
  Differential Revision:	https://reviews.freebsd.org/D14853

Changes:
  head/www/Makefile
  head/www/webhook/
  head/www/webhook/Makefile
  head/www/webhook/distinfo
  head/www/webhook/files/
  head/www/webhook/files/webhook.in
  head/www/webhook/files/webhook.yaml
  head/www/webhook/pkg-descr
Comment 20 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-26 12:26:26 UTC
Committed.  Thanks and sorry for the delay.