Bug 190866

Summary: dfilter seems unable to process commit messages with more than one PR listed
Product: Services Reporter: John Marino <marino>
Component: Bug TrackerAssignee: Bryan Drewery <bdrewery>
Status: Closed FIXED    
Severity: Affects Many People CC: bdrewery
Priority: ---    
Version: unspecified   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191064
Attachments:
Description Flags
commit hook fix none

Description John Marino freebsd_committer freebsd_triage 2014-06-10 07:57:57 UTC
As of yesterday, dfilter seems to be able to process commit messages with a single PR listed fine.  However, when more than one PR is listed it appears nothing gets sent to any bug report.

Recent xamples of failure:

http://svnweb.freebsd.org/changeset/ports/357273
================================================
Reset maintainership.
  
PR:		190858
PR:		190859
PR:		190860
PR:		190861
PR:		190862
PR:		190863
PR:		190864
Submitted by:	former maintainer
Approved by:	portmgr (blanket)
================================================


http://svnweb.freebsd.org/changeset/ports/357216
================================================
net/jicmp6: USES+= libtool, then reset maintainer

Thanks for being this port's first maintainer!

PR:		ports/190465
PR:		ports/190838
Submitted by:	maintainer (Sevan Janiyan)
================================================


http://svnweb.freebsd.org/changeset/ports/357211
================================================
databases: iplike[1], jrrd[2]: reset maintainer

No reason offered.  Thanks for maintaining these for a few years.

PR:		ports/190839 [1]
PR:		ports/190840 [2]
Submitted by:	maintainer (Sevan Janiyan)
================================================
Comment 1 John Marino freebsd_committer freebsd_triage 2014-06-14 19:46:22 UTC
Today I made a commit with the following message:

==================================================
devel/ecgi: Remove double entry from plist-plist

When PORTDOCS is used, the docs files can't be listed in pkg-plist,
otherwise they get listed twice (thus two attempts to remove them are
made).  This only counts as a warning and not fatal for Redports, which
is how it slipped by me.

Related PR: 189010
==================================================

I expected it to annotate PR 189010, but it did not.
Should it have?
Comment 2 Bryan Drewery freebsd_committer freebsd_triage 2014-06-19 00:43:20 UTC
(In reply to John Marino from comment #1)
> Today I made a commit with the following message:
> 
> ==================================================
> devel/ecgi: Remove double entry from plist-plist
> 
> When PORTDOCS is used, the docs files can't be listed in pkg-plist,
> otherwise they get listed twice (thus two attempts to remove them are
> made).  This only counts as a warning and not fatal for Redports, which
> is how it slipped by me.
> 
> Related PR: 189010
> ==================================================
> 
> I expected it to annotate PR 189010, but it did not.
> Should it have?

Not on this one, needs ^PR: format.
Comment 3 Bryan Drewery freebsd_committer freebsd_triage 2014-06-19 00:50:17 UTC
Created attachment 143914 [details]
commit hook fix

1. Fix handling of 'PR: XXXXX [stuff](here)'
2. Fix handling of multiple PR in 1 commit
Comment 4 Bryan Drewery freebsd_committer freebsd_triage 2014-06-19 00:52:46 UTC
It may be non-obvious. I added '.*' at the end of the each regex to swallow any remaining text.

From marino's example:
# svn log -vr r357273 ^/
------------------------------------------------------------------------
r357273 | linimon | 2014-06-10 01:31:43 -0500 (Tue, 10 Jun 2014) | 12 lines
Changed paths:
   M /head/databases/ocaml-mysql/Makefile
   M /head/devel/ats-contrib-testing/Makefile
   M /head/lang/ats/Makefile
   M /head/lang/clay/Makefile
   M /head/lang/rust/Makefile
   M /head/textproc/ats-contrib-parcomb/Makefile
   M /head/textproc/rubygem-ezamar/Makefile

Reset maintainership.

PR:             190858
PR:             190859
PR:             190860
PR:             190861
PR:             190862
PR:             190863
PR:             190864
Submitted by:   former maintainer
Approved by:    portmgr (blanket)

------------------------------------------------------------------------


# svn log -vr r357273 ^/|sed -nE -e 's/^[     ]*[pP][rR]:[    ]*[a-zA-Z]+\/([0-9]+).*/\1/p' -e 's/^[  ]*[pP][rR]:[    ]*([0-9]+).*/\1/p'
190858
190859
190860
190861
190862
190863
190864


From my own:

 # svn log -vr 358332 ^/
------------------------------------------------------------------------
r358332 | bdrewery | 2014-06-18 19:36:18 -0500 (Wed, 18 Jun 2014) | 11 lines
Changed paths:
   M /head/ports-mgmt/portmaster/Makefile
   M /head/ports-mgmt/portmaster/distinfo

- Update to 3.17.6

Changes:
  * Speedup --list-origins with pkg
  * Use proper /usr/local/etc/portmaster.rc path in portmaster.8
  * Fix running from deleted dirs.
  * Fix losing control of building run-depends of staged ports
    (thus breaking -g and counts). [1]

PR:             189398 [1]

------------------------------------------------------------------------

# svn log -vr 358332|sed -nE -e 's/^[         ]*[pP][rR]:[    ]*[a-zA-Z]+\/([0-9]+).*/\1/p' -e 's/^[  ]*[pP][rR]:[    ]*([0-9]+).*/\1/p'
189398
Comment 5 Peter Wemm freebsd_committer freebsd_triage 2014-06-19 04:45:46 UTC
The patch looks reasonable by inspection.

It is safe to commit and then do a test - nothing will break if the post-commit hook fails for some reason.

update-root.sh runs before the bugzilla notify hook, so you can test in production with a single commit by referencing this bug. :)

Just remember there's up to a 60 second processing delay, so don't panic if its not there immediately.
Comment 6 Bryan Drewery freebsd_committer freebsd_triage 2014-06-19 13:52:02 UTC
There's also a patch in Bug 191064
Comment 7 John Marino freebsd_committer freebsd_triage 2014-06-26 10:47:44 UTC
was the patch applied?  Has dfilter been fixed?  I can't really tell from the comments.
Comment 8 John Marino freebsd_committer freebsd_triage 2014-07-10 16:00:35 UTC
hmm, so according to bdrewery there are at least two bug reports on this with patches, but have any of them been applied yet?

Or in other words, was dfilter ever fixed or is it still confused by notation that used to work?
Comment 9 John Marino freebsd_committer freebsd_triage 2014-07-28 23:09:21 UTC
And the answer is: "No, dfilter was never fixed".

I just tried it, the two PRs below didn't get updated.
So is there any estimate available about when this will work again?

============================================================

Author: marino
Date: Mon Jul 28 22:58:33 2014
New Revision: 363270
URL: http://svnweb.freebsd.org/changeset/ports/363270
QAT: https://qat.redports.org/buildarchive/r363270/

Log:
  lang/rust: Fix link issue and build failure
  
  [1] Linking fails with missing reference to __morestack symbol
  [2] gpy fails with error about missing sem implementation
  
  PR:		189357 [1]
  PR:		191927 [2]
  Submitted by:	bertrand.augereau (gmail) [1]
  Submitted by:	maintainer: mitsuruike (gmail) [2]

Added:
  head/lang/rust/files/patch-src__libuv__gyp_uv.py   (contents, props changed)
Modified:
  head/lang/rust/Makefile
Comment 10 commit-hook freebsd_committer freebsd_triage 2014-09-08 16:10:23 UTC
A commit references this bug:

Author: bdrewery
Date: Mon Sep  8 16:09:47 UTC 2014
New revision: 367646
URL: http://svnweb.freebsd.org/changeset/ports/367646

Log:
  Fix bugzilla dfilter script to handle:
    1. Multiple PR per line
    2. Multiple PR
    3. PR lines with extra stuff after them such as references and (notes)

  PR:		ports/190866 [1]
  PR:		ports/191064 [2]
  Reported by:	many
  Submitted by:	ak, bdrewery

Changes:
  svnadmin/hooks/scripts/notify_bz.sh
Comment 11 John Marino freebsd_committer freebsd_triage 2014-09-08 16:14:32 UTC
interestingly, the fix seemed to work on it's own commit.  Now that's proof! :)
Comment 12 John Marino freebsd_committer freebsd_triage 2014-09-09 11:30:24 UTC
The issue has been resolved, at least as far as I (the submitter) is concerned.
Comment 13 commit-hook freebsd_committer freebsd_triage 2014-09-11 00:11:02 UTC
A commit references this bug:

Author: bdrewery
Date: Thu Sep 11 00:10:54 UTC 2014
New revision: 271412
URL: http://svnweb.freebsd.org/changeset/ports/271412

Log:
  Fix bugzilla dfilter script to handle:
    1. Multiple PR per line
    2. Multiple PR
    3. PR lines with extra stuff after them such as references and (notes)

  PR:		ports/190866 [1]
  PR:		ports/191064 [2]
  Reported by:	many
  Submitted by:	ak, bdrewery
  Acked by:	peter

Changes:
  svnadmin/hooks/scripts/notify_bz.sh
Comment 14 commit-hook freebsd_committer freebsd_triage 2014-09-11 00:20:05 UTC
A commit references this bug:

Author: bdrewery
Date: Thu Sep 11 00:19:38 UTC 2014
New revision: 45589
URL: http://svnweb.freebsd.org/changeset/ports/45589

Log:
  Fix bugzilla dfilter script to handle:
    1. Multiple PR per line
    2. Multiple PR
    3. PR lines with extra stuff after them such as references and (notes)

  PR:		190866 [1]
  PR:		191064 [2]
  Reported by:	many
  Submitted by:	ak, bdrewery
  Approved by:	doceng (gjb)

Changes:
  svnadmin/hooks/scripts/notify_bz.sh
Comment 15 Bryan Drewery freebsd_committer freebsd_triage 2014-09-11 00:24:35 UTC
Fixed in ports/base/doc.
Comment 16 John Marino freebsd_committer freebsd_triage 2014-10-15 11:38:40 UTC
Re-opening.

I just committed this:

Author: marino
Date: Wed Oct 15 11:32:09 2014
New Revision: 370906
URL: https://svnweb.freebsd.org/changeset/ports/370906
QAT: https://qat.redports.org/buildarchive/r370906/

Log:
  Close out a bunch of PRs that only remove @dirrm
  
  PR:		194331, 194332
  PR:		194365, 194366, 194369, 194371, 194374


Only two PRs were pinged: 194332 and 194374
Was that the correct result?
Do we really have to limit each line to 1 PR?
Comment 17 Bryan Drewery freebsd_committer freebsd_triage 2014-10-15 13:54:24 UTC
I'll look into this and fix it
Comment 18 John Marino freebsd_committer freebsd_triage 2015-02-23 09:55:07 UTC
So what was the conclusion?

I guess it is either
A) Yes, add one line per PR
B) No, it already takes multiple PRs per line, but under this format (what's the format?)
Comment 19 commit-hook freebsd_committer freebsd_triage 2015-03-19 16:13:55 UTC
A commit references this bug:

Author: bdrewery
Date: Thu Mar 19 16:13:46 UTC 2015
New revision: 381608
URL: https://svnweb.freebsd.org/changeset/ports/381608

Log:
  Fix dfilter script to support PR lines with commas.

  PR:		190866
  Approved by:	portmgr (implicit)

Changes:
  svnadmin/hooks/scripts/notify_bz.sh
Comment 20 commit-hook freebsd_committer freebsd_triage 2015-03-19 16:14:57 UTC
A commit references this bug:

Author: bdrewery
Date: Thu Mar 19 16:14:28 UTC 2015
New revision: 280265
URL: https://svnweb.freebsd.org/changeset/base/280265

Log:
  Fix dfilter script to support PR lines with commas.

  PR:		190866

Changes:
  svnadmin/hooks/scripts/notify_bz.sh
Comment 21 commit-hook freebsd_committer freebsd_triage 2015-03-19 16:15:58 UTC
A commit references this bug:

Author: bdrewery
Date: Thu Mar 19 16:15:31 UTC 2015
New revision: 46357
URL: https://svnweb.freebsd.org/changeset/doc/46357

Log:
  Fix dfilter script to support PR lines with commas.

  PR:		190866
  Approved by:	doceng (implicit from previous updates)

Changes:
  svnadmin/hooks/scripts/notify_bz.sh
Comment 22 Bryan Drewery freebsd_committer freebsd_triage 2015-03-19 16:16:36 UTC
(In reply to John Marino from comment #16)

It should be good now. I tested with all of the commits listed in this
Bug and all listed all PR.
Comment 23 John Marino freebsd_committer freebsd_triage 2015-03-19 16:51:22 UTC
Thanks Bryan!