Bug 250723 - Add `set pipefail` to most framework scripts to ensure early failures
Summary: Add `set pipefail` to most framework scripts to ensure early failures
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL: https://reviews.freebsd.org/D27007
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-29 15:05 UTC by Mathieu Arnold
Modified: 2020-11-11 13:31 UTC (History)
2 users (show)

See Also:
mat: exp-run?


Attachments
v1 (13.75 KB, patch)
2020-10-29 15:05 UTC, Mathieu Arnold
no flags Details | Diff
v2 (14.71 KB, patch)
2020-11-02 09:57 UTC, Mathieu Arnold
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Arnold freebsd_committer freebsd_triage 2020-10-29 15:05:22 UTC
Created attachment 219202 [details]
v1

Patch from review D27007 attached.
Comment 1 Antoine Brodin freebsd_committer freebsd_triage 2020-10-31 08:53:54 UTC
Exp-run can't start, pkg can't fetch:

=======================<phase: fetch          >============================
===>  License BSD2CLAUSE accepted by the user
=> pipefail is not in /usr/ports/ports-mgmt/pkg/distinfo.
=> Either /usr/ports/ports-mgmt/pkg/distinfo is out of date, or
=> pipefail is spelled incorrectly.
*** Error code 1
Comment 2 Mathieu Arnold freebsd_committer freebsd_triage 2020-11-02 09:57:12 UTC
Created attachment 219299 [details]
v2

Test the thing a bit before sending a new patch.
Comment 3 Antoine Brodin freebsd_committer freebsd_triage 2020-11-04 07:47:19 UTC
On FreeBSD 11,  there is this new failure causing a lot of skipped:

http://package22.nyi.freebsd.org/data/114amd64-default-foo/2020-11-03_20h02m01s/logs/errors/mesa-libs-20.2.0_1.log

(probably an error in the port)
Comment 4 John Hein 2020-11-05 01:11:44 UTC
(In reply to Antoine Brodin from comment #3)
Yeah, there's likely going to be a number of ports with stale or bogus patch files that now fail.  Another example: bug 250871

Antoine, is there a list of more ports that are now failing at the 'patch' stage or does the exp-run stop at the first error?
Comment 5 Mathieu Arnold freebsd_committer freebsd_triage 2020-11-05 08:42:45 UTC
(In reply to John Hein from comment #4)

The exp-run is simply poudriere bulk running, it stops when it has exhausted all the ports it can build.
Comment 6 Mathieu Arnold freebsd_committer freebsd_triage 2020-11-05 08:53:14 UTC
(In reply to John Hein from comment #4)

Also, not to be rude or something, but as the one asking for the exp-run, I am the one who is fixing the fallout, if you also fix the fallout, I am going to spend hours working on stuff for nothing, and I really, really, hate wasting time. So, if you want to fix the fallout, be my guest, but tell me, I will then work on something else.
Comment 7 commit-hook freebsd_committer freebsd_triage 2020-11-05 16:44:55 UTC
A commit references this bug:

Author: mat
Date: Thu Nov  5 16:44:48 UTC 2020
New revision: 554140
URL: https://svnweb.freebsd.org/changeset/ports/554140

Log:
  Remove non existing patch.

  PR:		250723

Changes:
  head/graphics/mesa-dri/Makefile.common
Comment 8 John Hein 2020-11-05 19:48:01 UTC
(In reply to Mathieu Arnold from comment #6)
I asked if there was a list because only one failure was mentioned, and there was at least one other case known to me (math/maxima) that was a problem that was not mentioned.  It was not clear that there was a list anywhere that was being worked.  Seeing such a list would inform whether I could help and devote some time for fixes.

It sounds like you are actively working on it.  If you want help, let me know.  Otherwise, it's all yours.

The cases I know about have been fixed: bug 250530 and bug 250871
Comment 10 commit-hook freebsd_committer freebsd_triage 2020-11-09 11:42:57 UTC
A commit references this bug:

Author: mat
Date: Mon Nov  9 11:42:35 UTC 2020
New revision: 554710
URL: https://svnweb.freebsd.org/changeset/ports/554710

Log:
  Remove references to non existent patches.

  PR:		250723

Changes:
  head/audio/geonkick-lv2/Makefile
  head/devel/bazel/Makefile
  head/dns/knot3/Makefile
  head/lang/ecl/Makefile
  head/mail/neomutt/Makefile
  head/sysutils/u-boot-rpi3/Makefile
  head/sysutils/u-boot-rpi3-32/Makefile
Comment 11 Antoine Brodin freebsd_committer freebsd_triage 2020-11-10 11:29:51 UTC
This should be ok to commit,  the failures were mostly on leaf ports
Comment 12 commit-hook freebsd_committer freebsd_triage 2020-11-11 13:30:42 UTC
A commit references this bug:

Author: mat
Date: Wed Nov 11 13:29:57 UTC 2020
New revision: 554893
URL: https://svnweb.freebsd.org/changeset/ports/554893

Log:
  Add set pipefail in most framework scripts.

  set pipefail changes the pipeline return status from being the return
  status of the last command to the last non 0 exit status of any command
  in the pipeline.  This is needed to make sure all the commands in a
  pipeline did actually return a non 0 status and not only the last one.

  PR:		250723
  Exp-run by:	antoine
  Reviewed by:	bapt
  Approved by:	bapt
  Differential Revision:	https://reviews.freebsd.org/D27007

Changes:
  head/Mk/Scripts/check-desktop-entries.sh
  head/Mk/Scripts/check-stagedir.sh
  head/Mk/Scripts/check-vulnerable.sh
  head/Mk/Scripts/check_leftovers.sh
  head/Mk/Scripts/checksum.sh
  head/Mk/Scripts/create-manifest.sh
  head/Mk/Scripts/depends-list.sh
  head/Mk/Scripts/desktop-categories.sh
  head/Mk/Scripts/dialog4ports.sh
  head/Mk/Scripts/do-depends.sh
  head/Mk/Scripts/do-fetch.sh
  head/Mk/Scripts/do-patch.sh
  head/Mk/Scripts/do-users-groups.sh
  head/Mk/Scripts/find-lib.sh
  head/Mk/Scripts/generate-symbols.sh
  head/Mk/Scripts/install-desktop-entries.sh
  head/Mk/Scripts/makesum.sh
  head/Mk/Scripts/plist_sub_sed_sort.sh
  head/Mk/Scripts/ports_env.sh
  head/Mk/Scripts/qa.sh
  head/Mk/Scripts/rust-compat11-canary.sh
  head/Mk/Scripts/smart_makepatch.sh
Comment 13 commit-hook freebsd_committer freebsd_triage 2020-11-11 13:30:43 UTC
A commit references this bug:

Author: mat
Date: Wed Nov 11 13:30:03 UTC 2020
New revision: 554894
URL: https://svnweb.freebsd.org/changeset/ports/554894

Log:
  Ignore grep return value.

  Most of the time, we use grep in a test, where its return value is
  important.  In this instance, it is only used for filtering, and we do
  not care about its return value, so ignore it.

  PR:             250723
  Exp-run by:     antoine
  Differential Revision:  https://reviews.freebsd.org/D27007

Changes:
  head/Mk/Scripts/create-manifest.sh