Bug 222355 - "make check-plist" can overflow sed's line buffer
Summary: "make check-plist" can overflow sed's line buffer
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Port Management Team
URL:
Keywords:
: 192777 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-09-15 17:19 UTC by Alan Somers
Modified: 2018-07-26 20:24 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alan Somers freebsd_committer freebsd_triage 2017-09-15 17:19:50 UTC
"make check-plist" generates a long sed command.  For some ports, this command is so long that it overflows sed's line buffer, sized to _POSIX2_LINE_MAX (currently 2048 characters).  An example port that demonstrates this problem is www/rubygem-passenger.  Running "make check-plist" will print this error:

sed: 1: "s!^/usr/local/!!g;  s!l ...": unterminated regular expression

The full sed command is:

sed $'s!^/usr/local/!!g;  s!lib/ruby/gems/2\\.3/specifications/passenger-5\\.1\\.8\\.gemspec!%%GEM_SPEC%%!g;  s!lib/ruby/gems/2\\.3/cache/passenger-5\\.1\\.8\\.gem!%%GEM_CACHE%%!g;  s!lib/ruby/vendor_ruby/2\\.3/amd64-freebsd11!%%RUBY_VENDORARCHLIBDIR%%!g;  s!lib/ruby/site_ruby/2\\.3/amd64-freebsd11!%%RUBY_SITEARCHLIBDIR%%!g;  s!lib/ruby/gems/2\\.3/gems/passenger-5\\.1\\.8!%%GEM_LIB_DIR%%!g;  s!lib/ruby/gems/2\\.3/doc/passenger-5\\.1\\.8!%%GEM_DOC_DIR%%!g;  s!lib/ruby/gems/2\\.3/specifications!%%SPEC_DIR%%!g;  s!share/examples/ruby23/passenger!%%RUBY_MODEXAMPLESDIR%%!g;  s!lib/ruby/gems/2\\.3/extensions!%%EXT_DIR%%!g;  s!lib/ruby/2\\.3/amd64-freebsd11!%%RUBY_ARCHLIBDIR%%!g;  s!share/doc/ruby23/passenger!%%RUBY_MODDOCDIR%%!g;  s!share/examples/passenger!%%EXAMPLESDIR%%!g;  s!lib/ruby/vendor_ruby/2\\.3!%%RUBY_VENDORLIBDIR%%!g;  s!lib/ruby/gems/2\\.3/cache!%%CACHE_DIR%%!g;  s!lib/ruby/site_ruby/2\\.3!%%RUBY_SITELIBDIR%%!g;  s!lib/ruby/gems/2\\.3/gems!%%GEMS_DIR%%!g;  s!share/examples/ruby23!%%RUBY_EXAMPLESDIR%%!g;  s!lib/ruby/gems/2\\.3/doc!%%DOC_DIR%%!g;  s!share/ri/2\\.3/system!%%RUBY_RIDIR%%!g;  s!share/doc/passenger!%%DOCSDIR%%!g;  s!share/ri/2\\.3/site!%%RUBY_SITERIDIR%%!g;  s!lib/ruby/gems/2\\.3!%%GEMS_BASE_DIR%%!g;  s!share/doc/ruby23!%%RUBY_DOCDIR%%!g;  s!libexec/apache24!%%APACHEMODDIR%%!g;  s!include/apache24!%%APACHEINCLUDEDIR%%!g;  s!share/passenger!%%DATADIR%%!g;  s!passenger-5\\.1\\.8!%%PORT%%!g;  s!passenger-5\\.1\\.8!%%GEM_NAME%%!g;  s!amd64-freebsd11!%%RUBY_ARCH%%!g;  s!www/passenger!%%WWWDIR%%!g;  s!etc/passenger!%%ETCDIR%%!g;  s!passenger\\.so!%%AP_MODULE%%!g;  s!lib/ruby/2\\.3!%%RUBY_LIBDIR%%!g;  s!etc/apache24!%%APACHEETCDIR%%!g;  s!passenger!%%AP_NAME%%!g;  s!ruby23!%%RUBY_NAME%%!g;  s!5\\.1\\.8!%%PORTVERSION%%!g;  s!2\\.3\\.4!%%RUBY_VERSION%%!g;  s!2\\.3!%%RUBY_VER%%!g;  s!24!%%APACHE_VERSION%%!g;  s!23!%%RUBY_SUFFIX%%!g;  s!23!%%RUBY_SHLIBVER%%!g;  s!23!%%RUBY_DEFAULT_SUFFIX%%!g;  s!-A!%%AP_MOD_EN%%!g; s,^,@dir ,; \t    /%%DOCSDIR%%/s!^!%%PORTDOCS%%!g; /%%EXAMPLESDIR%%/s!^!%%PORTEXAMPLES%%!g; /^%%PORTDOCS%%/d; \t    /^@dir share\\/licenses/d;'

I think the best solution would be to split up that command into multiple smaller commands, but I don't know that part of ports well enough to do it myself.
Comment 1 Mathieu Arnold freebsd_committer freebsd_triage 2017-09-16 08:59:07 UTC
which port produces that ?
Comment 2 Alan Somers freebsd_committer freebsd_triage 2017-09-16 16:39:44 UTC
www/rubygem-passenger, as I said.
Comment 3 Mathieu Arnold freebsd_committer freebsd_triage 2018-01-22 14:29:52 UTC
Working on it in https://reviews.freebsd.org/D14014
Comment 4 Matthias Fechner freebsd_committer freebsd_triage 2018-07-21 08:53:31 UTC
I see the same problem with textproc/rubygem-gitlab-markup which I currently updating to 1.6.4.
Comment 5 Matthias Fechner freebsd_committer freebsd_triage 2018-07-21 08:56:13 UTC
If it help, here the build log:
https://pkg.fechner.net/data/111amd64-gitlab/2018-07-21_10h54m18s/logs/rubygem-gitlab-markup-1.6.4.log
Comment 6 Mathieu Arnold freebsd_committer freebsd_triage 2018-07-21 09:09:01 UTC
Good, good, and does the proposed fix works?
Comment 7 Matthias Fechner freebsd_committer freebsd_triage 2018-07-26 06:51:07 UTC
(In reply to Mathieu Arnold from comment #6)
yes, I can confirm that the patch from https://reviews.freebsd.org/D14014 fixes the problem.
Comment 9 commit-hook freebsd_committer freebsd_triage 2018-07-26 11:10:08 UTC
A commit references this bug:

Author: mat
Date: Thu Jul 26 11:09:47 UTC 2018
New revision: 475361
URL: https://svnweb.freebsd.org/changeset/ports/475361

Log:
  Extract PLIST_SUB substitutions into a sed script.

  This fixes PLIST_SUB being too big and PLIST_SUB_SED getting bigger than
  _POSIX2_LINE_MAX.

  PR:		222355
  Reported by:	asomers
  Reviewed by:	asomers, mfechner
  Sponsored by:	Absolight
  Differential Revision:	 https://reviews.freebsd.org/D14014

Changes:
  head/Mk/Scripts/check-stagedir.sh
  head/Mk/Scripts/check_leftovers.sh
  head/Mk/Scripts/plist_sub_sed_sort.sh
Comment 10 Raphael Kubo da Costa freebsd_committer freebsd_triage 2018-07-26 20:24:29 UTC
*** Bug 192777 has been marked as a duplicate of this bug. ***