Bug 257063 - archivers/star: Do not install 'gnutar' for it is broken
Summary: archivers/star: Do not install 'gnutar' for it is broken
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Juraj Lutter
Depends on:
Reported: 2021-07-08 22:44 UTC by ice
Modified: 2021-09-01 01:34 UTC (History)
5 users (show)

See Also:
fuz: maintainer-feedback+

devel/schilybase: update to 2021.07.29 and fix issues (9.98 KB, patch)
2021-07-29 21:13 UTC, Robert Clausecker
fuz: maintainer-approval+
Details | Diff
devel/schilybase: fix PR #257063 (2021Q3) (9.27 KB, patch)
2021-07-30 15:20 UTC, Robert Clausecker
no flags Details | Diff
devel/schilybase: fix PR #257063 (2021Q3) (9.75 KB, patch)
2021-07-31 07:59 UTC, Robert Clausecker
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ice 2021-07-08 22:44:04 UTC
The recent addition of 'gnutar' to point to 'star' prohibits (at least) Firefox from building correctly:

../../config/nsinstall -D /usr/ports/www/firefox-esr/work/stage/usr/local/lib/firefox                                                   
(cd ../../dist/firefox && /usr/local/bin/gnutar --exclude=precomplete --exclude=.mkdir.done -chf - .) | \
  (cd /usr/ports/www/firefox-esr/work/stage/usr/local/lib/firefox && tar -xf -)                                                         
gnutar: Bad Option: -chf.                                                                                                               
Usage:  gnutar cmd [options] file1 ... filen                                                                                            
Use     gnutar --help                                                                                                                   
and     gnutar --xhelp                                                                                                                  
to get a list of valid cmds and options.                                                                                                
Use     gnutar -H help                                                                                                                  
to get a list of valid archive header formats.                                                                                          
For a more complete user interface use the star type command interface.                                                                 
See 'man star'. The gnutar command is more or less limited to the                                                                       
GNU tar command line interface.                                                                                                         
../../config/nsinstall -D /usr/ports/www/firefox-esr/work/stage/usr/local/bin
rm -f -f /usr/ports/www/firefox-esr/work/stage/usr/local/bin/firefox
ln -s /usr/local/lib/firefox/firefox /usr/ports/www/firefox-esr/work/stage/usr/local/bin
gmake[3]: Leaving directory '/usr/ports/www/firefox-esr/work/.build/browser/installer'

I believe the error is because something in here finds the 'gnutar' executable as a suitable GNU tar to use, which has recently been installed by archivers/star:

$ ls -l /usr/local/bin/gnutar
lrwxr-xr-x  1 root  wheel  4 Jul  2 00:22 /usr/local/bin/gnutar -> star

star is rather obviously completely unsuitable as a stand-in for GNU tar -- the  arguments star chokes on have been completely fine for GNU tar for close to two decades (I do not have an older one handy to check).

Please stop installing the 'gnutar' link.
Comment 1 Robert Clausecker 2021-07-08 23:04:20 UTC
(In reply to ice from comment #0)


It is unfortunate that Firefox-esr wrongly detects star's gnutar binary as a GNU tar.  This seems to be an error in the way the port is set up: archivers/gtar is not given as a build dependency and yet the port looks for a GNU tar binary.  Why is that so?  Adding the maintainer of the www/firefox-esr port to CC.

I would like to continue providing a gnutar binary if possible as that binary is part of the star distribution and its name does not collide with any other command I am aware of (FreeBSD ports install GNU tar as gtar not gnutar, so there is no collision here).  If a port wrongly picks this program over /usr/bin/tar or gtar, the port needs to be patched to always pick the correct tar program.  I mean we don't let ports pick random gcc binaries over ${CC} either...

If there is no other remedy to the issue, I can however go ahead and set up an option to make installing the gnutar binary conditional.
Comment 2 ice 2021-07-08 23:14:28 UTC
Fair reasoning, I certainly see the point you are making, and am not opposed.


$ man gnutar
GNUTAR(1)                    Schily´s USER COMMANDS                    GNUTAR(1)

       gnutar - emulates the GNU version of the tar archiver

This may have been true before dinosaurs roamed the Earth, but simply isn't true today, and hasn't been true for a very, very long time.
Comment 3 Robert Clausecker 2021-07-08 23:33:10 UTC
(In reply to ice from comment #2)

Okay I get your point.  Here's a proposed action plan:

 - I have reported the incompatible GNU tar emulation upstream, so with some
   luck, star will get better at mimicking GNU tar in the future.

 - I am going to make installing a gnutar binary optional with a
   non-default option as I agree that the behaviour is a bit surprising.
   I am planning to push this change with the next schilytools release
   which upstream told me will come in a week or two.

 - as a temporary workaround, unlink the gnutar binary:

       rm /usr/local/bin/gnutar

 - I believe I have identified the problem with firefox-esr.  In the file
   toolkit/moz.configure there is a line that says.

       check_prog('TAR', ('gnutar', 'gtar', 'tar'))

   If you change this line to say just

       check_prog('TAR', ('tar'))

   the problem should go away.  Please test this change and let me know if
   it fixes the issue for you.  If yes, I'll go ahead and file a patch
   for www/firefox-esr to harden its build in the presence of devel/star
   with the gnutar binary installed.
Comment 4 ice 2021-07-09 07:39:16 UTC
(In reply to Robert Clausecker from comment #3)

Sounds entirely reasonable, thank you.

Patching toolkit/moz.configure to only check for 'tar' (and thus finding base bsdtar) does allow a seemingly proper Firefox package to be built even in the presence of star's 'gnutar'.
Comment 5 Jörg Schilling 2021-07-09 13:47:43 UTC
The problem is that firefox seems to use an incorrect GNU tar command line.

Non-standard options like --exclude need to appear after the TAR key letter -c

The background is that GNU tar claims to implement a 100% TAR compatible CLI and TAR does not use a standard option parser to parse the options. Unfortunately, GNU tar is wrong with it's claims....and implements a CLI that is very different from the original TAR CLI.

In order to support the official TAR CLI, star calls a function that modifies the command line in a way that it later may be parsed by a standard option parser.

Option parsing in GNU tar always was a big problem ans many user still complain about problems they observe because of the strange option parsing in GNU tar.

If recent GNU tar versions changed their behaviour and firefox adopted to that, this seems to be a firefox problem.

BTW: GNU tar is never installed as "gnutar". This is why the name "gnutar" has been selected by star to switch to the GNU tar CLI.

So my proposal:

Either change firefox to have the non-standard options like --exclude appear late in the command line, or tell firefox to look for "gnutar" last, but never first.
Comment 6 Jörg Schilling 2021-07-09 13:55:18 UTC
Just in case I was not obvious....

 gtar --exclude=precomplete  chf - .

fails the same way as star failt in gtar emulation mode.
Comment 7 ice 2021-07-09 14:12:00 UTC
(In reply to Jörg Schilling from comment #6)

Just to be pedantic, the command line used is "/usr/local/bin/gnutar --exclude=precomplete --exclude=.mkdir.done -chf - ." (note *dash* chf), which GNU tar going back decades as well as libarchive tar accepts just fine.

On a passingly related note, gnutar's man page claims to "emulate the GNU version of the tar archiver". What GNU tar does or does not do right or wrong is completely irrelevant. The claim is GNU tar emulation, not "this is the implementation GNU tar should have been".
Comment 8 Jörg Schilling 2021-07-09 14:31:20 UTC
A real TAR always expects the key letter to be the first argument and ignores a dash in the first argument of TAR, if a dash is present. GNU tar is incompatible to a real TAR.

GNU tar never installs a binary called "gnutar". The only program that installs a binary named "gnutar" is star.

Since firefox explicitely decided to call "gnutar", it should know that the command line in question did never work with a binary called "gnutar".

BTW: star (when called as gnutar) always first rewrites the command line before calling the option parser.

In theory, I could not call this CLI rewriting function, in case the first character of the first argument is a dash, but then star's GNU tar emulation would be as dangerous as the original GNU tar and could easily destroy the content of the first file type argument by incorrectly taking it as the archive name.

Star currently prevents that from happening.... do you really prefer to destroy files that are expected to be archived just by a simple misspelling of the command line, ... in favor of implementing the same TAR incompatibility as seen by GNU tar?
Comment 9 Robert Clausecker 2021-07-09 15:53:02 UTC
Filed a patch for www/firefox-esr in bug #257084.
Comment 10 Robert Clausecker 2021-07-29 21:13:11 UTC
Created attachment 226785 [details]
devel/schilybase: update to 2021.07.29 and fix issues

The attached patch updates the schilytool ports to 2021.07.29 and adds the mentioned GNUTAR option.

Please apply to quarterly sans the version update (instead bump to devel/schilybase/Makefile.master PORTREVISION) to fix this issue there, too.  I can also make a separate patch for quarterly if desired.

Summary of changes:

 - archivers/star: install gnutar binary conditional on GNUTAR option (see PR #257063)
 - net/rscsi: have pkg-message show the right PREFIX
 - move README.rscsi from sysutils/cdrtools to this port so the advice given in pkg-message is actually useful.
 - devel/schilybase: improve build output
 - devel/sunpromake: register conflicts with devel/dmake
 - devel/schilybase: use ECHO_MSG instead of echo
 - devel/schilybase: use %%OPSYS%% instead of hard-coding freebsd in pkg-plist.  Perhaps this way the port can also be built on DragonFlyBSD.
Comment 11 commit-hook freebsd_committer 2021-07-30 13:28:11 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=774298c0f68f7b5f67dbde44d046184807423450

commit 774298c0f68f7b5f67dbde44d046184807423450
Author:     Robert Clausecker <fuz@fuz.su>
AuthorDate: 2021-07-30 11:52:31 +0000
Commit:     Juraj Lutter <otis@FreeBSD.org>
CommitDate: 2021-07-30 13:26:47 +0000

    devel/schilybase: update to 2021.07.29

    - Update to 2021.07.29
    - Fix related PR (Do not install broken gnutar by default)

    PR:             257063
    Reported by:    ice@extreme.hu

 archivers/star/Makefile                           |  9 ++++-----
 archivers/star/pkg-plist                          | 13 ++-----------
 devel/schilybase/Makefile.master                  | 18 +++++++++---------
 devel/schilybase/distinfo                         |  6 +++---
 devel/schilybase/files/pkg-message.rscsi.in (new) |  5 +++++
 devel/schilybase/pkg-plist                        |  6 +++---
 devel/sunpromake/Makefile                         |  2 ++
 net/rscsi/Makefile                                |  7 +++++++
 net/rscsi/pkg-message (gone)                      |  6 ------
 net/rscsi/pkg-plist                               |  1 +
 sysutils/cdrtools/Makefile                        |  3 +++
 sysutils/cdrtools/pkg-plist                       |  1 -
 sysutils/schilyutils/Makefile                     |  2 ++
 13 files changed, 41 insertions(+), 38 deletions(-)
Comment 12 Robert Clausecker 2021-07-30 15:20:29 UTC
Created attachment 226799 [details]
devel/schilybase: fix PR #257063 (2021Q3)

Patch against 2021Q3, as requested.
Comment 13 Robert Clausecker 2021-07-31 07:59:59 UTC
Created attachment 226820 [details]
devel/schilybase: fix PR #257063 (2021Q3)

Fixed patch against quarterly.  Should work now.  Tested on arm64 FreeBSD 13 with Poudriere.
Comment 14 commit-hook freebsd_committer 2021-08-05 18:57:33 UTC
A commit in branch 2021Q3 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=078db1f62fa8de558815f82e813ff919a8f90fda

commit 078db1f62fa8de558815f82e813ff919a8f90fda
Author:     Robert Clausecker <fuz@fuz.su>
AuthorDate: 2021-08-05 18:51:58 +0000
Commit:     Juraj Lutter <otis@FreeBSD.org>
CommitDate: 2021-08-05 18:56:38 +0000

    misc/schilytools: Do not install broken gnutar by default

    - Do not install broken gnutar by default

    This is a direct commit to 2021Q3 based on a commit
    774298c0f68f7b5f67dbde44d046184807423450 in main.

    Reported by:    ice@extreme.hu
    PR:             257063

 archivers/star/Makefile                           |  9 ++++-----
 archivers/star/pkg-plist                          | 13 ++-----------
 devel/schilybase/Makefile.master                  | 21 +++++++++++----------
 devel/schilybase/files/pkg-message.rscsi.in (new) |  5 +++++
 devel/schilybase/pkg-plist                        |  6 +++---
 devel/sunpromake/Makefile                         |  2 ++
 net/rscsi/Makefile                                |  7 +++++++
 net/rscsi/pkg-message (gone)                      |  6 ------
 net/rscsi/pkg-plist                               |  1 +
 sysutils/cdrtools/Makefile                        |  3 +++
 sysutils/cdrtools/pkg-plist                       |  1 -
 sysutils/schilyutils/Makefile                     |  4 ++--
 12 files changed, 40 insertions(+), 38 deletions(-)
Comment 15 Juraj Lutter freebsd_committer 2021-08-06 07:06:03 UTC
Committed, thanks.