Bug 209211

Summary:	11.0-CURRENT r298192 (file 5.26) breaks portupgrade and more on amd64 and powerpc64 (and more); its MFC r298920 breaks on 10-STABLE
Product:	Base System	Reporter:	Mark Millard <marklmi26-fbsd>
Component:	bin	Assignee:	Xin LI <delphij>
Status:	Closed FIXED
Severity:	Affects Many People	CC:	dpetrov67, junchoon, peter.thoenen, tschweikle
Priority:	---
Version:	CURRENT
Hardware:	Any
OS:	Any
See Also:	https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209237

Description Mark Millard 2016-05-02 18:47:50 UTC

In reply to list notices for amd64 I had written for powerpc64:

On 2016-Apr-26, at 11:16 AM, Mark Millard <markmi at dsl-only.net> wrote:

https://lists.freebsd.org/pipermail/freebsd-ports/2016-April/102983.html reported a "Broken pipe" problem with portupgrade on 11.0-CURRENT on amd64.

FYI: I had the/a "Broken pipe" portupgrade problem on powerpc64 under 11.0-CURRENT -r298518 on 2016-Apr-23 updating from /usr/ports -r413230 to -r413919. If this might be the same I do not know. The relevant part of the script log is:

====> Compressing man pages (compress-man)
--->  Backing up the old version
--->  Uninstalling the old version
[Reading data from pkg(8) ... - 68 packages found - done]
--->  Deinstalling 'perl5-5.22.1_7'
[Reading data from pkg(8) ... - 68 packages found - done]
** Listing the failed packages (-:ignored / *:skipped / !:failed)
       ! perl5-5.22.1_7        (Broken pipe)
--->  Skipping 'ports-mgmt/portlint' (portlint-2.16.8) because a requisite package 'perl5-5.22.1_7' (lang/perl5.22) failed (specify -k to force)

ruby was still lang/ruby21 at the time.

I used portmaster instead and everything worked fine.

===
Mark Millard
markmi at dsl-only.net



Now there is a reply to my note that identifies -r298192 (11.0-CURRENT) and -r298920 (MFC to 10-STABLE) as the revision that breaks portupgrade. . .



On Mon May 2 15:44:19 UTC 2016 Tomoaki AOKI wrote:

Hi.

Today I encountered this problem on stable/10 and could determine the
problematic commit is r298920, "Update file to 5.26".

This commit is MFC of r298192, and reverting it fixes the issue on
head, too.

What I did (for stable/10) is...

 1) Running at r298836: No problem.
 2) Update to r298920:  Problem is reproduced!
 3) Downgrade to previous commit (r298889): No problem.

3) was done by `zfs rollback` to r298836 state, `svnlite up r298889',
and proceeded usual rebuilding procedure. 

Fortunately, there was only 3 commits between r298836 and r298920,
and I got right one in first attempt.

But unfortunately, fixing portupgrade[-devel] or file/libmagic beyonds
my hand. :-<

Comment 1 Mark Linimon freebsd_committer

2016-05-03 00:12:59 UTC

Over to committer of file(1) changes (r298192).

Comment 2 Tomoaki AOKI 2016-05-07 12:32:22 UTC

Akinori MUSHA (original developer of ports-mgmt/portupgrade) noted that this problem is caused by upstream (file 5.26) bug on freebsd-current ML.

See https://lists.freebsd.org/pipermail/freebsd-current/2016-May/061072.html.
For more detail, see the first URL (Upstream bugtracker) mentioned in the post.

As mentioned by Akinori, file 5.26 has another severe bug.
So r298192 (head) and r298920 (stable/10) would be bettter reverted for now
(Back to file 5.25.) and wait for upstream fix.

Comment 3 Xin LI freebsd_committer

2016-05-08 09:31:01 UTC

Committed a fix as r299234 to head/.  MFC reminder.

Comment 4 commit-hook freebsd_committer

2016-05-08 09:31:27 UTC

A commit references this bug:

Author: delphij
Date: Sun May  8 09:30:36 UTC 2016
New revision: 299234
URL: https://svnweb.freebsd.org/changeset/base/299234

Log:
  MFV r299233: Apply upstream fix 11edb37a71851b5bcbd4e51ca6ad3dcbf57e9761:

  Revert: Don't close stdin...

  This have been observed by portupgrade in its interaction with file(1),
  as reported as a regression in bin/209211.

  PR:		209211
  MFC after:	3 days

Changes:
_U  head/contrib/file/
  head/contrib/file/src/magic.c

Comment 5 Tomoaki AOKI 2016-05-08 13:43:01 UTC

Confirmed fixed by manually applying to stable/10, r299232.
Thanks!

But please note that another bug Akinori mentioned would not be fixed.
Viewing upstream github (public, readonly one), fix for upstream head haven't been sync'ed yet.

 *On upstream bugtracker, said to be fixed at 6 May, but latest entry on github
  was 12 days ago (should be 27 April). So don't know how to pull in the fix.

Comment 6 Xin LI freebsd_committer

2016-05-08 18:09:45 UTC

(In reply to Tomoaki AOKI from comment #2)
I see, I'll revert it in stable/10 for now.

Comment 7 commit-hook freebsd_committer

2016-05-08 18:10:23 UTC

A commit references this bug:

Author: delphij
Date: Sun May  8 18:09:32 UTC 2016
New revision: 299236
URL: https://svnweb.freebsd.org/changeset/base/299236

Log:
  Revert r298920 until we have addressed all known regressions on head/ after
  settle.  file is now downgraded to 5.25.

  PR:		bin/209211
  Requested by:	Tomoaki AOKI
  Relnotes:	yes

Changes:
_U  stable/10/
  stable/10/contrib/file/ChangeLog
  stable/10/contrib/file/README
  stable/10/contrib/file/config.h.in
  stable/10/contrib/file/configure
  stable/10/contrib/file/configure.ac
  stable/10/contrib/file/doc/file.man
  stable/10/contrib/file/doc/libmagic.man
  stable/10/contrib/file/magic/Magdir/android
  stable/10/contrib/file/magic/Magdir/animation
  stable/10/contrib/file/magic/Magdir/apple
  stable/10/contrib/file/magic/Magdir/archive
  stable/10/contrib/file/magic/Magdir/audio
  stable/10/contrib/file/magic/Magdir/bioinformatics
  stable/10/contrib/file/magic/Magdir/c-lang
  stable/10/contrib/file/magic/Magdir/cafebabe
  stable/10/contrib/file/magic/Magdir/coff
  stable/10/contrib/file/magic/Magdir/commands
  stable/10/contrib/file/magic/Magdir/compress
  stable/10/contrib/file/magic/Magdir/console
  stable/10/contrib/file/magic/Magdir/database
  stable/10/contrib/file/magic/Magdir/der
  stable/10/contrib/file/magic/Magdir/filesystems
  stable/10/contrib/file/magic/Magdir/finger
  stable/10/contrib/file/magic/Magdir/flif
  stable/10/contrib/file/magic/Magdir/fonts
  stable/10/contrib/file/magic/Magdir/fortran
  stable/10/contrib/file/magic/Magdir/hitachi-sh
  stable/10/contrib/file/magic/Magdir/images
  stable/10/contrib/file/magic/Magdir/intel
  stable/10/contrib/file/magic/Magdir/java
  stable/10/contrib/file/magic/Magdir/lisp
  stable/10/contrib/file/magic/Magdir/mach
  stable/10/contrib/file/magic/Magdir/macintosh
  stable/10/contrib/file/magic/Magdir/microfocus
  stable/10/contrib/file/magic/Magdir/misctools
  stable/10/contrib/file/magic/Magdir/modem
  stable/10/contrib/file/magic/Magdir/msdos
  stable/10/contrib/file/magic/Magdir/msvc
  stable/10/contrib/file/magic/Magdir/msx
  stable/10/contrib/file/magic/Magdir/netbsd
  stable/10/contrib/file/magic/Magdir/polyml
  stable/10/contrib/file/magic/Magdir/psdbms
  stable/10/contrib/file/magic/Magdir/python
  stable/10/contrib/file/magic/Magdir/sendmail
  stable/10/contrib/file/magic/Magdir/sgml
  stable/10/contrib/file/magic/Magdir/sinclair
  stable/10/contrib/file/magic/Magdir/sql
  stable/10/contrib/file/magic/Magdir/terminfo
  stable/10/contrib/file/magic/Magdir/vacuum-cleaner
  stable/10/contrib/file/magic/Magdir/windows
  stable/10/contrib/file/magic/Magdir/wordprocessors
  stable/10/contrib/file/magic/Makefile.am
  stable/10/contrib/file/magic/Makefile.in
  stable/10/contrib/file/python/README
  stable/10/contrib/file/python/magic.py
  stable/10/contrib/file/python/setup.py
  stable/10/contrib/file/src/Makefile.am
  stable/10/contrib/file/src/Makefile.in
  stable/10/contrib/file/src/apprentice.c
  stable/10/contrib/file/src/ascmagic.c
  stable/10/contrib/file/src/compress.c
  stable/10/contrib/file/src/der.c
  stable/10/contrib/file/src/der.h
  stable/10/contrib/file/src/dprintf.c
  stable/10/contrib/file/src/file.c
  stable/10/contrib/file/src/file.h
  stable/10/contrib/file/src/file_opts.h
  stable/10/contrib/file/src/fmtcheck.c
  stable/10/contrib/file/src/funcs.c
  stable/10/contrib/file/src/magic.c
  stable/10/contrib/file/src/magic.h
  stable/10/contrib/file/src/print.c
  stable/10/contrib/file/src/readcdf.c
  stable/10/contrib/file/src/readelf.c
  stable/10/contrib/file/src/readelf.h
  stable/10/contrib/file/src/softmagic.c
  stable/10/lib/libmagic/Makefile
  stable/10/lib/libmagic/config.h

Comment 8 commit-hook freebsd_committer

2016-05-08 18:19:25 UTC

A commit references this bug:

Author: delphij
Date: Sun May  8 18:18:35 UTC 2016
New revision: 299238
URL: https://svnweb.freebsd.org/changeset/base/299238

Log:
  MFV r299237: apply two upstream revisions to address upstream bug PR/540.

  PR:		209211
  MFC after:	2 weeks
  X-MFC:		if there is no new file release

Changes:
_U  head/contrib/file/
  head/contrib/file/src/softmagic.c

Comment 9 Xin LI freebsd_committer

2016-05-08 18:19:35 UTC

I've fixed the other one (PR/540) in head/.  Will MFC if no complains in two weeks and if there is no new release of file.

Comment 10 Tomoaki AOKI 2016-05-09 12:53:25 UTC

(In reply to Xin LI from comment #9)
Confirmed fixed by manually applying to stable/10, r299232.
Also confirmed file and libmagic are reverted for stable/10 at r299236.
Thanks!

Comment 11 Peter Thoenen 2016-05-09 21:21:03 UTC

For us non-dev people can somebody post the quick and dirty "how to apply this" without rebuilding all of world, i.e. is there just a subsection I can rebuild after svn'ing something newer than r299232

Comment 12 Tomoaki AOKI 2016-05-09 22:01:47 UTC

(In reply to Peter Thoenen from comment #11)

If you have FreeBSD source tree at default /usr/src and setting no options
for buildworld,

  cd /usr/src/usr.bin/file
  make clean ; make ; make install
  cd /usr/src/lib/libmagic
  make clean ; make ; make install

should be sufficient for this problem.

Comment 13 Mark Millard 2016-06-13 08:00:40 UTC

(In reply to Xin LI from comment #9)

As of 11.0 -r310815 and /usr/ports/ -r46711 after building portupgrade my use of portinstall did not get any broken pipe notices.

[For some reason lua52 was not built when liblua-5.2.so was not found during a kyua build via portinstall -W --and portmaster -DK then had no such trouble. But I have no evidence that this would be a usefully related error.]

As far as I can tell the original problem reported is fixed for 11.0 -r301815.

I do not have a 10.x context to check.

Comment 14 Xin LI freebsd_committer

2016-06-19 07:27:14 UTC

Fixed by MFC'ing file.

Comment 15 Bryan Drewery freebsd_committer

2018-01-24 19:41:28 UTC

*** Bug 209237 has been marked as a duplicate of this bug. ***