Bug 224871 - `make fetchindex' may leave a corrupt INDEX file
Summary: `make fetchindex' may leave a corrupt INDEX file
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL: https://reviews.freebsd.org/D14088
Depends on: 224893
  Show dependency treegraph
Reported: 2018-01-03 17:15 UTC by Wolfram Schneider
Modified: 2021-09-01 08:50 UTC (History)
4 users (show)

See Also:

/usr/ports/Makefile patch (613 bytes, patch)
2018-01-03 17:23 UTC, Wolfram Schneider
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Wolfram Schneider freebsd_committer 2018-01-03 17:15:09 UTC
In /usr/ports/Makefile is a target fetchindex to fetch the latest INDEX file for the given OS.

$ cd /usr/ports; make fetchindex

In case of an error the target will fail with an error status, but leaves the ./INDEX file in an undefined corrupt state. An error can occurs if the disk is full, or the kernel kills the bunzip2 command (e.g. out of swap).


$ make fetchindex
bunzip2: I/O or other error, bailing out.  Possible reason follows.
bunzip2: No space left on device
	Input file = (stdin), output file = (stdout)
*** Error code 1

make: stopped in /home/projects/freebsd-ports

$ du -hs INDEX-12
 32M	INDEX-12

The expected size is 36MB.

The problem is in the line
    bunzip2 < ${INDEXDIR}/${INDEXFILE}.bz2 > ${INDEXDIR}/${INDEXFILE}
Comment 1 Wolfram Schneider freebsd_committer 2018-01-03 17:19:12 UTC
Another issue is: the fetch command runs in mirror mode (-am), but the INDEX.bz2 file will be deleted after extracted. That does not make sense.

make -n fetchindex
/usr/bin/env  fetch -am -o /home/projects/freebsd-ports/INDEX-12.bz2 http://www.FreeBSD.org/ports/INDEX-12.bz2
bunzip2 < /home/projects/freebsd-ports/INDEX-12.bz2 > /home/projects/freebsd-ports/INDEX-12 &&  chmod a+r /home/projects/freebsd-ports/INDEX-12 && /bin/rm -f /home/projects/freebsd-ports/INDEX-12.bz2

To run fetch(1) in mirror mode, we need to keep the bzip2'd INDEX file.
Comment 2 Wolfram Schneider freebsd_committer 2018-01-03 17:23:37 UTC
Created attachment 189374 [details]
/usr/ports/Makefile patch
Comment 3 Rene Ladan freebsd_committer 2019-04-08 19:23:53 UTC
Reassign to portmgr