Bug 228606 - makeroot.sh w/ -f fails with duplicate definition errors
Summary: makeroot.sh w/ -f fails with duplicate definition errors
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 11.1-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: Ed Maste
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-05-30 13:42 UTC by Ed Maste
Modified: 2018-08-12 14:15 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer 2018-05-30 13:42:57 UTC
I have the result of a -DNO_ROOT installworld and installkernel at /usr/obj/freebsd-root, and execute makeroot:

sh tools/tools/makeroot/makeroot.sh -s 32m -f filelist output.img /usr/obj/freebsd-root/

With a filelist of
```
boot/kernel/kernel
etc/pam.d/README
```
makeroot.sh fails with
```
/tmp/makeroot.KQ8In/manifest:3: warning: tags: unsupported keyword
/tmp/makeroot.KQ8In/manifest:4: error: duplicate definition of boot
/tmp/makeroot.KQ8In/manifest:8: warning: tags: unsupported keyword
/tmp/makeroot.KQ8In/manifest:9: error: duplicate definition of pam.d
/tmp/makeroot.KQ8In/manifest:10: error: duplicate definition of pam.d
/tmp/makeroot.KQ8In/manifest:11: error: duplicate definition of pam.d
/tmp/makeroot.KQ8In/manifest:12: error: duplicate definition of pam.d
/tmp/makeroot.KQ8In/manifest:13: warning: tags: unsupported keyword
makefs: 5 error(s) and 3 warning(s) in mtree manifest
```

The duplicates occur because some directory entries are emitted for multiple packages:
```
% grep 'pam.d type=dir' /usr/obj/freebsd-root/METALOG
./etc/pam.d type=dir uname=root gname=wheel mode=0755
./etc/pam.d type=dir mode=0755 tags=package=runtime
./etc/pam.d type=dir mode=0755 tags=package=at
./etc/pam.d type=dir mode=0755 tags=package=ftp
./etc/pam.d type=dir mode=0755 tags=package=telnet
```

With a filelist of 
```
/boot/kernel/kernel
/etc/pam.d/README
```
or
```
./boot/kernel/kernel
./etc/pam.d/README
```
the script succeeds but the created filesystem is empty.
Comment 1 commit-hook freebsd_committer 2018-05-30 13:51:48 UTC
A commit references this bug:

Author: emaste
Date: Wed May 30 13:51:00 UTC 2018
New revision: 334381
URL: https://svnweb.freebsd.org/changeset/base/334381

Log:
  makeroot.sh: allow duplicate entries even with -f <filelist>

  makefs disallows duplicate entries unless the -D option is specified.

  Previously makeroot.sh enabled -D unless a filelist was provided via
  the -f options.  The filelist logic creates an mtree manifest from the
  METALOG and the provided filelist by passing them through `sort -u`,
  so duplicates were not expected.  However, duplicates can still occur
  when a directory appears in multiple packages -- for example,

  ./etc/pam.d type=dir uname=root gname=wheel mode=0755
  ./etc/pam.d type=dir mode=0755 tags=package=runtime
  ./etc/pam.d type=dir mode=0755 tags=package=at
  ./etc/pam.d type=dir mode=0755 tags=package=ftp
  ./etc/pam.d type=dir mode=0755 tags=package=telnet

  For the purposes of makefs these directory entries are identical, but
  are of course not identical for sort -u.

  For now just leave the allow duplicates -D flag enabled.

  PR:		228606
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/tools/tools/makeroot/makeroot.sh
Comment 2 Ed Maste freebsd_committer 2018-05-30 13:52:23 UTC
Workaround committed, but a better fix is needed in makefs and/or makeroot.sh.