Bug 219673 - usr.bin/mkimg/@r319125 broke the mkimg gpt-specific tests
Summary: usr.bin/mkimg/@r319125 broke the mkimg gpt-specific tests
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Many People
Assignee: Enji Cooper
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-31 07:31 UTC by Enji Cooper
Modified: 2018-02-09 09:15 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer freebsd_triage 2017-05-31 07:31:18 UTC
^/head@r319125 broke the GPT mkimg(1) tests, as evidenced here (for example):
https://ci.freebsd.org/job/FreeBSD-head-amd64-test/3280/testReport/junit/usr.bin.mkimg/mkimg_test/gpt_1x1_4096_qcow/ .

The items need to be marked as expected failures until the tests are fixed, or the code is fixed.
Comment 1 commit-hook freebsd_committer freebsd_triage 2017-05-31 08:02:19 UTC
A commit references this bug:

Author: ngie
Date: Wed May 31 08:01:13 UTC 2017
New revision: 319295
URL: https://svnweb.freebsd.org/changeset/base/319295

Log:
  Update the usr.bin/mkimg golden test output files after ^/head@r319125

  ^/head@r319125 changed the location of the backup pmbr, requiring the
  output files to be regenerated, since they're binary disk dumps.

  The output files were regenerated with "make rebase"--fixed in
  ^/head@r319294.

  MFC with:	r319125, r319294
  PR:		219673
  Sponsored by:	Dell EMC Isilon

Changes:
  head/usr.bin/mkimg/tests/img-1x1-4096-gpt.qcow.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-4096-gpt.qcow2.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-4096-gpt.raw.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-4096-gpt.vhd.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-4096-gpt.vhdf.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-4096-gpt.vmdk.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-512-gpt.qcow.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-512-gpt.qcow2.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-512-gpt.raw.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-512-gpt.vhd.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-512-gpt.vhdf.gz.uu
  head/usr.bin/mkimg/tests/img-1x1-512-gpt.vmdk.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-4096-gpt.qcow.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-4096-gpt.qcow2.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-4096-gpt.raw.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-4096-gpt.vhd.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-4096-gpt.vhdf.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-4096-gpt.vmdk.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-512-gpt.qcow.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-512-gpt.qcow2.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-512-gpt.raw.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-512-gpt.vhd.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-512-gpt.vhdf.gz.uu
  head/usr.bin/mkimg/tests/img-63x255-512-gpt.vmdk.gz.uu
Comment 2 commit-hook freebsd_committer freebsd_triage 2018-02-07 15:33:07 UTC
A commit references this bug:

Author: manu
Date: Wed Feb  7 15:32:33 UTC 2018
New revision: 328974
URL: https://svnweb.freebsd.org/changeset/base/328974

Log:
  MFC r306325, r306329-r306330, r306333, r306620-r306622, r307387, r307544, r307550, r318137, r319125, r319295

  r306325 by marcel:
  Replace the use of linker sets with constructors for both the
  formats and schemes.  Formats and schemes are registered at
  runtime now, rather than collected at link time.

  r306329 by marcel:
  Eliminate the use of EDOOFUS.  The error code was used to signal
  programming errors, but is really a poor substitute for assert.
  And less portable as well.

  r306330 by marcel:
  Avoid depending on the <sys/endian.h> header for le*enc and be*enc.
  Not only is the header unportable, the encoding/decoding functions
  are as well.  Instead, duplicate the handful of small inlines we
  need into a private header called endian.h.

  Aside: an alternative approach is to move the encoding/decoding
  functions to a separate system header.  While the header is still
  nonportable, such an approach would make it possible to re-use the
  definitions by playing games with include paths. This may be the
  preferred approach if more (build) utilities need this.  This
  change does not preclude that.  In fact, it makes it easier.

  r306333 by marcel:
  Portability changes:
  1.  macOS nor Linux have MAP_NOCORE nor MAP_NOSYNC. Define as 0.
  2.  macOS doesn't have SEEK_DATA nor SEEK_HOLE. Define as -1
      so that lseek will return -1 (with errno set to EINVAL).
  3.  gcc correctly warns that error is assigned but not used in
      image_copyout_region().  Fix by returning on the first error.

  r306620 by marcel:
  Replace STAILQ with TAILQ. TAILQs are portable enough that they can
  be used on both macOS and Linux. STAILQs are not. In particular,
  STAILQ_LAST does not next on Linux. Since neither STAILQ_FOREACH_SAFE
  nor TAILQ_FOREACH_SAFE exist on Linux, replace its use with a regular
  TAILQ_FOREACH. The _SAFE variant was only used for having the next
  pointer in a local variable.

  r306621 by marcel:
  Prefer <stdint.h> over <sys/types.h>. While here remove redundant
  inclusion of <sys/queue.h>.

  Move the inclusion of the disk partitioning headers out of order
  and inbetween standard headers and local header. They will change
  in a subsequent commit.

  r306622 by marcel:
  Replace OFF_MAX with INT64_MAX. The former is defined on Linux.

  r307387 by marcel:
  Switch to using the portable partition scheme headers.

  r307544 by marcel:
   o  Provide a private definition for UUIDs (mkimg_uuid_t) because
      UUIDs are not portable.
   o  Move mkimg_uuid() to a new file and merge both gpt_uuid_enc()
      and vhd_uuid_enc() into a single mkimg_uuid_enc() that lives
      in the same file.
   o  Move the OS-specific implementation of generating a UUID to
      osdep_uuidgen() and provide the implementations for FreeBSD,
      macOS and Linux.
   o  Expect the partitioning scheme headers to be found by having
      a search to the directory in which the headers live. This
      avoids conflicts on non-FreeBSD machines.

  r307550 by imp:
  Add a new flag to mkimg (-a num) to specify the active partition for
  those partitioning schemes that have this concept. Implement it as an
  override for mbr's setting 0x80 in the flags for the first partition
  when we have boot code.

  Differential Revision: https://reviews.freebsd.org/D4403

  r318137:
  mkimg: Add -C argument to specify maximum capacity

  Add a -C option to specify a maximum capacity for the final image file.
  It is useful to control the size of the generated image for sdcard or
  when we will add dynamic size partition.

  Add --capacity which is a shorthand to define min and max capacity at
  the same time.

  Reviewed by:	bapt, marcel, wblock (manpages)
  Sponsored by:	Gandi.net
  Differential Revision:	https://reviews.freebsd.org/D10509

  r319125:
  mkimg: Correct an off by one error in the PMBR size

  The PMBR last sector should be number of sector - 1 (As stated in UEFI Spec
  2.6 page 118 table 17).
  This fixes warning printed by linux tools like parted or fdisk.

  Sponsored by:	Gandi.net

  r319295 by ngie:
  Update the usr.bin/mkimg golden test output files after ^/head@r319125

  ^/head@r319125 changed the location of the backup pmbr, requiring the
  output files to be regenerated, since they're binary disk dumps.

  The output files were regenerated with "make rebase"--fixed in
  ^/head@r319294.

  MFC with:	r319125, r319294
  PR:		219673
  Sponsored by:	Dell EMC Isilon

Changes:
_U  stable/11/
  stable/11/usr.bin/mkimg/Makefile
  stable/11/usr.bin/mkimg/apm.c
  stable/11/usr.bin/mkimg/bsd.c
  stable/11/usr.bin/mkimg/ebr.c
  stable/11/usr.bin/mkimg/endian.h
  stable/11/usr.bin/mkimg/format.c
  stable/11/usr.bin/mkimg/format.h
  stable/11/usr.bin/mkimg/gpt.c
  stable/11/usr.bin/mkimg/image.c
  stable/11/usr.bin/mkimg/mbr.c
  stable/11/usr.bin/mkimg/mkimg.1
  stable/11/usr.bin/mkimg/mkimg.c
  stable/11/usr.bin/mkimg/mkimg.h
  stable/11/usr.bin/mkimg/pc98.c
  stable/11/usr.bin/mkimg/qcow.c
  stable/11/usr.bin/mkimg/raw.c
  stable/11/usr.bin/mkimg/scheme.c
  stable/11/usr.bin/mkimg/scheme.h
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/uuid.c
  stable/11/usr.bin/mkimg/vhd.c
  stable/11/usr.bin/mkimg/vmdk.c
  stable/11/usr.bin/mkimg/vtoc8.c
Comment 3 commit-hook freebsd_committer freebsd_triage 2018-02-09 09:15:56 UTC
A commit references this bug:

Author: manu
Date: Fri Feb  9 09:15:44 UTC 2018
New revision: 329059
URL: https://svnweb.freebsd.org/changeset/base/329059

Log:
  MFC r306325, r306329-r306330, r306333, r306620-r306622, r307544, r307550, r318137, r319125, r319295

  r306325 by marcel:
  Replace the use of linker sets with constructors for both the
  formats and schemes.  Formats and schemes are registered at
  runtime now, rather than collected at link time.

  r306329 by marcel:
  Eliminate the use of EDOOFUS.  The error code was used to signal
  programming errors, but is really a poor substitute for assert.
  And less portable as well.

  r306330 by marcel:
  Avoid depending on the <sys/endian.h> header for le*enc and be*enc.
  Not only is the header unportable, the encoding/decoding functions
  are as well.  Instead, duplicate the handful of small inlines we
  need into a private header called endian.h.

  Aside: an alternative approach is to move the encoding/decoding
  functions to a separate system header.  While the header is still
  nonportable, such an approach would make it possible to re-use the
  definitions by playing games with include paths. This may be the
  preferred approach if more (build) utilities need this.  This
  change does not preclude that.  In fact, it makes it easier.

  r306333 by marcel:
  Portability changes:
  1.  macOS nor Linux have MAP_NOCORE nor MAP_NOSYNC. Define as 0.
  2.  macOS doesn't have SEEK_DATA nor SEEK_HOLE. Define as -1
      so that lseek will return -1 (with errno set to EINVAL).
  3.  gcc correctly warns that error is assigned but not used in
      image_copyout_region().  Fix by returning on the first error.

  r306620 by marcel:
  Replace STAILQ with TAILQ. TAILQs are portable enough that they can
  be used on both macOS and Linux. STAILQs are not. In particular,
  STAILQ_LAST does not next on Linux. Since neither STAILQ_FOREACH_SAFE
  nor TAILQ_FOREACH_SAFE exist on Linux, replace its use with a regular
  TAILQ_FOREACH. The _SAFE variant was only used for having the next
  pointer in a local variable.

  r306621 by marcel:
  Prefer <stdint.h> over <sys/types.h>. While here remove redundant
  inclusion of <sys/queue.h>.

  Move the inclusion of the disk partitioning headers out of order
  and inbetween standard headers and local header. They will change
  in a subsequent commit.

  r306622 by marcel:
  Replace OFF_MAX with INT64_MAX. The former is defined on Linux.

  r307544 by marcel:
   o  Provide a private definition for UUIDs (mkimg_uuid_t) because
      UUIDs are not portable.
   o  Move mkimg_uuid() to a new file and merge both gpt_uuid_enc()
      and vhd_uuid_enc() into a single mkimg_uuid_enc() that lives
      in the same file.
   o  Move the OS-specific implementation of generating a UUID to
      osdep_uuidgen() and provide the implementations for FreeBSD,
      macOS and Linux.
   o  Expect the partitioning scheme headers to be found by having
      a search to the directory in which the headers live. This
      avoids conflicts on non-FreeBSD machines.

  r307550 by imp:
  Add a new flag to mkimg (-a num) to specify the active partition for
  those partitioning schemes that have this concept. Implement it as an
  override for mbr's setting 0x80 in the flags for the first partition
  when we have boot code.

  Differential Revision: https://reviews.freebsd.org/D4403

  r318137:
  mkimg: Add -C argument to specify maximum capacity

  Add a -C option to specify a maximum capacity for the final image file.
  It is useful to control the size of the generated image for sdcard or
  when we will add dynamic size partition.

  Add --capacity which is a shorthand to define min and max capacity at
  the same time.

  Reviewed by:	bapt, marcel, wblock (manpages)
  Sponsored by:	Gandi.net
  Differential Revision:	https://reviews.freebsd.org/D10509

  r319125:
  mkimg: Correct an off by one error in the PMBR size

  The PMBR last sector should be number of sector - 1 (As stated in UEFI Spec
  2.6 page 118 table 17).
  This fixes warning printed by linux tools like parted or fdisk.

  Sponsored by:	Gandi.net

  r319295 by ngie:
  Update the usr.bin/mkimg golden test output files after ^/head@r319125

  ^/head@r319125 changed the location of the backup pmbr, requiring the
  output files to be regenerated, since they're binary disk dumps.

  The output files were regenerated with "make rebase"--fixed in
  ^/head@r319294.

  MFC with:	r319125, r319294
  PR:		219673
  Sponsored by:	Dell EMC Isilon

Changes:
_U  stable/11/
  stable/11/usr.bin/mkimg/Makefile
  stable/11/usr.bin/mkimg/apm.c
  stable/11/usr.bin/mkimg/bsd.c
  stable/11/usr.bin/mkimg/ebr.c
  stable/11/usr.bin/mkimg/endian.h
  stable/11/usr.bin/mkimg/format.c
  stable/11/usr.bin/mkimg/format.h
  stable/11/usr.bin/mkimg/gpt.c
  stable/11/usr.bin/mkimg/image.c
  stable/11/usr.bin/mkimg/mbr.c
  stable/11/usr.bin/mkimg/mkimg.1
  stable/11/usr.bin/mkimg/mkimg.c
  stable/11/usr.bin/mkimg/mkimg.h
  stable/11/usr.bin/mkimg/pc98.c
  stable/11/usr.bin/mkimg/qcow.c
  stable/11/usr.bin/mkimg/raw.c
  stable/11/usr.bin/mkimg/scheme.c
  stable/11/usr.bin/mkimg/scheme.h
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-4096-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-1x1-512-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-4096-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.qcow.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.qcow2.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.raw.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.vhd.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.vhdf.gz.uu
  stable/11/usr.bin/mkimg/tests/img-63x255-512-gpt.vmdk.gz.uu
  stable/11/usr.bin/mkimg/uuid.c
  stable/11/usr.bin/mkimg/vhd.c
  stable/11/usr.bin/mkimg/vmdk.c
  stable/11/usr.bin/mkimg/vtoc8.c