Bug 228812

Description Yasuhiro Kimura 2018-06-07 18:56:14 UTC

Created attachment 194071 [details]
patch file

I'm investigating breakage of this port. And below is report of current status.

1. This port is not broken with real environment (i.e. not poudriere).

Conditions:
* OS is newly installed 11.1-RELEASE amd64
* Port tree is ports r471895.
* /var/db/ports is empty. So options settings of any ports aren't  customized.
* Settings in /etc/make.conf are as following.
----------------------------------------------------------------------
DEVELOPER=yes

DISTDIR=/usr0/freebsd/ports/distfiles
PACKAGES=/usr0/freebsd/ports/packages
WRKDIRPREFIX=/usr0/freebsd/ports/work

MASTER_SITE_GNOME=http://ftp.nara.wide.ad.jp/pub/X11/GNOME/%SUBDIR%/
MASTER_SITE_ISC=http://ftp.iij.ad.jp/pub/network/isc/%SUBDIR%/
.for mirror in osdn iij jaist ymu
MASTER_SITE_OSDN+= http://${mirror}.dl.osdn.jp/%SUBDIR%/
.endfor
MASTER_SITE_PERL_CPAN_BY=http://ftp.jaist.ac.jp/pub/CPAN/%CPANSORT%/%SUBDIR%/
MASTER_SITE_SUDO=http://core.ring.gr.jp/archives/misc/sudo/ http://www.ring.gr.jp/archives/misc/sudo/

FTP_PROXY='http://proxy.home.utahime.org:8080/'
HTTP_PROXY='http://proxy.home.utahime.org:8080/'
----------------------------------------------------------------------

At first I modified /usr/ports/editors/tamago/Makefile as following.

root@rolling-vm-freebsd2[83]# svnlite diff /usr/ports/editors/tamago/Makefile                                              ~
Index: /usr/ports/editors/tamago/Makefile
===================================================================                                                         
--- /usr/ports/editors/tamago/Makefile  (revision 471895)
+++ /usr/ports/editors/tamago/Makefile  (working copy)
@@ -13,13 +13,7 @@

 LICENSE=       GPLv2

-.if ${FLAVOR:U} != devel && ${FLAVOR:U} != devel_nox
-# http://pkg.awarnach.mathstat.dal.ca/data/11i386-default/2018-05-30_07h36m11s/logs/errors/tamago-emacs26-4.0.6.0.20041122.19.14_4.log
-BROKEN=                fails to build
-.endif
-
 USES=          emacs
-EMACS_FLAVORS_EXCLUDE= devel_full # fails to build
 USE_GITHUB=    yes
 GH_ACCOUNT=    hrs-allbsd

root@rolling-vm-freebsd2[84]#

Then I executed following commands.

# make -C /usr/ports/editors/tamago BATCH=yes FLAVOR=full install
# pkg delete emacs-26.1,3
# make -C /usr/ports/editors/tamago BATCH=yes FLAVOR=canna install
# pkg delete emacs-canna-26.1,3
# make -C /usr/ports/editors/tamago BATCH=yes FLAVOR=nox install
# pkg delete emacs-nox-26.1,3
# make -C /usr/ports/editors/tamago BATCH=yes FLAVOR=devel_full install
# pkg delete emacs-devel-27.0.50.20180529,2
# make -C /usr/ports/editors/tamago BATCH=yes FLAVOR=devel_nox install

And builds of all flavors succeeded.

2. With poudriere some flavors fail to build, and others not.

Conditions:
* Host OS is the one used in section 1.
* ports-mgmt/poudriere is used ad poudriere. Version is 3.2.7.
* Settings in /usr/local/etc/poudriere.conf and
/usr/local/etc/poudriere.d/make.conf are as following.
----------------------------------------------------------------------
# Poudriere can optionally use ZFS for its ports/jail storage. For
# ZFS define ZPOOL, otherwise set NO_ZFS=yes
# 
#### ZFS
# The pool where poudriere will create all the filesystems it needs
# poudriere will use tank/${ZROOTFS} as its root
#
# You need at least 7GB of free space in this pool to have a working
# poudriere.
#
ZPOOL=zroot

### NO ZFS
# To not use ZFS, define NO_ZFS=yes
#NO_ZFS=yes

# root of the poudriere zfs filesystem, by default /poudriere
# ZROOTFS=/poudriere

# the host where to download sets for the jails setup
# You can specify here a host or an IP
# replace _PROTO_ by http or ftp
# replace _CHANGE_THIS_ by the hostname of the mirrors where you want to fetch
# by default: ftp://ftp.freebsd.org
#
# Also note that every protocols supported by fetch(1) are supported here, even
# file:///
# Suggested: https://download.FreeBSD.org
FREEBSD_HOST=https://download.FreeBSD.org

# By default the jails have no /etc/resolv.conf, you will need to set
# RESOLV_CONF to a file on your hosts system that will be copied has
# /etc/resolv.conf for the jail, except if you don't need it (using an http
# proxy for example)
RESOLV_CONF=/etc/resolv.conf

# The directory where poudriere will store jails and ports
BASEFS=/usr/local/poudriere

# The directory where the jail will store the packages and logs
# by default a zfs filesystem will be created and set to
# ${BASEFS}/data
#
#POUDRIERE_DATA=${BASEFS}/data

# Use portlint to check ports sanity
USE_PORTLINT=no

# When building packages, a memory device can be used to speedup the build.
# Only one of MFSSIZE or USE_TMPFS is supported. TMPFS is generally faster
# and will expand to the needed amount of RAM. MFS is a slower since it
# uses UFS and several abstraction layers.

# If set WRKDIRPREFIX will be mdmfs of the given size (mM or gG)
#MFSSIZE=4G

# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
# data      - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all       - Run the entire build in memory, including builder jails.
# yes       - Enables tmpfs(5) for wrkdir and data
# no        - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS=yes

# How much memory to limit tmpfs size to for *each builder* in GiB
# (default: none)
#TMPFS_LIMIT=8

# How much memory to limit jail processes to for *each builder*
# in GiB (default: none)
#MAX_MEMORY=8

# How many file descriptors to limit each jail process to (default: 1024)
#MAX_FILES=1024

# If set the given directory will be used for the distfiles
# This allows to share the distfiles between jails and ports tree
# If this is "no", poudriere must be supplied a ports tree that already has
# the required distfiles.
DISTFILES_CACHE=/usr0/freebsd/ports/distfiles

# If set the ports tree or source tree marked to use svn will use the defined
# mirror (default: svn.FreeBSD.org)
# The SSL fingerprints are published here:
# https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/svn.html#svn-mirrors
#SVN_HOST=svn.FreeBSD.org

# Automatic OPTION change detection
# When bulk building packages, compare the options from kept packages to
# the current options to be built. If they differ, the existing package
# will be deleted and the port will be rebuilt.
# Valid options: yes, no, verbose
# verbose will display the old and new options
#CHECK_CHANGED_OPTIONS=verbose

# Automatic Dependency change detection
# When bulk building packages, compare the dependencies from kept packages to
# the current dependencies for every port. If they differ, the existing package
# will be deleted and the port will be rebuilt. This helps catch changes such
# as DEFAULT_RUBY_VERSION, PERL_VERSION, WITHOUT_X11 that change dependencies
# for many ports.
# Valid options: yes, no
# Default: yes
#CHECK_CHANGED_DEPS=yes

# Consider bad dependency lines on the wrong PKGNAME as fatal.
# For example:
#    BUILD_DEPENDS=  p5-List-MoreUtils>=0:lang/p5-List-MoreUtils
# If this port's PKGNAME were really "List-MoreUtils" then it would
# not be recorded into the resulting package.  The next build with
# CHECK_CHANGED_DEPS enabled would consider it a "new dependency"
# since it is in the port but not in the package.  This is usually
# a warning but can be made fatal instead by enabling this option.
# Default: no
#BAD_PKGNAME_DEPS_ARE_FATAL=yes


# Path to the RSA key to sign the PKG repo with. See pkg-repo(8)
#PKG_REPO_SIGNING_KEY=/etc/ssl/keys/repo.key


# ccache support. Supply the path to your ccache cache directory.
# It will be mounted into the jail and be shared among all jails.
# It is recommended that extra ccache configuration be done with
# ccache -o rather than from the environment.
#CCACHE_DIR=/var/cache/ccache

# Static ccache support from host.  This uses the existing
# ccache from the host in the build jail.  This is useful for
# using ccache+memcached which cannot easily be bootstrapped
# otherwise.  The path to the PREFIX where ccache was installed
# must be used here, and ccache must have been built statically.
# Note also that ccache+memcached will require network access
# which is normally disabled.  Separately setting RESTRICT_NETWORKING=no
# may be required for non-localhost memcached servers.
#CCACHE_STATIC_PREFIX=/usr/local

# The jails normally only allow network access during the 'make fetch'
# phase.  This is a security restriction to prevent random things
# ran during a build from accessing the network.  Disabling this
# is not advised.  ALLOW_NETWORKING_PACKAGES may be used to allow networking
# for a subset of packages only.
#RESTRICT_NETWORKING=yes
#ALLOW_NETWORKING_PACKAGES="npm-foo"

# parallel build support.
#
# By default poudriere uses hw.ncpu to determine the number of builders.
# You can override this default by changing PARALLEL_JOBS here, or
# by specifying the -J flag to bulk/testport.
#
# Example to define PARALLEL_JOBS to one single job
# PARALLEL_JOBS=1

# How many jobs should be used for preparing the build? These tend to
# be more IO bound and may be worth tweaking. Default: PARALLEL_JOBS * 1.25
# PREPARE_PARALLEL_JOBS=1


# If set, failed builds will save the WRKDIR to ${POUDRIERE_DATA}/wrkdirs
# SAVE_WRKDIR=yes

# Choose the default format for the workdir packing: could be tar,tgz,tbz,txz
# default is tbz
WRKDIR_ARCHIVE_FORMAT=txz

# Disable linux support
# NOLINUX=yes

# By default poudriere sets FORCE_PACKAGE
# To disable it (useful when building public packages):
# NO_FORCE_PACKAGE=yes

# By default poudriere sets PACKAGE_BUILDING
# To disable it:
# NO_PACKAGE_BUILDING=yes

# If you are using a proxy define it here:
export HTTP_PROXY=http://proxy.home.utahime.org:8080/
export FTP_PROXY=http://proxy.home.utahime.org:8080/
#
# Cleanout the restricted packages
# NO_RESTRICTED=yes

# By default MAKE_JOBS is disabled to allow only one process per cpu
# Use the following to allow it anyway
# ALLOW_MAKE_JOBS=yes

# List of packages that will always be allowed to use MAKE_JOBS
# regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports
# which holdup the rest of the queue to build more quickly.
#ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py*"

# Timestamp every line of build logs
# Default: no
#TIMESTAMP_LOGS=no

# URL where your POUDRIERE_DATA/logs are hosted
# This will be used for giving URL hints to the HTML output when
# scheduling and starting builds
#URL_BASE=http://yourdomain.com/poudriere/


# This defines the max time (in seconds) that a command may run for a build
# before it is killed for taking too long. Default: 86400
#MAX_EXECUTION_TIME=86400

# This defines the time (in seconds) before a command is considered to
# be in a runaway state for having no output on stdout. Default: 7200
#NOHANG_TIME=7200


# The repository is updated atomically if set yes. This leaves the
# repository untouched until the build completes. This involves using
# hardlinks and symlinks. The operations are fast, but can be intrusive
# for remote syncing or backups.
# Recommended to always keep on.
# Default: yes
#ATOMIC_PACKAGE_REPOSITORY=yes

# When using ATOMIC_PACKAGE_REPOSITORY, commit the packages if some
# packages fail to build. Ignored ports are considered successful.
# This can be set to 'no' to only commit the packages once no failures
# are encountered.
# Default: yes
#COMMIT_PACKAGES_ON_FAILURE=yes

# Keep older package repositories. This can be used to rollback a system
# or to bisect issues by changing the repository to one of the older
# versions and reinstalling everything with `pkg upgrade -f`
# ATOMIC_PACKAGE_REPOSITORY is required for this.
# Default: no
#KEEP_OLD_PACKAGES=no

# How many old package repositories to keep with KEEP_OLD_PACKAGES
# Default: 5
#KEEP_OLD_PACKAGES_COUNT=5

# Make testing errors fatal.
# If set to 'no', ports with test failure will be marked as failed but still
# packaged to permit testing dependent ports (useful for bulk -t -a)
# Default: yes
#PORTTESTING_FATAL=yes

# Define the building jail hostname to be used when building the packages
# Some port/packages hardcode the hostname of the host during build time
# This is a necessary setup for reproducible builds.
#BUILDER_HOSTNAME=pkg.FreeBSD.org

# Define to get a predictable timestamp on the ports tree
# This is a necessary setup for reproducible builds.
#PRESERVE_TIMESTAMP=yes

# Define to yes to build and stage as a regular user
# Default: yes, unless CCACHE_DIR is set and CCACHE_DIR_NON_ROOT_SAFE is not
# set.  Note that to use ccache with BUILD_AS_NON_ROOT you will need to
# use a non-shared CCACHE_DIR that is only built by PORTBUILD_USER and chowned
# to that user.  Then set CCACHE_DIR_NON_ROOT_SAFE to yes.
#BUILD_AS_NON_ROOT=no

# Define to the username to build as when BUILD_AS_NON_ROOT is yes.
# Default: nobody (uid PORTBUILD_UID)
#PORTBUILD_USER=nobody

# Define to the uid to use for PORTBUILD_USER if the user does not
# already exist in the jail.
# Default: 65532
#PORTBUILD_UID=65534

# Define pkgname globs to boost priority for
# Default: none
#PRIORITY_BOOST="pypy openoffice*"

# Define format for buildnames
# Default: %Y-%m-%d_%Hh%Mm%Ss
# ISO8601:
#BUILDNAME_FORMAT="%FT%TZ"

# Define format for build duration times
# Default: %H:%M:%S
#DURATION_FORMAT="%H:%M:%S"

# Use colors when in a TTY
# Default: yes
#USE_COLORS=yes

# Only build what is requested. Do not rebuild build deps if nothing requested
# depends on them. This can create an inconsistent repository if you often
# build one-off packages but expect the repository to stay consistent.
# Defaut: yes
#TRIM_ORPHANED_BUILD_DEPS=yes

# A list of directories to exclude from leftover and filesystem violation
# mtree checks.  Ccache is used here as an example but is already
# excluded by default.  There is no need to add it here unless a
# special configuration is used where it is a problem.
# Default: none
#LOCAL_MTREE_EXCLUDES="/usr/obj /var/tmp/ccache"

# Set to hosted to use the /data directory instead of inline style HTML
# Default: inline
#HTML_TYPE="hosted"

# Set to track remaining ports in the HTML interface.  This can slow down
# processing of the queue slightly, especially for bulk -a builds.
# Default: no
#HTML_TRACK_REMAINING=yes
----------------------------------------------------------------------
MASTER_SITE_GNOME=http://ftp.nara.wide.ad.jp/pub/X11/GNOME/%SUBDIR%/
MASTER_SITE_ISC=http://ftp.iij.ad.jp/pub/network/isc/%SUBDIR%/
.for mirror in osdn iij jaist ymu
MASTER_SITE_OSDN+= http://${mirror}.dl.osdn.jp/%SUBDIR%/
.endfor
MASTER_SITE_PERL_CPAN_BY=http://ftp.jaist.ac.jp/pub/CPAN/%CPANSORT%/%SUBDIR%/
MASTER_SITE_SUDO=http://core.ring.gr.jp/archives/misc/sudo/ http://www.ring.gr.jp/archives/misc/sudo/

LICENSES_ACCEPTED=AnotherHTMLlint NONE
----------------------------------------------------------------------
* 11.1-RELEASE amd64 jail is created and used.
* Ports tree used in section 1 is used as 'default' one of poudriere.

Then I executed following commands.

# poudriere testport -j 111amd64 editors/tamago@full
# poudriere testport -j 111amd64 editors/tamago@canna
# poudriere testport -j 111amd64 editors/tamago@nox
# poudriere testport -j 111amd64 editors/tamago@devel_full
# poudriere testport -j 111amd64 editors/tamago@devel_nox

And result is as following.

* Builds of full, canna and nox are failed with build phase error.
* Builds of devel_full and devel_nox are succeeded.

3. Builds with poudriere are failed because of bug of Emacs 26.1?

In order to make build log easy to understand, I added 'MAKE_JOBS_UNSAFE=yes' to editors/tamago/Makefile. Then I retried build of editors/tamago@full and got following logs.

----------------------------------------------------------------------
build started at Fri Jun  8 01:13:41 JST 2018
port directory: /usr/ports/editors/tamago
package name: tamago-emacs26-4.0.6.0.20041122.19.14_3
building for: FreeBSD 111amd64-default 11.1-RELEASE-p10 FreeBSD 11.1-RELEASE-p10 amd64
maintained by: hrs@FreeBSD.org
Makefile ident:      $FreeBSD: head/editors/tamago/Makefile 471191 2018-05-31 01:18:05Z jrm $
Poudriere version: 3.2.7
Host OSVERSION: 1101001
Jail OSVERSION: 1101001
Job Id: 

(snip)

=======================<phase: build          >============================
===>  Building for tamago-emacs26-4.0.6.0.20041122.19.14_3
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing aclocal-1.15 
(CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing autoheader)
rm -f stamp-h1
touch ./config.h.in
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing autoconf
cd . && /bin/sh ./config.status config.h
config.status: creating config.h
config.status: config.h is unchanged
/usr/bin/make  all-recursive
Making all in egg
cd .. && /usr/bin/make  am--refresh
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing aclocal-1.15 
 cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing automake-1.15 --gnu
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing autoconf
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing aclocal-1.15 
 cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing automake-1.15 --gnu
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing autoconf
cd .. && /usr/bin/make  am--refresh
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing aclocal-1.15 
 cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing automake-1.15 --gnu
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/missing autoconf
sed -e 's,[@]libexecdir@,/usr/local/libexec,' < canna.el.in > canna.el
sed -e 's,[@]libexecdir@,/usr/local/libexec,' < wnn.el.in > wnn.el
for F in cannarpc.el  canna.el  sj3rpc.el  sj3.el  wnnrpc.el  wnn.el  anthyipc.el  anthy.el; do  gzip -9 < $F > $F.gz;  done
if test "/usr/local/bin/emacs-26.1" != "no"; then  am__dir=. am__subdir_includes='';  case cannarpc.elc in */*)  am__dir=`echo 'cannarpc.elc' | sed 's,/[^/]*$,,'`;  am__subdir_includes="-L $am__dir -L ./$am__dir";  esac;  test -d "$am__dir" || /bin/mkdir -p "$am__dir" || exit 1;  /usr/local/bin/emacs-26.1 --batch  -q -no-site-file -no-init-file  --eval="(setq load-path (append (list (expand-file-name \"..\")) load-path))"  --eval="(load \"docomp.el\")"   $am__subdir_includes -L . -L .  --eval "(defun byte-compile-dest-file (f) \"cannarpc.elc\")"  --eval "(unless (byte-compile-file \"cannarpc.el\") (kill-emacs 1))";  else :; fi
Loading /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/docomp.el (source)...
Copying permissions to: Invalid argument, /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/egg/cannarpc.elc
*** Error code 255

Stop.
make[3]: stopped in /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/egg
*** Error code 1

Stop.
make[2]: stopped in /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14
*** Error code 1

Stop.
make[1]: stopped in /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14
*** Error code 1

Stop.
make: stopped in /usr/ports/editors/tamago
build of editors/tamago@full | tamago-emacs26-4.0.6.0.20041122.19.14_3 ended at Fri Jun  8 01:14:20 JST 2018
build time: 00:00:39
!!! build failure encountered !!!
----------------------------------------------------------------------

According to above logs, cause of build failure seems to be following error message.

Copying permissions to: Invalid argument, /wrkdirs/usr/ports/editors/tamago/work-full/tamago-4.0.6.0.20041122.19.14/egg/cannarpc.elc

Then I searched source tree of Emacs 26.1 and got following result.

yasu@rolling-vm-freebsd2[2041]% find -s emacs-26.1 -type f -print | xargs grep 'Copying permissions to'
emacs-26.1/src/fileio.c:      report_file_error ("Copying permissions to", newname);
emacs-26.1/src/fileio.c:      case -1: report_file_error ("Copying permissions to", newname);
yasu@rolling-vm-freebsd2[2041]%

I looked at emacs-26.1/src/fileio.c and found these lines are included in the C function that corresponds to 'copy-file' emacs lisp function.

So probably following steps result in above error message.

a. Makefile of tamago executes emacs to byte compile cannarpc.el.
b. While byte compiling 'copy-file' emasc-lisp function is called.
c. C function coressponding to 'copy-file' is invoked.
d. Some Error happens inside the C function.
e. Above error message is displayed.

But if above steps are right, step a to c should also taken when
building devel_full and devel_nox flavors.

So I think direct cause of build failure may be bug of Emacs 26.1.

I'll continue to investigating this problem and would like to fix it if possbile.

But anyway it is now clear that build of devel_full flavor is not broken. And attached patch fixes it. So please commit it.