Bug 209786 - ports-mgmt/poudriere: WITH_CCACHE_BUILD not compatible with BUILD_AS_NON_ROOT
Summary: ports-mgmt/poudriere: WITH_CCACHE_BUILD not compatible with BUILD_AS_NON_ROOT
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Bryan Drewery
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2016-05-27 09:28 UTC by Victor Sudakov
Modified: 2017-02-07 22:08 UTC (History)
6 users (show)

See Also:
bugzilla: maintainer-feedback? (cy)


Attachments
Complete poudriere build log (7.79 KB, text/plain)
2016-05-27 09:28 UTC, Victor Sudakov
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Victor Sudakov 2016-05-27 09:28:14 UTC
Created attachment 170719 [details]
Complete poudriere build log

===>  Building for mkcmd-8.14_1
cc -O2 -pipe -fstack-protector -fno-strict-aliasing   -DFREEBSD  -c main.c
ccache: error: Failed to create temporary file for /root/.ccache/tmp/tmp.cpp_stderr: Permission denied
*** [main.o] Error code 1

Stop in /wrkdirs/usr/ports/devel/mkcmd/work/mkcmd-8.14/bin/mkcmd.
*** [do-build] Error code 1

Stop in /usr/ports/devel/mkcmd.
====>> Cleaning up wrkdir
===>  Cleaning for mkcmd-8.14_1
build of devel/mkcmd ended at Fri May 27 12:14:39 OMST 2016
build time: 00:00:03
!!! build failure encountered !!!
Comment 1 Victor Sudakov 2016-05-27 09:29:19 UTC
poudriere-devel-3.1.99.20160520
Comment 2 Victor Sudakov 2016-05-27 16:36:12 UTC
Excuse me, what does "needs-qa" mean?
Comment 3 Cy Schubert freebsd_committer 2016-05-27 19:40:04 UTC
Look at your output:

cc -O2 -pipe -fstack-protector -fno-strict-aliasing   -DFREEBSD  -c main.c
ccache: error: Failed to create temporary file for /root/.ccache/tmp/tmp.cpp_stderr: Permission denied
*** [main.o] Error code 1

let's look at the last poudriere-devel update:

415468 | bdrewery | 2016-05-18 15:59:51 -0700 (Wed, 18 May 2016) | 14 lines

- Update to 3.0-1657-g0b204f0

Changes:
  - jail/ports -u: Default git pull to use --rebase.
  - destroyfs: Run chflags -R 0 and on a smaller subset of files.
  - ports: Ignore failures when removing the ports tree
  - jail: Ignore errors trying to remove the jail
  - Create port-specific directories in do_portbuild_mounts.
  - Rework the cleanup to respect -x inside of the directory.
  - Support -(j|p|z)<param> without space for poudriere.conf lookups
  - hash_var_name: Quote the pattern to _gsub to avoid globbing local files.
  - Update sh from head @ r296813
  - Switch BUILD_AS_NON_ROOT as default to on.

It sets BUILD_AS_NON_ROOT to on. You're getting permission denied. Your /root/.ccache disallow BUILD_AS_NON_ROOT to write to to it. Either open up /root/.ccache to allow non-owner to write, add an ACL, or configure ccache to use another directory tree.
Comment 4 Victor Sudakov 2016-05-28 12:10:39 UTC
Dear Cy,

I cannot possibly "open up /root/.ccache to allow non-owner to write" or change ccache configuration because the jail and the build environment are created automatically by poudriere and destroyed by poudriere after the build.

Perhaps poudriere maintainers should be notified. I insist it's a bug, even if not a bug in devel/mkcmd (though /root/.ccache seems a strange place to me).

For the present, I'll define BUILD_AS_NON_ROOT=no, thank you for the hint.

Still, what's "needs-qa"?
Comment 5 Bryan Drewery freebsd_committer 2016-05-28 16:04:24 UTC
The only answer for this is set BUILD_AS_NON_ROOT=no.  It's a general problem
with CCACHE_BUILD + BUILD_AS_NON_ROOT.
Comment 6 Victor Sudakov 2016-05-31 00:05:08 UTC
(In reply to Bryan Drewery from comment #5)
Bryan,

However, of all the 800+ packages I compile in poudriere, only devel/mkcmd has this problem. All the rest are fine with CCACHE_BUILD + BUILD_AS_NON_ROOT=yes.
Comment 7 commit-hook freebsd_committer 2016-05-31 16:47:58 UTC
A commit references this bug:

Author: bdrewery
Date: Tue May 31 16:47:49 UTC 2016
New revision: 416212
URL: https://svnweb.freebsd.org/changeset/ports/416212

Log:
  - Update to 3.0-1662-gc11d922

  Changes:
   - Fix incorrect handling of null-mounting /nxb-bin in jail
   - Disable BUILD_AS_NON_ROOT if CCACHE_DIR is set [1]
   - Fix mount path too long when using linprocfs

  PR:		209786 [1]

Changes:
  head/ports-mgmt/poudriere-devel/Makefile
  head/ports-mgmt/poudriere-devel/distinfo
Comment 8 Bryan Drewery freebsd_committer 2016-05-31 16:48:23 UTC
Not fixed, but mitigated.
Comment 9 Adam Weinberger freebsd_committer 2016-05-31 16:58:43 UTC
If I've manually set CCACHE_DIR to something specific for this jail, chowned to nobody:nobody, BUILD_AS_NON_ROOT works just fine.

Could it perhaps only disable BUILD_AS_NON_ROOT if CCACHE_DIR is /root/.ccache, or if CCACHE_DIR is not nobody:nobody?
Comment 10 Bryan Drewery freebsd_committer 2016-05-31 21:35:04 UTC
(In reply to Adam Weinberger from comment #9)
> If I've manually set CCACHE_DIR to something specific for this jail, chowned
> to nobody:nobody, BUILD_AS_NON_ROOT works just fine.
> 
> Could it perhaps only disable BUILD_AS_NON_ROOT if CCACHE_DIR is
> /root/.ccache, or if CCACHE_DIR is not nobody:nobody?

Even that is not a sure way to detect if it is OK.  I would need to check
every directory inside of the ccache dir, and even then it would assume the
user is not sharing the ccache dir outside of Poudriere from root which would
then break.

~/.ccache # chown nobody:nobody .
~/.ccache # touch hm # Simulating running ccache from outside of Poudriere
~/.ccache # ls -al hm
-rw-r--r--  1 root  nobody  0 May 31 14:31 hm

Bam, broken for the next Poudriere build.  The stats files would be a problem with
this as well.

The only thing I see working for everyone is an override option to force
using these options together.

I didn't flip the default for BUILD_AS_NON_ROOT for all of this time precisely
because ccache would break.  Then I did.  Then I've received numerous reports
about it breaking.  This commit was to restore *working* behavior for everyone,
not just some.
Comment 11 commit-hook freebsd_committer 2016-05-31 22:31:34 UTC
A commit references this bug:

Author: bdrewery
Date: Tue May 31 22:31:17 UTC 2016
New revision: 416224
URL: https://svnweb.freebsd.org/changeset/ports/416224

Log:
  - Update to 3.0-1663-gf6c1bad

  Changes:
   - Tweak ccache+BUILD_AS_NON_ROOT so that BUILD_AS_NON_ROOT is just default
     off, with a warning, when CCACHE_DIR is set.

  PR:		209786

Changes:
  head/ports-mgmt/poudriere-devel/Makefile
  head/ports-mgmt/poudriere-devel/distinfo
Comment 12 commit-hook freebsd_committer 2016-06-03 22:37:44 UTC
A commit references this bug:

Author: bdrewery
Date: Fri Jun  3 22:36:48 UTC 2016
New revision: 416341
URL: https://svnweb.freebsd.org/changeset/ports/416341

Log:
  - Update to checkin 3.0-1668-g5880c23

  Changes:
    - jail: Fix usage for -m src=PATH
    - jail -v: Note that non-listed methods used the value only for display.
    - QEMU: Copy the latest emulator into the jail at startup. This fixes
      the jail running into bugs with qemu that are fixed with the latest
      qemu.  It was only copying it in during creation before.
    - bulk: Avoid redundant cleaning message with jailversion cleanup.
    - Add and document PORTBUILD_UID and CCACHE_DIR_NON_ROOT_SAFE. [1]

  PR:		209786 [1]

Changes:
  head/ports-mgmt/poudriere-devel/Makefile
  head/ports-mgmt/poudriere-devel/distinfo