Created attachment 193087 [details] ruby24-bdb-0.6.6_5.log I just got these errors in attached poudriere log ruby24-bdb-0.6.6_5.log: chmod: ./css/fonts.css: Operation not permitted chmod: ./css/rdoc.css: Operation not permitted chmod: ./js/jquery.js: Operation not permitted chmod: ./js/darkfish.js: Operation not permitted chmod: ./js/search.js: Operation not permitted These files are installed by lang/ruby24 and located in ${LOCALBASE}/lib/ruby/2.4/rdoc/generator/template/darkfish/ They would be copied into ${WRKSRC}/docs/doc of database/ruby-bdb while running the rdoc target. Actually, rdoc creates hard links for them: https://svn.ruby-lang.org/cgi-bin/viewvc.cgi/tags/v2_4_4/lib/rdoc/generator/darkfish.rb?revision=63013&view=markup#l557 563 begin 564 FileUtils.ln source, destination, options Hard links preserve the ownership of these files, even when poudriere runs building/staging as a regular user. % stat -f '%Sp %Su:%Sg %N' docs/doc/css/ drwxr-xr-x nobody:wheel docs/doc/css/ % stat -f '%Sp %Su:%Sg %N' docs/doc/css/*.css -rw-r--r-- root:wheel docs/doc/css/fonts.css -rw-r--r-- root:wheel docs/doc/css/rdoc.css databases/ruby-bdb uses ${COPYTREE_SHARE} to copy theses files from ${WRKSRC}/docs/doc to ${STAGEDIR}${DOCSDIR}/doc https://svnweb.freebsd.org/ports/head/databases/ruby-bdb/Makefile?revision=458847&view=markup#l65 65 post-install-DOCS-on: 66 ${MKDIR} ${STAGEDIR}${DOCSDIR}/doc 67 (cd ${WRKSRC} && ${INSTALL_DATA} ${DOCS} ${STAGEDIR}${DOCSDIR}) 68 (cd ${WRKSRC}/docs/doc && ${COPYTREE_SHARE} . ${STAGEDIR}${DOCSDIR}/doc) The cpio -l option in ${COPYTREE_SHARE} creates links instead of copying files that also preserve the ownership root:wheel https://svnweb.freebsd.org/ports/head/Mk/bsd.port.mk?revision=468917&view=markup#l2205 2205 COPYTREE_SHARE= ${SH} -c '(${FIND} -Ed $$1 $$3 | ${CPIO} -dumpl $$2 >/dev/null 2>&1) && \ 2206 ${FIND} -Ed $$1 $$3 \( -type d -exec ${SH} -c '\''cd '\''$$2'\'' && chmod 755 "$$@"'\'' . {} + \ 2207 -o -type f -exec ${SH} -c '\''cd '\''$$2'\'' && chmod ${_SHAREMODE} "$$@"'\'' . {} + \)' COPYTREE_SHARE It's the chmod ${_SHAREMODE} command in ${COPYTREE_SHARE} that wrote the error message in poudriere log: chmod: ./css/fonts.css: Operation not permitted chmod: ./css/rdoc.css: Operation not permitted ... On UFS, running "chmod 644 docs/doc/css/fonts.css" as a regular user "nobody" and that file has the identical mode (-rw-r--r--) owned by root doesn't raise an error. on ZFS, may_have_nfs4acl() returns 1 and then it runs fchmodat() https://svnweb.freebsd.org/base/head/bin/chmod/chmod.c?revision=326025&view=markup#l197 197 /* 198 * With NFSv4 ACLs, it is possible that applying a mode 199 * identical to the one computed from an ACL will change 200 * that ACL. 201 */ 202 if (may_have_nfs4acl(p, hflag) == 0 && 203 (newmode & ALLPERMS) == (p->fts_statp->st_mode & ALLPERMS)) 204 continue; 205 if (fchmodat(AT_FDCWD, p->fts_accpath, newmode, atflag) == -1 databases/ruby-bdb is a RUN_DEPENDS of ports-mgmt/portupgrade, I'm not aware if there are other ports that also use rdoc + ${COPYTREE_SHARE} which might cause the similar problem on ZFS, so I just attached a patch to databases/ruby-bdb rather than modifying the ${COPYTREE_SHARE} in bsd.port.mk. I've searched freebsd-pkg-fallout@ lists but didn't find anything about databases/ruby-bdb, it seems poudriere.conf has BUILD_AS_NON_ROOT=no enabled on the package cluster :) % fetch -q -o - http://beefy5.nyi.freebsd.org/data/104i386-default/468751/logs/bash-4.4.19.log | grep 'root[[:space:]]+wheel' -rwxr-xr-x 1 root wheel 1036686 May 1 01:18 bash
Created attachment 193088 [details] patch
@Fukang Thanks for the detailed bug report and patch. For patches to ports with maintainers (who are not you), could you in addition to creating Bugzilla issues (which is needed for approving with maintainer timeout if necessary), please create a review in Phabricator with a proposed commit log and QA confirmation notes as we have previously done and include that review in the issue's URL field. I believe this change is/can be considered Approved by: portmgr (infrastructure compliance blanket) CC'ing swills@ for opinion/question on other (ruby) ports using "rdoc + ${COPYTREE_SHARE}", for which modifying bsd.port.mk COPYSHARE_TREE may be valuable. If that's the preferred path, the change should probably be exp-run'd.
Thanks Kubilay. :) Sure, I will create a review in Phabricator when I figure out how to run the unit tests. # ruby24 tests/recnum.rb VERSION of BDB is Berkeley DB 5.3.28: (September 9, 2013) Loaded suite tests/recnum Started /usr/ports/databases/ruby-bdb/work/bdb-0.6.6/src/bdb.so: Undefined symbol "RCLASS_M_TBL"
Created attachment 193388 [details] make regression
Created attachment 193389 [details] make test
Patch here should be updated to match the review, or removed (obsoleted) in favour of the review code. Consider however: maintainer cant approve here if there's no patch (and must accept in phabricator)
Created attachment 193414 [details] patch v2
(In reply to Fukang Chen from comment #0) > % stat -f '%Sp %Su:%Sg %N' docs/doc/css/ > drwxr-xr-x nobody:wheel docs/doc/css/ > > % stat -f '%Sp %Su:%Sg %N' docs/doc/css/*.css > -rw-r--r-- root:wheel docs/doc/css/fonts.css > -rw-r--r-- root:wheel docs/doc/css/rdoc.css I cannot reproduce this: root@11amd64-ports:/wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6 # stat -f '%Sp %Su:%Sg %N' docs/doc/css/ drwxr-xr-x nobody:wheel docs/doc/css/ root@11amd64-ports:/wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6 # root@11amd64-ports:/wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6 # stat -f '%Sp %Su:%Sg %N' docs/doc/css/*.css -rw-r--r-- nobody:wheel docs/doc/css/fonts.css -rw-r--r-- nobody:wheel docs/doc/css/rdoc.css how did you manage to get those files extracted as with the root user? The only thing that is done as root is fetching, extracting is done as a user, so it cannot produces files owned by root.
(In reply to Mathieu Arnold from comment #8) Have you tried to set USE_TMPFS=no in poudriere.conf, I think wrkdirs is on tmpfs by default. tmpfs worked for me too.
Created attachment 193425 [details] ruby24-bdb-0.6.6_5.log (USE_TMPFS=yes)
Created attachment 193426 [details] ruby24-bdb-0.6.6_5.log (USE_TMPFS=no)
Created attachment 193427 [details] ruby24-bdb-0.6.6_5.log (USE_TMPFS=yes)
rdoc tries to create hard links using FileUtils.ln (line 564) if it's failed then copy the file using FileUtils.cp (line 570) https://svn.ruby-lang.org/cgi-bin/viewvc.cgi/tags/v2_4_4/lib/rdoc/generator/darkfish.rb?revision=63013&view=markup#l557 557 def install_rdoc_static_file source, destination, options # :nodoc: 558 return unless source.exist? 559 560 begin 561 FileUtils.mkdir_p File.dirname(destination), options 562 563 begin 564 FileUtils.ln source, destination, options 565 rescue Errno::EEXIST 566 FileUtils.rm destination 567 retry 568 end 569 rescue 570 FileUtils.cp source, destination, options 571 end 572 end I just rebuild databases/ruby-bdb with the rdoc -D option ($DEBUG_RDOC): --- extconf.rb.orig 2011-04-06 19:35:39 UTC +++ extconf.rb @@ -50,7 +50,7 @@ SUBDIRS = #{subdirs.join(' ')} rdoc: docs/doc/index.html docs/doc/index.html: $(RDOC) -\t@-(cd docs; rdoc .) +\t@-(cd docs; ${RUBY_RDOC} -D .) ri: \t@-(rdoc -r docs/*rb) When USE_TMPFS=no, both source and target files are within the same filesytems, ln worked and hard links preserve the ownership of these files (root:wheel): Generating Darkfish format into /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc... Copying static files mkdir -p ./css ln /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/css/fonts.css ./css/fonts.css mkdir -p ./fonts When USE_TMPFS=yes, the wrkdirs direcotry is on tmpfs, hard link doesn't work these files were copied to ${WRKSRC}/docs/doc/ and they have new ownership (nobody:wheel): Generating Darkfish format into /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc... Copying static files mkdir -p ./css ln /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/css/fonts.css ./css/fonts.css cp /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/css/fonts.css ./css/fonts.css mkdir -p ./fonts
Mmmm, this looks like a poudriere bug to me. With my usual USE_TMPFS, I get: root@11amd64-ports:~ # find /wrkdirs/ -name jquery.js -exec ls -l {} + -rw-r--r-- 2 nobody wheel 91669 May 16 10:23 /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js -rw-r--r-- 2 nobody wheel 91669 May 16 10:23 /wrkdirs/usr/ports/databases/ruby-bdb/work/stage/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js With USE_TMPFS=no, I get: root@11amd64-ports:~ # find /wrkdirs/ -name jquery.js -exec ls -l {} + -rw-r--r-- 3 root wheel 91669 Sep 5 2014 /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js -rw-r--r-- 3 root wheel 91669 Sep 5 2014 /wrkdirs/usr/ports/databases/ruby-bdb/work/stage/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js I don't understand why one uses the normal nobody and the other has root.
(In reply to Mathieu Arnold from comment #14) > With my usual USE_TMPFS, I get: > > root@11amd64-ports:~ # find /wrkdirs/ -name jquery.js -exec ls -l {} + > -rw-r--r-- 2 nobody wheel 91669 May 16 10:23 /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js > -rw-r--r-- 2 nobody wheel 91669 May 16 10:23 /wrkdirs/usr/ports/databases/ruby-bdb/work/stage/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js This file belongs to the port lang/ruby24, the ownership is root:wheel: /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/js/jquery.js and it's located on ZFS, rdoc process (run as user nobody) tries to create a hard link for this file to /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js With USE_TMPFS=yes, /wrkdirs is on tmpfs, both source file and target file were not within the same filesystem, so hard link didn't work there, then the rdoc process (run as user nobody) created a copy of it, that's why the file owner is nobody. > With USE_TMPFS=no, I get: > > root@11amd64-ports:~ # find /wrkdirs/ -name jquery.js -exec ls -l {} + >-rw-r--r-- 3 root wheel 91669 Sep 5 2014 /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js >-rw-r--r-- 3 root wheel 91669 Sep 5 2014 /wrkdirs/usr/ports/databases/ruby-bdb/work/stage/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js With USE_TMPFS=no, now both /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/js/jquery.js and /wrkdirs/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js are on ZFS, the rdoc process (run as user nobody) created a hard link for this file, they have the same inode number and ownership, even the process was run by a normal user. and ${COPYTREE_SHARE} uses cpio -l, that's why it's also a hard link in the stage directory: /wrkdirs/usr/ports/databases/ruby-bdb/work/stage/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js Hope this helps.
Okay, maybe we could try to explain it by compiling this port as a normal user, my /usr/local and /tmp are on the same filesystem: % id uid=1001(loader) gid=1001(loader) groups=1001(loader) % cd /usr/ports/databases/ruby-bdb % make WRKDIRPREFIX=/tmp ... ... ... ... ... ... % ls -il /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/js/jquery.js \ $(make WRKDIRPREFIX=/tmp -V WRKSRC)/docs/doc/js/jquery.js \ $(make WRKDIRPREFIX=/tmp -V STAGEDIR)/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js 1730459 -rw-r--r-- 3 root wheel 91669 Sep 5 2014 /tmp/usr/ports/databases/ruby-bdb/work/bdb-0.6.6/docs/doc/js/jquery.js 1730459 -rw-r--r-- 3 root wheel 91669 Sep 5 2014 /tmp/usr/ports/databases/ruby-bdb/work/stage/usr/local/share/doc/ruby24/bdb/doc/js/jquery.js 1730459 -rw-r--r-- 3 root wheel 91669 Sep 5 2014 /usr/local/lib/ruby/2.4/rdoc/generator/template/darkfish/js/jquery.js They are hard links and have the same inode number.