Created attachment 182193 [details] Add NO_ARCH=yes for USES=cran (without :compiles) Please test the attached patch that marks platform independent R-cran code as such if USES=cran:compiles is not specified. Affects all ports with USES=cran
The patch seems wrong: .if ${cran_ARGS:Mcompiles} +NO_ARCH= yes This sets NO_ARCH to ports *with* compiles, not without.
That is very awkward: I locally tested the patch without any issues. I'll enhance qa.sh and try again.
Created attachment 182647 [details] Add NO_ARCH=yes for USES=cran (without :compiles) [v1.1.0] Update patch to fix NO_ARCH setting, and introduce a qa.sh check to verify NO_ARCH doesn't introduce any architecture specific code (currently assumes to be application/x-(executable|object|sharedlib)).
There are false positives with this qa check so it shouldn't give an error, just a warning. (for instance the following ports I maintain give false positives : devel/py-pyelftools or security/rubygem-metasploit-payloads )
Created attachment 182714 [details] Add NO_ARCH=yes for USES=cran (without:compiles) [v1.1.1] Switch to reading the more specific ELF details, include checking for the FreeBSD specifically. Also switch to warning due to the potential for a false positive. I really don't like false positives, so I am happy to investigate and eliminate them.
I have the following qa warnings: R-cran-AMORE-0.2.14_2.log:Warning: './usr/local/lib/R/library/AMORE/libs/AMORE.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-Amelia-1.7.4.log:Warning: './usr/local/lib/R/library/Amelia/libs/Amelia.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-DCluster-0.2.7.log:Warning: './usr/local/lib/R/library/DCluster/libs/DCluster.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-GDD-0.1.13.1_10.log:Warning: './usr/local/lib/R/library/GDD/libs/GDD.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-Hmisc-3.10.1_7.log:Warning: './usr/local/lib/R/library/Hmisc/libs/Hmisc.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-Matching-4.9.2.log:Warning: './usr/local/lib/R/library/Matching/libs/Matching.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-NMF-0.20.6.log:Warning: './usr/local/lib/R/library/NMF/libs/NMF.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-PerformanceAnalytics-1.4.3541.log:Warning: './usr/local/lib/R/library/PerformanceAnalytics/libs/PerformanceAnalytics.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-RCurl-1.95.4.8.log:Warning: './usr/local/lib/R/library/RCurl/libs/RCurl.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-RHmm-2.0.3_5.log:Warning: './usr/local/lib/R/library/RHmm/libs/RHmm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-RJSONIO-1.3.0.log:Warning: './usr/local/lib/R/library/RJSONIO/libs/RJSONIO.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-RSQLite-1.0.0.log:Warning: './usr/local/lib/R/library/RSQLite/libs/RSQLite.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-RSQLite.extfuns-0.0.1_7.log:Warning: './usr/local/lib/R/library/RSQLite.extfuns/libs/RSQLite.extfuns.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-RSvgDevice-0.6.4.4_1.log:Warning: './usr/local/lib/R/library/RSvgDevice/libs/RSvgDevice.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-TTR-0.23.1.log:Warning: './usr/local/lib/R/library/TTR/libs/TTR.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-VGAM-1.0.3.log:Warning: './usr/local/lib/R/library/VGAM/libs/VGAM.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-bayesm-3.0.2.log:Warning: './usr/local/lib/R/library/bayesm/libs/bayesm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-bdsmatrix-1.3.2_1.log:Warning: './usr/local/lib/R/library/bdsmatrix/libs/bdsmatrix.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-bit-1.1.12.log:Warning: './usr/local/lib/R/library/bit/libs/bit.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-bitops-1.0.6_4.log:Warning: './usr/local/lib/R/library/bitops/libs/bitops.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-caTools-1.17.1.log:Warning: './usr/local/lib/R/library/caTools/libs/caTools.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-ccgarch-0.2.3_1.log:Warning: './usr/local/lib/R/library/ccgarch/libs/ccgarch.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-chron-2.3.43_7.log:Warning: './usr/local/lib/R/library/chron/libs/chron.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-cmprsk-2.2.7.log:Warning: './usr/local/lib/R/library/cmprsk/libs/cmprsk.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-dplyr-0.5.0.log:Warning: './usr/local/lib/R/library/dplyr/libs/dplyr.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-eco-3.1.6_5.log:Warning: './usr/local/lib/R/library/eco/libs/eco.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-etm-0.6.2.log:Warning: './usr/local/lib/R/library/etm/libs/etm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-fBasics-3011.87.log:Warning: './usr/local/lib/R/library/fBasics/libs/fBasics.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-fracdiff-1.4.2_5.log:Warning: './usr/local/lib/R/library/fracdiff/libs/fracdiff.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-gbm-2.0.8_6.log:Warning: './usr/local/lib/R/library/gbm/libs/gbm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-geepack-1.2.1.log:Warning: './usr/local/lib/R/library/geepack/libs/geepack.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-glmnet-1.8.5_6.log:Warning: './usr/local/lib/R/library/glmnet/libs/glmnet.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-gpclib-1.5.5_5.log:Warning: './usr/local/lib/R/library/gpclib/libs/gpclib.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-gtools-3.5.0.log:Warning: './usr/local/lib/R/library/gtools/libs/gtools.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-httpuv-1.3.3.log:Warning: './usr/local/lib/R/library/httpuv/libs/httpuv.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-igraph-1.0.1.log:Warning: './usr/local/lib/R/library/igraph/libs/igraph.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-lazyeval-0.2.0.log:Warning: './usr/local/lib/R/library/lazyeval/libs/lazyeval.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-lmtest-0.9.35.log:Warning: './usr/local/lib/R/library/lmtest/libs/lmtest.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-markdown-0.8.log:Warning: './usr/local/lib/R/library/markdown/libs/markdown.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-microbenchmark-1.4.2.log:Warning: './usr/local/lib/R/library/microbenchmark/libs/microbenchmark.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-minqa-1.2.4.log:Warning: './usr/local/lib/R/library/minqa/libs/minqa.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-mvtnorm-0.9.96_15.log:Warning: './usr/local/lib/R/library/mvtnorm/libs/mvtnorm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-nloptr-1.0.4.log:Warning: './usr/local/lib/R/library/nloptr/libs/nloptr.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-nnls-1.4_7.log:Warning: './usr/local/lib/R/library/nnls/libs/nnls.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-plyr-1.8.4.log:Warning: './usr/local/lib/R/library/plyr/libs/plyr.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-png-0.1.7_3.log:Warning: './usr/local/lib/R/library/png/libs/png.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-pystr-2.0.0.log:Warning: './usr/local/lib/R/library/pystr/libs/pystr.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-quadprog-1.5.5_5.log:Warning: './usr/local/lib/R/library/quadprog/libs/quadprog.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-rgenoud-5.7.12.4.log:Warning: './usr/local/lib/R/library/rgenoud/libs/rgenoud.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-rjson-0.2.15.log:Warning: './usr/local/lib/R/library/rjson/libs/rjson.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-rtiff-1.4.5.log:Warning: './usr/local/lib/R/library/rtiff/libs/rtiff.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-sm-2.2.5.4_1.log:Warning: './usr/local/lib/R/library/sm/libs/sm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-sp-1.2.4.log:Warning: './usr/local/lib/R/library/sp/libs/sp.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-truncnorm-1.0.7.log:Warning: './usr/local/lib/R/library/truncnorm/libs/truncnorm.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-urca-1.3.0.log:Warning: './usr/local/lib/R/library/urca/libs/urca.so' is a architecture specific binary file and you have NO_ARCH=yes set R-cran-xts-0.9.7_5.log:Warning: './usr/local/lib/R/library/xts/libs/xts.so' is a architecture specific binary file and you have NO_ARCH=yes set
Created attachment 182834 [details] Add NO_ARCH=yes for USES=cran (without:compiles) [v1.1.2] Correctly mark cran ports as compiling. This (in my tests) fixes all the qa warnings. I also fixed the warning message. Some questions: 1a) Can you approve the changes to qa.sh?, or 1b) Do I need to submit a separate qa.sh for qa.sh. 2a) Is there appetite to run a exp-run against all ports setting NO_ARCH=yes with the `no_arch' check?, and 2b) If there are no false positives (or a way to mitigate them) to make the check error out. 3) Do you want to do another exp-run to check the patch? Otherwise, I think we have a patch that is ready to land :-D
There is already a warning when pkg is ran with developper mode (poudriere does this with testport or bulk -t) Here are the packages with warnings: ./bluemix-cli-0.4.6.log ./intellij-ultimate-2016.2.5.log ./metasploit-4.14.21.log ./p5-CDB_File-0.98.log ./p5-Data-Structure-Util-0.16.log ./p5-Mouse-2.4.9,1.log ./p5-Net-SSH-Perl-2.09.01.log ./p5-Opcodes-0.14.log ./p5-Selenium-Remote-Driver-1.12.log ./py27-Genshi-0.7_1.log ./py27-SpeechRecognition-3.5.0.log ./py27-atomiclong-0.1.1.log ./py27-psycopg2cffi-2.7.5.log ./py27-pyelftools-0.24.log ./py27-rcssmin-1.0.6.log ./py27-rjsmin-1.0.12.log ./py27-rxp-0.9_2.log ./pyside-tools-py27-0.2.15.log ./rubygem-hamlit-2.8.1_1.log ./rubygem-metasploit-payloads-1.2.29.log ./rubygem-metasploit_payloads-mettle-0.1.9.log ./rubygem-rbpdf-font-1.19.1.log ./rubygem-tarantool-0.4.4.0.log ./rubygem-vmstat-2.3.0.log ./rubygem-websocket-driver-0.6.5.log ./timelimit-1.8.1.log ./u-boot-rpi-2016.01_1.log ./u-boot-rpi2-2015.04.log ./u-boot-rpi3-2017.01.log Some are false positives (metasploit, rubygem-metasploit-payloads, rubygem-metasploit_payloads-mettle, u-boot-rpi, u-boot-rpi2, u-boot-rpi3) The others may be errors.
py27-pyelftools is false positive too
Created attachment 182973 [details] Add NO_ARCH=yes for USES=cran (without:compiles) [v1.2.0] Changes: - Switch no_arch to err from warning (no more false positives) - Added ability to list files that are FreeBSD binaries but should be considered architecure neutral - Fixed the ports listed previously bluemix-cli: false positive fixed p5-CDB_File: NO_ARCH removed p5-Data-Structure-Util: NO_ARCH removed p5-Mouse: NO_ARCH removed p5-Net-SSH-Perl: NO_ARCH removed p5-Opcodes: NO_ARCH removed p5-Selenium-Remote-Driver: false positive fixed (stripped gives a false positive here) py27-Genshi: NO_ARCH removed py27-SpeechRecognition: false positive fixed (stripped gives a false positive here) (this port should probably set ONLY_FOR_ARCHS=i386 amd64 as it includes a x86 linux binary) py27-atomiclong: NO_ARCH removed py27-psycopg2cffi: NO_ARCH removed py27-pyelftools: false positive fixes py27-rcssmin: NO_ARCH removed py27-rjsmin: NO_ARCH removed py27-rxp: NO_ARCH removed pyside-tools: NO_ARCH removed rubygem-hamlit: NO_ARCH removed rubygem-metasploit-payloads: false positive fixed rubygem-metasploit_payloads-mettle: false positive fixed rubygem-rbpdf-font: false positive fixed (stripped gives a false positive here) rubygem-tarantool: NO_ARCH removed rubygem-vmstat: NO_ARCH removed rubygem-websocket-driver: NO_ARCH removed timelimit: NO_ARCH removed u-boot-rpi: false positive fixed u-boot-rpi2: false positive fixed u-boot-rpi3: false positive fixed The remaining false positives were fixed by listing the offending files with NO_ARCH. I felt this approach was the most targetted way to override the checks while maintaining the ability detect regressions in these ports (i.e. they actually become architecure specific): intellij-ultimate: bundles shared libraries, (this port should probably set ONLY_FOR_ARCHS=i386 amd64 to match the bundles libraries) metasploit: bundles binary files
I like it. Can another portmgr@ review it?
Instead of changing NO_ARCH semantics, maybe a new variable should be added, as NO_ARCH is used in other places. For example, NO_ARCH_IGNORE where files to ignore are stored. For the qa script: I wonder if it should not be using the list_stagedir_elf function. file(1) has a passthrough mode, see the proxydeps function (no need for xargs). files can contain :, so using cut with that anchor is a bad idea, better do it like it is done in proxydeps. f#${STAGEDIR}${PREFIX}/} should probably be f#${STAGEDIR}} because files can be outside of PREFIX. grep probably needs -a to treat every file as text.
Created attachment 183393 [details] Add NO_ARCH=yes for USES=cran (without:compiles) [v1.2.1] Changes: - Incorporate all of mat@'s comments - Bump PORTREVISION for all ports that had NO_ARCH removed - Added documentation about NO_ARCH_IGNORE to bsd.ports.mk FYI, I checked that all current uses of NO_ARCH didn't break with using it with NO_ARCH_IGNORE, however given the potential for future breakage I followed mat@'s advice.
This looks fine except for 1 port: www/trac-iniadmin There is a backquote in the ARCH variable that causes problem: ====> Running Q/A tests (stage-qa) /bin/sh: Syntax error: EOF in backquote substitution /bin/sh: Syntax error: Error in command substitution *** Error code 2
Thus my point of using another variable :-)
mat@, FYI: this error came about because I used a different variable and stopped passing NO_ARCH to qa.sh with quotes. In this case: # make -C /usr/ports/www/trac-iniadmin -V NO_ARCH yes` (not that trailing '`'). I'll fix the port and commit this patch
A commit references this bug: Author: dbn Date: Sun Jun 18 17:58:40 UTC 2017 New revision: 443844 URL: https://svnweb.freebsd.org/changeset/ports/443844 Log: www/trac-iniadmin: remove trailing backtick from NO_ARCH The unescaped backtick causes issues when NO_ARCH is passed, unquoted, to a script. PR: 218976 Approved by: portmgr (implicit: minor fixes) Changes: head/www/trac-iniadmin/Makefile
A commit references this bug: Author: dbn Date: Sun Jun 18 18:09:22 UTC 2017 New revision: 443845 URL: https://svnweb.freebsd.org/changeset/ports/443845 Log: qa: add check for NO_ARCH If NO_ARCH is set then check that no FreeBSD elf(5) files are in $STAGEDIR. If an elf(5) file is bundles as part of the package, but is not meant to be run directly (i.e. the elf(5) file is a payload, and not compiled) then those files can be added to NO_ARCH_IGNORE to avoid the check from failing, Changes to ports: - Ports that have NO_ARCH set, but actually compile files have had NO_ARCH removed. - Ports that have elf(5) payloads have had those files added to NO_ARCH_IGNORE. - R-cran ports that do not set USES=cran:compiles have NO_ARCH set, PR: 218976 Reviewed by: antoine, mat Approved by: portmgr Changes: head/Mk/Scripts/qa.sh head/Mk/Uses/cran.mk head/Mk/bsd.port.mk head/archivers/py-rcssmin/Makefile head/archivers/py-rjsmin/Makefile head/converters/R-cran-RJSONIO/Makefile head/converters/R-cran-rjson/Makefile head/databases/R-cran-RSQLite/Makefile head/databases/R-cran-RSQLite.extfuns/Makefile head/databases/p5-CDB_File/Makefile head/databases/py-psycopg2cffi/Makefile head/databases/rubygem-tarantool/Makefile head/devel/R-cran-Hmisc/Makefile head/devel/R-cran-bit/Makefile head/devel/R-cran-bitops/Makefile head/devel/R-cran-caTools/Makefile head/devel/R-cran-chron/Makefile head/devel/R-cran-gbm/Makefile head/devel/R-cran-glmnet/Makefile head/devel/R-cran-gtools/Makefile head/devel/R-cran-microbenchmark/Makefile head/devel/R-cran-plyr/Makefile head/devel/p5-Data-Structure-Util/Makefile head/devel/p5-Mouse/Makefile head/devel/p5-Opcodes/Makefile head/devel/py-atomiclong/Makefile head/devel/pyside-tools/Makefile head/finance/R-cran-PerformanceAnalytics/Makefile head/finance/R-cran-TTR/Makefile head/finance/R-cran-ccgarch/Makefile head/finance/R-cran-fBasics/Makefile head/finance/R-cran-lmtest/Makefile head/finance/R-cran-urca/Makefile head/ftp/R-cran-RCurl/Makefile head/graphics/R-cran-GDD/Makefile head/graphics/R-cran-png/Makefile head/graphics/R-cran-rtiff/Makefile head/java/intellij-ultimate/Makefile head/math/R-cran-Amelia/Makefile head/math/R-cran-Matching/Makefile head/math/R-cran-NMF/Makefile head/math/R-cran-RHmm/Makefile head/math/R-cran-RSvgDevice/Makefile head/math/R-cran-VGAM/Makefile head/math/R-cran-bdsmatrix/Makefile head/math/R-cran-car/Makefile head/math/R-cran-dplyr/Makefile head/math/R-cran-fracdiff/Makefile head/math/R-cran-geepack/Makefile head/math/R-cran-gpclib/Makefile head/math/R-cran-igraph/Makefile head/math/R-cran-lazyeval/Makefile head/math/R-cran-minqa/Makefile head/math/R-cran-mvtnorm/Makefile head/math/R-cran-nloptr/Makefile head/math/R-cran-nnls/Makefile head/math/R-cran-quadprog/Makefile head/math/R-cran-rgenoud/Makefile head/math/R-cran-sm/Makefile head/math/R-cran-sp/Makefile head/math/R-cran-truncnorm/Makefile head/math/R-cran-xts/Makefile head/net/p5-Net-SSH-Perl/Makefile head/science/R-cran-AMORE/Makefile head/science/R-cran-DCluster/Makefile head/science/R-cran-bayesm/Makefile head/science/R-cran-cmprsk/Makefile head/science/R-cran-eco/Makefile head/science/R-cran-etm/Makefile head/security/metasploit/Makefile head/sysutils/rubygem-vmstat/Makefile head/sysutils/timelimit/Makefile head/textproc/R-cran-R2HTML/Makefile head/textproc/R-cran-markdown/Makefile head/textproc/R-cran-pystr/Makefile head/textproc/py-genshi/Makefile head/textproc/py-rxp/Makefile head/www/R-cran-httpuv/Makefile head/www/rubygem-websocket-driver/Makefile
Committed, thanks for reviewing :-)