Bug 243331

Summary: net/samba412: build_fs_violation in poudriere
Product: Ports & Packages Reporter: Po-Chuan Hsieh <sunpoet>
Component: Individual Port(s)Assignee: Timur I. Bakeyev <timur>
Status: Closed FIXED    
Severity: Affects Some People CC: andrew, mandree
Priority: --- Flags: bugzilla: maintainer-feedback? (timur)
Version: Latest   
Hardware: Any   
OS: Any   

Description Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-01-13 18:44:04 UTC
build_fs_violation in poudriere

=>> Checking for filesystem violations... done
=>> Error: Filesystem touched during build:
extra: usr/local/lib/python3.7/site-packages/__pycache__/tevent.cpython-37.pyc
=>> Cleaning up wrkdir

Compile and install tevent.py to fix this problem.
Comment 1 Timur I. Bakeyev freebsd_committer freebsd_triage 2020-01-14 01:29:01 UTC
Hi, Sunpoet!

Can you confirm that this is the latest (4.10.11) version of the port?

As it has:

USES+=                          python:3.4+
PLIST_SUB+=                     SAMBA4_PYTHON=""
SUB_LIST+=                      SAMBA4_PYTHON=""
CONFIGURE_ARGS+=                --nopycache

Which supposed not to generate any pycache files.
Comment 2 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-01-18 11:30:59 UTC
Hi Timur, 

Yes, It's the latest verision. I put the log on https://people.FreeBSD.org/~sunpoet/temp/samba410-4.10.11.log
Comment 3 Andrew Fyfe 2020-02-02 14:37:44 UTC
I only have this issue when running poudriere bulk with the -t option (Add some testing to the specified ports), without the -t option the port builds without issue.
Comment 4 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-02-28 16:35:06 UTC
(In reply to Timur I. Bakeyev from comment #1)
(In reply to Andrew Fyfe from comment #3)

It seems poudriere only checks filesystem violations when -t option is used.

I've prepared two log with/without poudriere -t option.

https://people.FreeBSD.org/~sunpoet/temp/samba410-4.10.13.log_poudriere_with_-t_flag
https://people.FreeBSD.org/~sunpoet/temp/samba410-4.10.13.log_poudriere_without_-t_flag
Comment 5 Timur I. Bakeyev freebsd_committer freebsd_triage 2020-04-09 21:36:17 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #4)

I feel really dumb here, as I still can't get it reproduced... Maybe, I incidentally fixed it or that error requires some additional configuration to be spotted...

# # poudriere bulk -t -j FreeBSD-12_1-amd64 -p devel -z samba net/samba410

[4073/4073] Compiling pidl/lib/Parse/Pidl/Wireshark/NDR.pm
Waf: Leaving directory `/wrkdirs/usr/ports/net/samba410/work/samba-4.10.14/bin/default'
'build' finished successfully (2m53.597s)
===========================================================================
=>> Checking for filesystem violations... done
=======================<phase: run-depends    >============================
===>   samba410-4.10.14 depends on package: jansson>=2.10 - found
===>   samba410-4.10.14 depends on package: libarchive>=3.1.2 - found
===>   samba410-4.10.14 depends on package: talloc>=2.2.0 - found
===>   samba410-4.10.14 depends on package: tevent>=0.10.0 - found
===>   samba410-4.10.14 depends on package: tdb>=1.4.0 - found
===>   samba410-4.10.14 depends on package: lmdb>=0.9.16 - found
===>   samba410-4.10.14 depends on package: py37-iso8601>=0.1.11 - found
===>   samba410-4.10.14 depends on file: /usr/local/bin/python3.7 - found


If it's still not fixed... Maybe you can find the solution in your setup, as I, seems, can't reproduce it.

Potential fix should be placing PYTHONDONTWRITEBYTECODE=1 in the environment on the appropriate stage(build, install, test).

At the moment I don't see where to look :(
Comment 6 Matthias Andree freebsd_committer freebsd_triage 2020-09-20 20:15:40 UTC
I also see this issue. Timur, have you tried poudriere bulk -t net/samba410? Do you have special make.conf options, or nondefault Python 3.7 configuration?
Comment 7 Matthias Andree freebsd_committer freebsd_triage 2020-09-20 20:21:13 UTC
I think this is also an indication that the tevent module was used at run-time during the build.  Possibly we have differences in privileges of the build, or paths, or it's an indication that the build might want to use

-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x

(i. e. run ${PYTHON_CMD} -B ..., or export PYTHONDONTWRITEBYTECODE=x to the environment)
Comment 8 Timur I. Bakeyev freebsd_committer freebsd_triage 2020-09-20 23:52:25 UTC
(In reply to Matthias Andree from comment #7)

Hi, Matthias!

I'm pretty sure that tevent gets called at some stage of the build that leads to .pyc file. But I, seems, can't find when it happend.

Indeed, my setup is not standard - I use host system /usr/ports inside poudriere.

Also, at the moment it is set to use Python 3.8(for the test purposes), but the same I couldn't reproduce the problem with 3.6 either.

Port now has:

# Don't cache Python modules
CONFIGURE_ARGS+=                --nopycache
MAKE_ENV+=                      PYTHONDONTWRITEBYTECODE=1
TEST_ENV+=                      PYTHON="${PYTHON_CMD}" \
                                SHA1SUM=/sbin/sha1 \
                                SHA256SUM=/sbin/sha256 \
                                MD5SUM=/sbin/md5 \
                                PYTHONDONTWRITEBYTECODE=1

And still produces bytecode.

I was wondering, would it be sufficient just to `rm -f` the .pyc file in the post-install section or that still would indicate the FS violation?
Comment 9 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-10-15 19:24:02 UTC
Samba 4.12 is the new default and here's the result of poudriere with "-t" flag.

=>> Checking for filesystem violations... done
=>> Error: Filesystem touched during build:
extra: usr/local/lib/python3.7/site-packages/__pycache__
=>> Cleaning up wrkdir
===>  Cleaning for samba412-4.12.8
build of net/samba412 | samba412-4.12.8 ended at Fri Oct 16 03:05:32 CST 2020
build time: 00:25:01
!!! build failure encountered !!!

(In reply to Timur I. Bakeyev from comment #8)

While building tevent, can you generate the .pyc files and add them to the PLIST?
It should be similar with ports r550817.
Thanks.
Comment 10 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-11-24 20:59:06 UTC
Hi Timur,

Please try this patch [1]. It fixes the filesystem violations which building samba412. I guess it should also work with other samba versions.

[1] https://people.FreeBSD.org/~sunpoet/patch/devel-tevent.txt
Comment 11 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-12-19 22:15:30 UTC
Committed. Thanks!
Comment 12 commit-hook freebsd_committer freebsd_triage 2020-12-19 22:15:58 UTC
A commit references this bug:

Author: sunpoet
Date: Sat Dec 19 22:14:49 UTC 2020
New revision: 558701
URL: https://svnweb.freebsd.org/changeset/ports/558701

Log:
  Fix poudriere build_fs_violation for net/samba*

  - Bump PORTREVISION for package change

  The tevent module is being used during build of net/samba*, therefore the
  bytecode .pyc was generated. When running poudriere with -t flag, it checks
  filesystem violations and raises the tevent module was used error. The fix is to
  generate the bytecode and package it.

  PR:		243331
  Submitted by:	sunpoet (myself)
  Approved by:	maintainer (timeout, 25 days)

Changes:
  head/devel/tevent/Makefile