Hi!

It would appear that - for reasons I can't quite fathom at the moment - your synapse installation is missing critical files, but it's hard to pinpoint the precise issue from the error message itself. Can you perhaps provide more information on your setup/environment?

From the error itself it would appear as if 

/usr/local/lib/python3.8/site-packages/synapse/app/homeserver.py

(or the entire app folder) is absent from your package. Does that file exist on your system? Does it appear in the output of `pkg info -l py38-matrix-synapse`?

Did you build that package from ports? Are you perhaps mixing python versions somewhere? Does a simple invocation like `python3.8 -m synapse.app.homeserver` in a shell work (it should print a usage message and complain about missing config paths at the very least).

If you'd like to debug this together (and are able to roll back to a working synapse version) you can also poke me in matrix via @karasu:skyforge.at. :)

Cheers,
Sascha

Hi!

Thanks for the fast response.

I did install the packages with postmaster, but a clean install of the port leads to the same result: All packages within `synapse` are missing.

$ ls /usr/local/lib/python3.8/site-packages/synapse 
__init__.py   __pycache__   event_auth.py notifier.py   py.typed      server.py     types.py      visibility.py
$ pkg info -l py38-matrix-synapse-1.59.1 
py38-matrix-synapse-1.59.1:
        /usr/local/bin/export_signing_key
        /usr/local/bin/generate_config
        /usr/local/bin/generate_log_config
        /usr/local/bin/generate_signing_key
        /usr/local/bin/hash_password
        /usr/local/bin/register_new_matrix_user
        /usr/local/bin/synapse_homeserver
        /usr/local/bin/synapse_port_db
        /usr/local/bin/synapse_review_recent_signups
        /usr/local/bin/synapse_worker
        /usr/local/bin/synctl
        /usr/local/bin/update_synapse_database
        /usr/local/etc/matrix-synapse/log.config.sample
        /usr/local/etc/rc.d/synapse
        /usr/local/lib/python3.8/site-packages/matrix_synapse-1.59.1-py3.8.egg-info/PKG-INFO
        /usr/local/lib/python3.8/site-packages/matrix_synapse-1.59.1-py3.8.egg-info/SOURCES.txt
        /usr/local/lib/python3.8/site-packages/matrix_synapse-1.59.1-py3.8.egg-info/dependency_links.txt
        /usr/local/lib/python3.8/site-packages/matrix_synapse-1.59.1-py3.8.egg-info/entry_points.txt
        /usr/local/lib/python3.8/site-packages/matrix_synapse-1.59.1-py3.8.egg-info/requires.txt
        /usr/local/lib/python3.8/site-packages/matrix_synapse-1.59.1-py3.8.egg-info/top_level.txt
        /usr/local/lib/python3.8/site-packages/synapse/__init__.py
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/__init__.cpython-38.opt-1.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/__init__.cpython-38.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/event_auth.cpython-38.opt-1.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/event_auth.cpython-38.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/notifier.cpython-38.opt-1.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/notifier.cpython-38.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/server.cpython-38.opt-1.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/server.cpython-38.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/types.cpython-38.opt-1.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/types.cpython-38.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/visibility.cpython-38.opt-1.pyc
        /usr/local/lib/python3.8/site-packages/synapse/__pycache__/visibility.cpython-38.pyc
        /usr/local/lib/python3.8/site-packages/synapse/event_auth.py
        /usr/local/lib/python3.8/site-packages/synapse/notifier.py
        /usr/local/lib/python3.8/site-packages/synapse/py.typed
        /usr/local/lib/python3.8/site-packages/synapse/server.py
        /usr/local/lib/python3.8/site-packages/synapse/types.py
        /usr/local/lib/python3.8/site-packages/synapse/visibility.py
        /usr/local/share/licenses/py38-matrix-synapse-1.59.1/APACHE20
        /usr/local/share/licenses/py38-matrix-synapse-1.59.1/LICENSE
        /usr/local/share/licenses/py38-matrix-synapse-1.59.1/catalog.mk
$ python3.8 -m synapse.app.homeserver                             
/usr/local/bin/python3.8: Error while finding module specification for 'synapse.app.homeserver' (ModuleNotFoundError: No module named 'synapse.app')

The only Python version installed is 3.8.13.

For now, I installed a backup package of version 1.55.2 to keep the server running.

Regards,
Tobias

After looking into it some more I noticed that the problem is that poetry2setup doesn’t work correctly when run as root. Then, the list of packages in the generated setup.py file only contains the `synapse` package. If I build the package as user all files/python packages are there. So maybe this is a bug in devel/py-poetry2setup and not net-im/py-matrix-synapse.

(In reply to Tobias Bölz from comment #3)

Sorry for the delay, I've been a bit busy and hadn't had as much time to devote to this as I would have liked. I've been trying to reproduce this issue, but even with portmaster things build fine on my testbuilder. I've tried

* building the port as root using make/make package
* building the port using portmaster as root using `portmaster net-im/py-matrix-synapse`.
* building the port using poudriere

all of which yield a perfectly fine port for me. I'm sorry for the trouble, but can you perhaps detail a way to reproduce your issue (ideally starting from a clean jail or something similar)?

(In reply to Sascha Biberhofer from comment #4)

I just tested it on a fresh installation of FreeBSD 13.1 in a VM and there the problem exists, too. So the steps to reproduce would be:

1. Install FreeBSD
2. Log in as root
3. Install git-tiny package and clone ports to /usr/ports
4. `cd /usr/ports/net-im/py-matrix-synapse` and run `make install`

I have got the same problem, downgraded to 1.57.1.

I'm a bit at a loss here as I still can't reproduce this locally. 

I've also tried to trigger this issue as described by Tobias, cloning the ports tree with git (and git-tiny for good measure) in a clean jail and build the port as root, all of which works just fine on my systems.

I'd be happy to look into this some more, but I feel like my hands are tied without the ability to reproduce the behaviour on my systems. We could try and switch the port to a pep 517 build, but this would require a lot of manual changes as the ports framework doesn't support these builds ootb yet and I'm not even sure that the setup.py workaround is entirely at fault here.

(In reply to Sascha Biberhofer from comment #7)

I could send you a disk image of my VM.

Is it somehow possible to make the port use the source distribution from PyPI[1] or the GitHub Releases page[2]? These already contain a generated setup.py file.

[1] https://pypi.org/project/matrix-synapse/#files
[2] https://github.com/matrix-org/synapse/releases

Version 1.61.0 is affected, too.

Comment 10 commit-hook 2022-06-29 04:30:14 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=c4501a503083e9de1d8cfd936cfb244642884434

commit c4501a503083e9de1d8cfd936cfb244642884434
Author:     Sascha Biberhofer <sascha.biberhofer@skyforge.at>
AuthorDate: 2022-06-28 19:31:14 +0000
Commit:     Ashish SHUKLA <ashish@FreeBSD.org>
CommitDate: 2022-06-29 04:14:17 +0000

    net-im/py-matrix-synapse: Update to 1.61.1

    This minor update contains a security fix for GHSA-22p3-qrh9-cx32 /
    CVE-2022-31052.

    While we're at it, switch to pypi as package source. This not only
    follows general python port guidelines but also eliminates the
    need for poetry2setup, hopefully fixing #264178[0].

    Changes: https://github.com/matrix-org/synapse/releases/tag/v1.61.1

    Signed-off-by: Sascha Biberhofer <sascha.biberhofer@skyforge.at>
    PR:             264178
    MFH:            2022Q2
    Security:       07c0d782-f758-11ec-acaa-901b0e9408dc
    Security:       CVE-2022-31052

 net-im/py-matrix-synapse/Makefile                   | 13 +++----------
 net-im/py-matrix-synapse/distinfo                   |  6 +++---
 net-im/py-matrix-synapse/files/patch-pyproject.toml |  6 +++---
 net-im/py-matrix-synapse/files/patch-setup.py (new) | 11 +++++++++++
 4 files changed, 20 insertions(+), 16 deletions(-)

Comment 11 commit-hook 2022-06-29 04:30:16 UTC

A commit in branch 2022Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=46b9a76d82a03461de14551e9cdd89ce784ce409

commit 46b9a76d82a03461de14551e9cdd89ce784ce409
Author:     Sascha Biberhofer <sascha.biberhofer@skyforge.at>
AuthorDate: 2022-06-28 19:31:14 +0000
Commit:     Ashish SHUKLA <ashish@FreeBSD.org>
CommitDate: 2022-06-29 04:14:56 +0000

    net-im/py-matrix-synapse: Update to 1.61.1

    This minor update contains a security fix for GHSA-22p3-qrh9-cx32 /
    CVE-2022-31052.

    While we're at it, switch to pypi as package source. This not only
    follows general python port guidelines but also eliminates the
    need for poetry2setup, hopefully fixing #264178[0].

    Changes: https://github.com/matrix-org/synapse/releases/tag/v1.61.1

    Signed-off-by: Sascha Biberhofer <sascha.biberhofer@skyforge.at>
    PR:             264178
    MFH:            2022Q2
    Security:       07c0d782-f758-11ec-acaa-901b0e9408dc
    Security:       CVE-2022-31052
    (cherry picked from commit c4501a503083e9de1d8cfd936cfb244642884434)

 net-im/py-matrix-synapse/Makefile                   | 13 +++----------
 net-im/py-matrix-synapse/distinfo                   |  6 +++---
 net-im/py-matrix-synapse/files/patch-pyproject.toml |  6 +++---
 net-im/py-matrix-synapse/files/patch-setup.py (new) | 11 +++++++++++
 4 files changed, 20 insertions(+), 16 deletions(-)

Comment 12 Ashish SHUKLA 2022-06-29 16:03:17 UTC

(In reply to Tobias Bölz from comment #9)
(In reply to Tomasz Sowa from comment #6)


Could you folks please test if this works for you now ?

Thanks!

I had same issue and it works for me, thanks!

(In reply to Ashish SHUKLA from comment #12)

Yes, 1.61.1 works for me as well.

Awesome! :D 

The build has been switched to rely on the pypi package as Tobias suggested, since it's closer to the ports policy for python ports and I haven't had the time to really dig into this issue yet. We now need to fix our py-cryptography dependency in two places, but it's probably for the better and removes the need to run poetry2setup. With any luck, this will continue to work until support for PEP 517 has landed in the ports python framework.

I'll still try and dig into the VM provided by Tobias (thank you!), since we might face similar problems with other ports, but I'm happy that we've at least got a port that appears to be working for everbody now. :)

Comment 16 Ashish SHUKLA 2022-06-29 17:40:58 UTC

Thanks for the confirmation. Marking as resolved.