Bug 276766 - net-im/py-matrix-synapse: Update to 1.105.0
Summary: net-im/py-matrix-synapse: Update to 1.105.0
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Ashish SHUKLA
URL: https://github.com/element-hq/synapse...
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-01 14:16 UTC by Ashish SHUKLA
Modified: 2024-04-20 13:16 UTC (History)
2 users (show)

See Also:
ashish: maintainer-feedback+


Attachments
Update diff (9.93 KB, patch)
2024-02-01 14:16 UTC, Ashish SHUKLA
no flags Details | Diff
Update diff v2 (11.53 KB, patch)
2024-03-13 21:40 UTC, Ashish SHUKLA
no flags Details | Diff
update to 1.104.0 patch (12.46 KB, patch)
2024-04-13 10:48 UTC, Marko Cupać
no flags Details | Diff
Update diff v3 (13.30 KB, patch)
2024-04-16 18:20 UTC, Ashish SHUKLA
ashish: maintainer-approval?
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ashish SHUKLA freebsd_committer freebsd_triage 2024-02-01 14:16:46 UTC
Created attachment 248115 [details]
Update diff

- Switch LICENSE to AGPLv3
Comment 1 Ashish SHUKLA freebsd_committer freebsd_triage 2024-03-13 21:40:53 UTC
Created attachment 249137 [details]
Update diff v2

- Update to 102.0
- Update dependency versions to match what is in ports tree
Comment 2 Ashish SHUKLA freebsd_committer freebsd_triage 2024-03-14 13:45:19 UTC
Excerpt from the "make test" run:

...
...
===============================================================================
[SKIPPED]
Requires PyICU

tests.storage.test_user_directory.UserDirectoryICUTestCase.test_icu_word_boundary
tests.storage.test_user_directory.UserDirectoryICUTestCase.test_icu_word_boundary_punctuation
tests.storage.test_user_directory.UserDirectoryICUTestCase.test_regex_word_boundary_punctuation
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_all_users
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_ascii_case_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_start_of_user_id
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_stop_words
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_unicode_case_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_unicode_normalization
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_limit_correct
===============================================================================
[SKIPPED]
not supported yet

tests.storage.test_user_directory.UserDirectoryStoreTestCase.test_search_user_dir_accent_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_accent_insensitivity
===============================================================================
[SKIPPED]
not supported

tests.storage.test_user_directory.UserDirectoryStoreTestCase.test_search_user_dir_dotted_dotless_i_case_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_dotted_dotless_i_case_insensitivity
-------------------------------------------------------------------------------
Ran 3084 tests in 141.723s

PASSED (skips=174, successes=2910)
Comment 3 Marko Cupać 2024-04-04 08:40:33 UTC
(In reply to Ashish SHUKLA from comment #2)

I just gave it a shot to to upgrade port to 1.104.0, it built and installed fine, I don't see any problems running it on 14.0-RELEASE x64.

Changes to the port were trivial (bumping version, removing portrevision, changing license,  recreating Makefile.crates and distinfo, removing files/patch-pyproject.toml).

I can send diffs if it would speed up upgrading the port.
Comment 4 Ashish SHUKLA freebsd_committer freebsd_triage 2024-04-04 12:23:43 UTC
(In reply to Marko Cupać from comment #3)

Definitely it would be of great assistance. Please just make sure the dependency constraints in Makefile are in line with upstream, or include a patch.

And also, if it's the output of `git format-patch ...' based off HEAD that would ease applying it.

Thanks!
Comment 5 Marko Cupać 2024-04-13 10:48:43 UTC
Created attachment 249948 [details]
update to 1.104.0 patch
Comment 6 Marko Cupać 2024-04-13 11:00:39 UTC
I created a patch, I have tested build in poudriere in 14.0 amd64 jail. It built and installed fine. My synapse already runs on this port:

me@mybox:~ % pkg info -x synapse
py39-matrix-synapse-1.104.0

me@mybox:~ % cat /var/log/matrix-synapse/homeserver.log
2024-04-13 12:33:50,866 - root - 355 - WARNING - main- ***** STARTING SERVER *****
2024-04-13 12:33:50,867 - root - 356 - WARNING - main- Server /usr/local/lib/python3.9/site-packages/synapse/app/homeserver.py version 1.104.0
2024-04-13 12:33:50,867 - root - 361 - WARNING - main- Copyright (c) 2023 New Vector, Inc
2024-04-13 12:33:50,867 - root - 362 - WARNING - main- Licensed under the AGPL 3.0 license. Website: https://github.com/element-hq/synapse
Comment 7 Ashish SHUKLA freebsd_committer freebsd_triage 2024-04-16 18:20:35 UTC
Created attachment 250009 [details]
Update diff v3

Update to v1.105.0
Comment 8 Ashish SHUKLA freebsd_committer freebsd_triage 2024-04-16 20:25:25 UTC
Test Suite also passes:

---------------------------------------------------------------------------------

===============================================================================
[SKIPPED]
Test only applies when postgres is used as the database

tests.storage.test_room_search.MessageSearchTest.test_postgres_web_search_for_phrase
===============================================================================
[SKIPPED]
Requires PyICU

tests.storage.test_user_directory.UserDirectoryICUTestCase.test_icu_word_boundary
tests.storage.test_user_directory.UserDirectoryICUTestCase.test_icu_word_boundary_punctuation
tests.storage.test_user_directory.UserDirectoryICUTestCase.test_regex_word_boundary_punctuation
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_all_users
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_ascii_case_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_start_of_user_id
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_stop_words
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_unicode_case_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_unicode_normalization
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_limit_correct
===============================================================================
[SKIPPED]
not supported yet

tests.storage.test_user_directory.UserDirectoryStoreTestCase.test_search_user_dir_accent_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_accent_insensitivity
===============================================================================
[SKIPPED]
not supported

tests.storage.test_user_directory.UserDirectoryStoreTestCase.test_search_user_dir_dotted_dotless_i_case_insensitivity
tests.storage.test_user_directory.UserDirectoryStoreTestCaseWithIcu.test_search_user_dir_dotted_dotless_i_case_insensitivity
-------------------------------------------------------------------------------
Ran 3099 tests in 137.160s

PASSED (skips=175, successes=2924)


---------------------------------------------------------------------------------

Spoke to Sascha (maintainer) in private, and he's extremely busy to review, and has given his approval to commit, if all things look good. I'm running 1.105.0 and have no issues in the past few hours. I'll wait for 2-3 days, if I don't hear anything negative, I'll commit the update.

Right now in my rc.conf(5), I have:

   synapse_env=CRYPTOGRAPHY_OPENSSL_NO_LEGACY=yes

This is needed for running with OpenSSL 3.x. I guess we could add an UPDATING, and pkg-message entry documenting this requirement, if users are running it in OpenSSL 3.x environment, and when FreeBSD 13.x is min(EOL'ed, OpenSSL 3.x makes it to the base of FreeBSD 13), then we can add it to the rc.d script.

What do you all say to this ?

Thanks!
Comment 9 Marko Cupać 2024-04-17 08:28:55 UTC
(In reply to Ashish SHUKLA from comment #8)

I say thanks :)

I didn't know about the need for env record in rc.conf in order to run proper cryptography. Perhaps that has something to do with "tlsv1 alert protocol version":
https://github.com/element-hq/synapse/issues/17046

I'm still getting those on my 1.104.0:

2024-04-17 10:20:27,256 - synapse.http.federation.well_known_resolver - 291 - INFO - _maybe_retry_device_resync-21- Error fetching https://example.org/.well-known/matrix/server: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'tlsv1 alert protocol version')]>]

Is rc.conf env record something new to 1.105.0?

I would go for setting proper cryptography by default, and override to legacy if needed, not vice versa. 13.x EoL is at least two years away.
Comment 10 Ashish SHUKLA freebsd_committer freebsd_triage 2024-04-17 15:12:26 UTC
(In reply to Marko Cupać from comment #9)

Well, if you're running OpenSSL 3.x only then you'll need that environment variable. If you're still on OpenSSL 1.1.x then you don't need it.

FreeBSD 13.x has OpenSSL 1.1 in its base.
Comment 11 Marko Cupać 2024-04-18 09:20:20 UTC
(In reply to Ashish SHUKLA from comment #10)

My host is 14.0-RELEASE-p6:

me@jailhost:~ % sudo freebsd-version -kru
14.0-RELEASE-p6
14.0-RELEASE-p6
14.0-RELEASE-p6

jail as well:

me@jailhost:~ % sudo freebsd-version -j synapse
14.0-RELEASE-p6

openssl version on both:

me@[jailhost|synapse]:~ % openssl version
OpenSSL 3.0.12 24 Oct 2023 (Library: OpenSSL 3.0.12 24 Oct 2023)

I don't see any difference regarding existence of 'synapse_env=CRYPTOGRAPHY_OPENSSL_NO_LEGACY=yes' in either /etc/rc.conf or /etc/rc.conf.d/synapse.

Best regards,
Comment 12 Ashish SHUKLA freebsd_committer freebsd_triage 2024-04-18 22:22:46 UTC
(In reply to Marko Cupać from comment #11)

If I comment that "synapse_env=..." in my rc.conf(5), I get:

# service synapse restart
Stopping synapse.
Waiting for PIDS: 92011.
Starting synapse.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/local/lib/python3.9/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.9/site-packages/synapse/__init__.py", line 32, in <module>
    from synapse.util.rust import check_rust_lib_up_to_date
  File "/usr/local/lib/python3.9/site-packages/synapse/util/__init__.py", line 38, in <module>
    from synapse.logging import context
  File "/usr/local/lib/python3.9/site-packages/synapse/logging/__init__.py", line 24, in <module>
    from synapse.logging._remote import RemoteHandler
  File "/usr/local/lib/python3.9/site-packages/synapse/logging/_remote.py", line 35, in <module>
    from twisted.internet.endpoints import (
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/endpoints.py", line 69, in <module>
    from OpenSSL.SSL import Error as SSLError
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import SSL, crypto
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/SSL.py", line 9, in <module>
    from OpenSSL._util import (
  File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 15, in <module>
    from cryptography.exceptions import InternalError
  File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py", line 9, in <module>
    from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
RuntimeError: OpenSSL 3.0's legacy provider failed to load. This is a fatal error by default, but cryptography supports running without legacy algorithms by setting the environment variable CRYPTOGRAPHY_OPENSSL_NO_LEGACY. If you did not expect this error, you have likely made a mistake with your OpenSSL configuration.
/usr/local/etc/rc.d/synapse: WARNING: failed to start synapse


Can you share the output of: pkg info -d security/py-cryptography ?

Thanks!
Comment 13 Marko Cupać 2024-04-19 08:31:54 UTC
(In reply to Ashish SHUKLA from comment #12)

me@mybox:~ % pkg info -d security/py-cryptography
py39-cryptography-42.0.5_1,1:
	python39-3.9.18_2
	py39-cffi-1.16.0

My py39-matrix-synapse-1.104.0 built from port I sent patch for seem to run fine regardless of env setting.
Comment 14 Marko Cupać 2024-04-19 08:58:46 UTC
I just upgraded to 1.105.0, your patch applied fine, port built fine in my 14.0_x64 poudriere jail, package installed fine, synapse runs fine without any env records in rc.conf

2024-04-19 10:56:21,895 - root - 355 - WARNING - main- ***** STARTING SERVER *****
2024-04-19 10:56:21,895 - root - 356 - WARNING - main- Server /usr/local/lib/python3.9/site-packages/synapse/app/homeserver.py version 1.105.0
Comment 15 commit-hook freebsd_committer freebsd_triage 2024-04-20 13:15:03 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=a73b9123eb70124ee42329a743a803be9ad064e1

commit a73b9123eb70124ee42329a743a803be9ad064e1
Author:     Ashish SHUKLA <ashish@FreeBSD.org>
AuthorDate: 2024-04-13 12:38:22 +0000
Commit:     Ashish SHUKLA <ashish@FreeBSD.org>
CommitDate: 2024-04-20 13:14:03 +0000

    net-im/py-matrix-synapse: Update to 1.105.0

    - Update LICENSE to AGPLv3

    PR:             276766
    Tested by:      Marko Cupać <marko.cupac at mimar dot rs>
    Approved by:    maintainer timeout
    MFH:            2024Q2 (long due)

 net-im/py-matrix-synapse/Makefile                  |  9 ++-
 net-im/py-matrix-synapse/Makefile.crates           | 33 +++++-----
 net-im/py-matrix-synapse/distinfo                  | 72 +++++++++++-----------
 .../py-matrix-synapse/files/patch-pyproject.toml   | 11 +---
 net-im/py-matrix-synapse/files/patch-setup.py      |  2 +-
 5 files changed, 60 insertions(+), 67 deletions(-)
Comment 16 commit-hook freebsd_committer freebsd_triage 2024-04-20 13:16:06 UTC
A commit in branch 2024Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=3644d195074327f9c8fdf77a4595b6c02cb5c49f

commit 3644d195074327f9c8fdf77a4595b6c02cb5c49f
Author:     Ashish SHUKLA <ashish@FreeBSD.org>
AuthorDate: 2024-04-13 12:38:22 +0000
Commit:     Ashish SHUKLA <ashish@FreeBSD.org>
CommitDate: 2024-04-20 13:14:43 +0000

    net-im/py-matrix-synapse: Update to 1.105.0

    - Update LICENSE to AGPLv3

    PR:             276766
    Tested by:      Marko Cupać <marko.cupac at mimar dot rs>
    Approved by:    maintainer timeout
    MFH:            2024Q2 (long due)

    (cherry picked from commit a73b9123eb70124ee42329a743a803be9ad064e1)

 net-im/py-matrix-synapse/Makefile                  |  9 ++-
 net-im/py-matrix-synapse/Makefile.crates           | 33 +++++-----
 net-im/py-matrix-synapse/distinfo                  | 72 +++++++++++-----------
 .../py-matrix-synapse/files/patch-pyproject.toml   | 11 +---
 net-im/py-matrix-synapse/files/patch-setup.py      |  2 +-
 5 files changed, 60 insertions(+), 67 deletions(-)
Comment 17 Ashish SHUKLA freebsd_committer freebsd_triage 2024-04-20 13:16:39 UTC
Committed.