Created attachment 257213 [details]
searxng-20250203.diff

Description:

ChangeLog: https://github.com/searxng/searxng/commits/4ab7984edde27b36b158f678d83b993ad4126619

QA:

* portlint: OK (looks fine.)
* testport: (poudriere: 14.2-RELEASE, amd64):
  - Warning: Symlink '/usr/local/lib/python3.11/site-packages/searx/settings.yml' pointing to '/usr/local/etc/searxng.yml' which does not exist 
in the stage directory or in localbase

Notes:

1.- SearxNG is broken at runtime because it uses an older version of HTTPX [1] which is incompatible with the newer version. I have patched this port to use the latest version of HTTPX, however there are some deprecation warnings showing up in the log file:

[1] https://github.com/searxng/searxng/blob/master/requirements.txt#L10

Initial log:

```
2025-02-04 03:17:02,864 ERROR:searx: git is not found, fallback to the default version
2025-02-04 03:17:45,015 WARNING:searx.botdetection.config: missing config file: /usr/local/etc/limiter.toml
 * Serving Flask app 'searx.webapp'
 * Debug mode: off
2025-02-04 03:17:48,107 ERROR:searx.engines.wikidata: Fail to initialize
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 96, in request
    return future.result(timeout)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/abstract.py", line 73, in initialize
    self.engine.init(get_engine_from_settings(self.engine_name))
  File "/usr/local/lib/python3.11/site-packages/searx/engines/wikidata.py", line 811, in init
    jsonresponse = send_wikidata_query(query)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/engines/wikidata.py", line 164, in send_wikidata_query
    http_response = get(SPARQL_ENDPOINT_URL + '?' + urlencode({'query': query}), headers=get_headers())
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 165, in get
    return request('get', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 98, in request
    raise httpx.TimeoutException('Timeout', request=None) from e
httpx.TimeoutException: Timeout
2025-02-04 03:17:48,127 ERROR:searx.engines.soundcloud: Fail to initialize
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 96, in request
    return future.result(timeout)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/abstract.py", line 73, in initialize
    self.engine.init(get_engine_from_settings(self.engine_name))
  File "/usr/local/lib/python3.11/site-packages/searx/engines/soundcloud.py", line 69, in init
    guest_client_id = get_client_id()
                      ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/engines/soundcloud.py", line 57, in get_client_id
    resp = http_get(app_js_url)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 165, in get
    return request('get', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 98, in request
    raise httpx.TimeoutException('Timeout', request=None) from e
httpx.TimeoutException: Timeout
```

After searching:

```
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/dateutil/parser/_parser.py:1207: UnknownTimezoneWarning: tzname K identified but not understood.  Pass `tzinfos` argument in order to correctly return a timezone-aware datetime.  In a future version, this will raise an exception.
  warnings.warn("tzname {tzname} identified but not understood.  "
2025-02-04 03:19:18,508 WARNING:searx.engines.qwant: ErrorContext('searx/engines/qwant.py', 196, 'raise SearxEngineCaptchaException()', 'searx.exceptions.SearxEngineCaptchaException', None, ('CAPTCHA',)) False
2025-02-04 03:19:18,509 ERROR:searx.engines.qwant: CAPTCHA
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 160, in search
    search_results = self._search_basic(query, params)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 148, in _search_basic
    return self.engine.response(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/engines/qwant.py", line 156, in response
    return parse_web_api(resp)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/engines/qwant.py", line 196, in parse_web_api
    raise SearxEngineCaptchaException()
searx.exceptions.SearxEngineCaptchaException: CAPTCHA, suspended_time=86400
```

Searching in images:

```
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/httpx/_models.py:408: DeprecationWarning: Use 'content=<...>' to upload raw bytes/text content.
  headers, stream = encode_request(
2025-02-04 03:20:14,766 ERROR:searx.engines.pinterest: engine timeout
2025-02-04 03:20:14,767 ERROR:searx.engines.brave.images: engine timeout
2025-02-04 03:20:14,939 ERROR:searx: call to ResultContainer.add_unresponsive_engine after ResultContainer.close
2025-02-04 03:20:14,972 ERROR:searx: call to ResultContainer.add_unresponsive_engine after ResultContainer.close
2025-02-04 03:20:14,975 WARNING:searx.engines.pinterest: ErrorContext('searx/search/processors/online.py', 116, "response = req(params['url'], **request_args)", 'httpx.TimeoutException', None, (None, None, None)) False
2025-02-04 03:20:14,975 ERROR:searx.engines.pinterest: HTTP requests timeout (search duration : 4.210507460986264 s, timeout: 4.0 s) : TimeoutException
2025-02-04 03:20:15,122 WARNING:searx.engines.brave.images: ErrorContext('searx/engines/brave.py', 259, 'json_data = js_variable_to_python(datastr)', 'json.decoder.JSONDecodeError', None, ('Expecting value',)) False
2025-02-04 03:20:15,123 ERROR:searx.engines.brave.images: exception : Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 160, in search
    search_results = self._search_basic(query, params)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 148, in _search_basic
    return self.engine.response(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/engines/brave.py", line 259, in response
    json_data = js_variable_to_python(datastr)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/utils.py", line 802, in js_variable_to_python
    return json.loads(s)
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

Searching in videos:

```
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
/usr/local/lib/python3.11/site-packages/searx/network/network.py:276: DeprecationWarning: Setting per-request cookies=<...> is being deprecated, because the expected behaviour on cookie persistence is ambiguous. Set cookies directly on the client instance instead.
  response = await client.request(method, url, **kwargs)
2025-02-04 03:21:01,283 WARNING:searx.network.vimeo: HTTP Request failed: GET https://vimeo.com/search/page:1?q=FreeBSD
2025-02-04 03:21:01,309 WARNING:searx.engines.vimeo: ErrorContext('searx/search/processors/online.py', 116, "response = req(params['url'], **request_args)", 'searx.exceptions.SearxEngineAccessDeniedException', None, ('HTTP error 403',)) False
2025-02-04 03:21:01,310 ERROR:searx.engines.vimeo: SearXNG is blocked
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 160, in search
    search_results = self._search_basic(query, params)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 144, in _search_basic
    response = self._send_http_request(params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/search/processors/online.py", line 116, in _send_http_request
    response = req(params['url'], **request_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 165, in get
    return request('get', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/__init__.py", line 96, in request
    return future.result(timeout)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.11/site-packages/searx/network/network.py", line 295, in request
    return await self.call_client(False, method, url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/network.py", line 278, in call_client
    return self.patch_response(response, do_raise_for_httperror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/searx/network/network.py", line 249, in patch_response
    raise_for_httperror(response)
  File "/usr/local/lib/python3.11/site-packages/searx/network/raise_for_httperror.py", line 74, in raise_for_httperror
    raise SearxEngineAccessDeniedException(message='HTTP error ' + str(resp.status_code))
searx.exceptions.SearxEngineAccessDeniedException: HTTP error 403, suspended_time=86400
2025-02-04 03:21:02,497 ERROR:searx.engines.brave.videos: engine timeout
2025-02-04 03:21:02,701 ERROR:searx: call to ResultContainer.add_unresponsive_engine after ResultContainer.close
2025-02-04 03:21:02,731 WARNING:searx.engines.brave.videos: ErrorContext('searx/search/processors/online.py', 116, "response = req(params['url'], **request_args)", 'httpx.TimeoutException', None, (None, None, None)) False
2025-02-04 03:21:02,731 ERROR:searx.engines.brave.videos: HTTP requests timeout (search duration : 3.2360264089656994 s, timeout: 3.0 s) : TimeoutException
```

2.- Issues related:
  - https://github.com/searxng/searxng/discussions/4219
  - https://github.com/DtxdF/searxng-port/issues/1