Bug 257157 - www/nextcloud: After upgrade to 22.0.0.11: Internal Server Error The server was unable to complete your request
Summary: www/nextcloud: After upgrade to 22.0.0.11: Internal Server Error The server w...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Bernard Spil
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-07-13 17:47 UTC by O. Hartmann
Modified: 2021-07-15 11:44 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (brnrd)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description O. Hartmann 2021-07-13 17:47:04 UTC
running www/nextcloud (php74) on FreeBSD 12.2-RELENG-p9. System is mainatined with official FreeBSD packages (nothing home-brewn).

After the recent upgrade of the ports (including php74, Apache and Nextcloud (from21.X.X to 22.0.0), the nextcloud service is corrupted: the Apache server reports the request could not served, contact administrator. Checking the log /var/log/httpd-error.log when requesting I see

[...]
[Tue Jul 13 17:22:15.847179 2021] [authz_core:error] [pid 71606] [client xxx.xxx.xxx.xxx:48910] AH01630: client denied by server configuration: /usr/local/www/apache24/data/csrftoken
[Tue Jul 13 17:22:16.110620 2021] [authz_core:error] [pid 66249] [client xxx.xxx.xxx.xxx:11753] AH01630: client denied by server configuration: /usr/local/www/apache24/data/login
[...]

and /var/log/httpd-access.log

[...]
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:41 +0000] "GET /cloud/index.php/login HTTP/1.1" 500 16855
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/js/dist/files_fileinfo.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/files_sharing/l10n/de_DE.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/l10n/de_DE.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/js/dist/files_client.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/js/dist/main.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps/apporder/l10n/de_DE.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/files_sharing/js/dist/main.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps/apporder/js/apporder.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps/epubreader/l10n/de_DE.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps/epubreader/js/plugin.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/files_videoplayer/js/main.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/files_rightclick/l10n/de_DE.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/files_rightclick/js/script.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/files_rightclick/js/files.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/theming/l10n/de_DE.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/apps-pkg/theming/js/theming.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/js/dist/login.js?v=1db512a4-5 HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/img/loading-dark.gif HTTP/1.1" 304 -
xxx.xxx.xxx.xxx - - [13/Jul/2021:17:43:43 +0000] "GET /cloud/core/img/actions/toggle.svg HTTP/1.1" 304 -
[...]


It looks like the documentroot is some kind of weird, since the nextcloud's document root is located not in /usr/local/www/apache24/data!

Anyway, when upgrading nextcloud via

su -m www -c 'php ./occ upgrade'

nothing unusual has been observed except the fact that when it came to updating apps, the connection to the app store seemed to have timed out, there was a further unnoticed error of the php script and I simply restarted the procedure as shown above and then it went through all steps. After that, restarting Apache, the server quit the request with not servicing nextcloud content. I checked the log and revealed the above and issued the command:

root@websrv:/usr/local/www/nextcloud # su -m www -c "php ./occ maintenance:update:htaccess"
Error updating .htaccess file, not enough permissions or "overwrite.cli.url" set to an invalid URL?

I searched the web for this error and found that one has to set overwrite.cli.url to the URL in config/config.php, but this object has never been set before and even setting it either to the local (IP based) or fully qualified URL doe not solve anything. Further, some tell that the access rights need to be adjusted to the owner of the running apache (www:www) but on which .htaccess file and where is a big mystery of internet sloppyness. Most of them are not usefull.


The access rights of .htaccess file located in the data folder for nextcloud is www:www and untouched, also the .htaccess file located in /usr/local/www/nextcloud/config.

/usr/local/etc/apache24/httpd.conf is untouched, so the (prior to the update working) .conf file in  /usr/local/etc/apache24/Includes.


Now, with this next painin the arse, I'mfloating like a deadman in the water with nextcloud on freeBSD :-(
Comment 1 Thibault Payet 2021-07-13 17:52:57 UTC
I think this is due to the auto plist
 * Auto-generate plist

Normally nextcloud install with user www, but now it is with root
Comment 2 Bernard Spil freebsd_committer freebsd_triage 2021-07-13 18:28:45 UTC
Hi,

This is not a result of the plist changes, it has never been writable for WWWOWN or WWWGROUP (see e.g. 2021Q2 branch). What changes in that no .htaccess is supplied by default.

> @(,%%NEXTCLOUD_GROUPNAME%%,644) %%WWWDIR%%/.htaccess

The nextcloud directory is also not writable, trying to create .htaccess will therefore fail. Apart from the apps, config and data dirs are also writable, and have been from before I took maintainership.

11.0.1 in 2017:
https://github.com/freebsd/freebsd-ports/blob/3c92fae2de5ab542780f9b2b466fa58855301ab0/www/nextcloud/pkg-plist#L1
https://github.com/freebsd/freebsd-ports/blob/3c92fae2de5ab542780f9b2b466fa58855301ab0/www/nextcloud/pkg-plist#L10787

The whole purpose of the Nextcloud port in FreeBSD is that your webserver (or FastCGI/PHP binaries) should NOT have write-access to anything it serves. Any write access is thus considered a security issue. I revoke write access to htaccess and config on my local install, and allow write only when it's required, data is stored outside of WWWDIR.

If you have no issue with these things being writable, I encourage you to download the tarball, unpack it into /usr/local/www/nextcloud and `chown -R www` the nextcloud dir. This will also enable the bundled app install and updater to work as expected.

With kind regards, Bernard.
Comment 3 O. Hartmann 2021-07-13 19:16:04 UTC
(In reply to Bernard Spil from comment #2)


I beg your pardon, please? Why are you closing this PR? This is not about an ACL that is mssing, not set or something similar, it is about an upgrade from a former, WORKING! installation towards a regular FreeBSD-pkg based update that wrecks an installation!
If this is "works as intended/expected", then I'd like to feel free to call it sabotage.
Comment 4 O. Hartmann 2021-07-13 19:24:20 UTC
I have no intention to make the port's installation directory writeable as this is considered a serious security issue. The standard is fine for me.

I try to figure out what the differences are between 22.0.0.11 and its predecessor, 21.0.2.1. Something must have changed, otherwise the upgrade wouldn't end up in a nonworking installation and the maintenance script in question could fix the problem - which it does not. So I consider this a bug - not a work as intended.
Comment 5 Bernard Spil freebsd_committer freebsd_triage 2021-07-13 20:29:00 UTC
This probably violates the principal of least astonishment, but is intended. The "upgrade" pkg message reflects this.

> After a version migration you should upgrade your nextcloud instance
> using command line:
>
>   cd %%WWWDIR%%
>   su -m www -c "php ./occ upgrade"
>
> Merge any changes to %%WWWDIR%%/.htaccess.dist into .htaccess (above the
> '#### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####' divider if it exists)
> and update the dynamic part of the file using the commandline:
> 
>  su -m www -c "php ./occ maintenance:update:htaccess"

21.0.3: .htaccess is created, has uid 0, gid WWWGRP, mode 644
22.0.0: .htaccess is no longer created

> su -m www -c "php ./occ maintenance:update:htaccess"

can only have succeeded if in 21.0.3 your .htaccess was either owned by WWWOWN, writable by WWWGRP, or writable for all.

To test, create the .htaccess file with the appropriate permissions as for 21.0.3

> install -o 0 -g 80 -m 644 /dev/null /usr/local/www/nextcloud/.htaccess

then run the maintenance:update:htaccess command again. This should fail on permissions.

This patch restores the old behavior:

diff --git a/www/nextcloud/pkg-plist b/www/nextcloud/pkg-plist
index 91918518542f..7b8a705bcc95 100644
--- a/www/nextcloud/pkg-plist
+++ b/www/nextcloud/pkg-plist
@@ -1,3 +1,3 @@
-@(,%%NEXTCLOUD_GROUPNAME%%,644) %%WWWDIR%%/.htaccess.dist
+@sample(,%%NEXTCLOUD_GROUPNAME%%,644) %%WWWDIR%%/.htaccess.dist %%WWWDIR%%/.htaccess
 @(%%NEXTCLOUD_USERNAME%%,%%NEXTCLOUD_GROUPNAME%%,664) %%WWWDIR%%/.user.ini
 @sample %%WWWDIR%%/config/config.sample.php %%WWWDIR%%/config/config.php
Comment 6 commit-hook freebsd_committer freebsd_triage 2021-07-13 20:34:03 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=b7b8246bac6f27e01b84ae126878611445560737

commit b7b8246bac6f27e01b84ae126878611445560737
Author:     Bernard Spil <brnrd@FreeBSD.org>
AuthorDate: 2021-07-13 20:30:00 +0000
Commit:     Bernard Spil <brnrd@FreeBSD.org>
CommitDate: 2021-07-13 20:30:00 +0000

    www/nextcloud: Make sure there's an .htaccess file at install

    PR:             257157
    Reported by:    O. Hartmann <ohartmann walstatt org>

 www/nextcloud/Makefile  | 1 +
 www/nextcloud/pkg-plist | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
Comment 7 Bernard Spil freebsd_committer freebsd_triage 2021-07-13 20:36:07 UTC
There will be an .htaccess at install now,
and it will not clobber modified .htaccess on uninstall
Comment 8 O. Hartmann 2021-07-13 20:55:14 UTC
(In reply to Bernard Spil from comment #7)

F... mid-air collision ...
Comment 9 O. Hartmann 2021-07-13 21:00:10 UTC
The problem is NOT the missing .htaccess (sorry, I have mistaken the hint of nextcloud's php framework as the source of the problem).

The problem is, after an successful upgrade, a non working nextcloud instance, giving the error:

[...]
Internal Server Error The server was unable to complete your request. If this happens again, please send the technical details below to the server administrator. More details can be found in the server log
[some cryptic hash]
[...]

The log then states for the hashcode:

[...]
Comment 10 O. Hartmann 2021-07-13 21:05:13 UTC
[...]
{"reqId":"sOt2yLAvU7VezWeqnemo","level":3,"time":"2021-07-13T21:01:31+00:00","remoteAddr":"xxx.xxx.xxx.xxx","user":"--","app":"index","method":"GET","url":"/cloud/index.php/login","messa
ge":"Class 'OCP\\User' not found","userAgent":"Mozilla/5.0 (X11; FreeBSD amd64; rv:90.0) Gecko/20100101 Firefox/90.0","version":"22.0.0.11","exception":{"Exception":"Error","Message":"C
lass 'OCP\\User' not found","Code":0,"Trace":[{"file":"/usr/local/www/nextcloud/apps/epubreader/lib/Hooks.php","line":41,"function":"get","class":"OCA\\Epubreader\\Config","type":"::"},
{"file":"/usr/local/www/nextcloud/lib/private/legacy/OC_Hook.php","line":106,"function":"announce_settings","class":"OCA\\Epubreader\\Hooks","type":"::"},{"file":"/usr/local/www/nextclo
ud/lib/private/Template/JSConfigHelper.php","line":323,"function":"emit","class":"OC_Hook","type":"::"},{"file":"/usr/local/www/nextcloud/lib/private/TemplateLayout.php","line":220,"fun
ction":"getConfig","class":"OC\\Template\\JSConfigHelper","type":"->"},{"file":"/usr/local/www/nextcloud/lib/private/legacy/OC_Template.php","line":182,"function":"__construct","class":
"OC\\TemplateLayout","type":"->"},{"file":"/usr/local/www/nextcloud/lib/public/AppFramework/Http/TemplateResponse.php","line":204,"function":"fetchPage","class":"OC_Template","type":"->
"},{"file":"/usr/local/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":171,"function":"render","class":"OCP\\AppFramework\\Http\\TemplateResponse","type":"->"},{"file
":"/usr/local/www/nextcloud/lib/private/AppFramework/App.php","line":156,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/local/www/nextclou
d/lib/private/Route/Router.php","line":301,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/usr/local/www/nextcloud/lib/base.php","line":1000,"function":"match",
"class":"OC\\Route\\Router","type":"->"},{"file":"/usr/local/www/nextcloud/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/usr/local/www/nextcloud/ap
ps/epubreader/lib/config.php","Line":28,"CustomMessage":"--"}}
[...]


Hopefully, this will point to the problem, not that .htaccess incident, which is something I probably messed up with by the headline (changed).
Comment 11 O. Hartmann 2021-07-13 21:29:35 UTC
Problem identified and solved. 

Please revert the revert of the patch for .htaccess, since it is not causing the problem.

The problem has been caused by a rogue app.
Comment 12 Bernard Spil freebsd_committer freebsd_triage 2021-07-14 18:01:15 UTC
Thanks for the confirmation.

As the .htaccess file is not writable by WWWOWN or WWWGRP, I see no need to revert the change. May prevent some issues and the having dist as sample makes some sense.

Cheers, Bernard.
Comment 13 Bernard Spil freebsd_committer freebsd_triage 2021-07-15 11:44:57 UTC
A .htaccess file is required. See https://github.com/nextcloud/server/issues/27931#issuecomment-880338426