Bug 269979 - net/megacmd: webdav server fail on any version after 1.4.1 (was: v1.5.0 broken / core dump during start)
Summary: net/megacmd: webdav server fail on any version after 1.4.1 (was: v1.5.0 broke...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords: crash
Depends on:
Blocks:
 
Reported: 2023-03-05 16:31 UTC by Riccardo Torrini
Modified: 2024-05-24 17:45 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Riccardo Torrini 2023-03-05 16:31:03 UTC
Old port v1.4.1 works, after upgrade to v1.5.0 it crash (make a core) during start of mega-cmd-server

(lldb) bt all
* thread #1, name = 'mega-cmd-server', stop reason = signal SIGSEGV
  * frame #0: 0x0000000800e8ce35
    frame #1: 0x0000000800a74448
  thread #2, name = 'mega-cmd-server', stop reason = signal SIGSEGV
    frame #0: 0x00000008017aa1da

I found only this on the same argument:
https://forums.freebsd.org/threads/cannot-get-megacmd-to-run.86644/
(please note we can't use rclone because mega slows it down, I opened a ticket and they convinced me to stop using rclone, the risk of seeing account banned is a good argument, I pay for my backup storage there)

Being unable to fix it myself I pkg-locked previous version ;)

I also noticed this:
-rw-r--r--  1 root  wheel  2032642 Sep 15  2021 meganz-MEGAcmd-1.4.1_Win_GH0.tar.gz
-rw-r--r--  1 root  wheel  2070287 Mar  5 15:56 meganz-MEGAcmd-1.5.0_Linux_GH0.tar.gz

We switched from Win to Linux version (but doubt this is the crash problem source)

Because Makefile has no fancy patches I ingenuously changed distinfo to fetch new v1.5.1, build obviously fail  0= )

Being probably unfixable can we revert to 1.4.1 and/or split pkg to working/not-working flavor?

--
Riccardo.
Comment 1 Fernando Apesteguía freebsd_committer freebsd_triage 2023-03-06 12:57:25 UTC
It looks to me that the m_state attribute is not properly initialized. I'm not familiar with the code in security/cryptocpp. Pinging maintainer to see if he can have more info.

CryptoPP::(anonymous namespace)::SHA256_HashBlock_CXX (state=0x10, data=data@entry=0x7fffffffd090) at sha.cpp:424
424         memcpy(T, state, sizeof(T));
(gdb) p state
$1 = (CryptoPP::word32 *) 0x10
(gdb) p *state
Cannot access memory at address 0x10
Comment 2 kikadf 2023-04-18 16:28:31 UTC
I fix this segmentation fault myself to build megacmd with add -CRYPTOPP_DISABLE_ASM to CXXFLAGS. You can try with the patch in bug #270916.
Comment 3 Riccardo Torrini 2023-04-19 06:24:20 UTC
My two tests:
- rebuild of 1.5.0 with CXX flag: no more crash
- applied your patch, build of 1.6.1, no crash

But :-(

2023-04-19 08:06:48 DEBUG : data/7f/7f...: Received error: HTTP/1.1 500 Internal Server Error
Connection: close: 500 Internal Server Error - low level retry 9/10

Started a sync yesterday night, nothing transferred, only errors.
This evening I will revert to 1.4.1 and check my account.
Anyway, mega-cmd shell seems to work, I can "df", "cd" and "ls" all my stuff, probably only webdav is broken (and/or get/put files)

Thanks a lot for your effort, let me know if you have some ideas about this.

--
Riccardo.
Comment 4 kikadf 2023-04-19 07:17:12 UTC
(In reply to Riccardo Torrini from comment #3)
I have no idea what problem do you have. For me works fine, the folders synced to my local folder, and the new files synced to the cloud. I didn't have older version of megacmd, the 1.6.1 is the first install for me. Where did you see this error messages?
Comment 5 commit-hook freebsd_committer freebsd_triage 2023-04-20 13:36:21 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=c72cc0314bc0e0fe56f220585c57fb09303aea1d

commit c72cc0314bc0e0fe56f220585c57fb09303aea1d
Author:     kikadf <kikadf.01@gmail.com>
AuthorDate: 2023-04-18 15:51:40 +0000
Commit:     Robert Clausecker <fuz@FreeBSD.org>
CommitDate: 2023-04-20 11:49:23 +0000

    net/megacmd: update to 1.6.1

     - update sdk to 4.17.1a
     - fix segmentation fault with cryptopp-8.7.0:
       add -DCRYPTOPP_DISABLE_ASM to CXXFLAGS on i386 and amd64
     - adopt port

    PR:             269979, 270916

 net/megacmd/Makefile  | 13 +++++++++----
 net/megacmd/distinfo  | 10 +++++-----
 net/megacmd/pkg-plist |  4 ++--
 3 files changed, 16 insertions(+), 11 deletions(-)
Comment 6 Riccardo Torrini 2023-04-20 15:15:51 UTC
(In reply to kikadf from comment #4)

I use restic + rclone serve restic (for local compressed/encrypted repository) + megacmd with webdav (to send local copy to mega)

Errors are from rclone (using mega-cmd webdav[1] as backend), when it checks old files works fine, but when it send a new file it receive a 500.

[1] from mega-cmd webdav --help: Caveat: This functionality is in BETA state. If you experience any issue with this, please contact: support@mega.nz


Reinstalled 1.4.1, started again rclone sync using webdav, it works :-(
I'm on 13.1-p3-p3-p5, I can try on 13.2 in the next few days

Transferred:      124.297 MiB / 5.934 GiB, 2%, 4.505 MiB/s, ETA 22m1s
Checks:             16423 / 16423, 100%
Transferred:           51 / 512, 10%
Elapsed time:        51.5s

[...] DEBUG : pacer: Rate limited, increasing sleep to 640ms

2023/04/20 17:07:59 INFO  : 
Transferred:        6.030 GiB / 6.030 GiB, 100%, 4.956 MiB/s, ETA 0s
Checks:             16423 / 16423, 100%
Transferred:          512 / 512, 100%
Elapsed time:     31m28.6s

2023/04/20 17:07:59 DEBUG : 7 go routines active

And yes, mega is really not the best backend for an offsite backup  0= )
Comment 7 Riccardo Torrini 2023-05-27 13:28:36 UTC
(In reply to Riccardo Torrini from comment #6)

updated to 1.6.3, same error using webdav through rclone.
opened a ticket to mega, I'll keep you informed.
Comment 8 Riccardo Torrini 2023-06-11 10:06:48 UTC
(In reply to Riccardo Torrini from comment #7)

mega answer:

Our developers have reviewed your issue and tried to reproduce the issue.
The issue was replicated in 1.5.0 however it works fine in 1.6.3.


Not too bad. But I still have error 500 during sync, runned rclone with --log-level=DEBUG and --dump headers, updated mega ticket with this (here you can see a modified log for readability)

{date} DEBUG : PROPFIND /{webdav_path}/data/52/ HTTP/1.1
{date} DEBUG : HTTP/1.1 207 Multi-Status
{date} DEBUG : MKCOL /{webdav_path}/data/52/ HTTP/1.1
{date} DEBUG : HTTP/1.1 405 Unknown Error
{date} DEBUG : MKCOL /{webdav_path}/data/52/ HTTP/1.1
{date} DEBUG : HTTP/1.1 405 Unknown Error
{date} DEBUG : PUT /{webdav_path}/data/52/{sha256-file_1} HTTP/1.1
{date} DEBUG : PUT /{webdav_path}/data/52/{sha256-file_2} HTTP/1.1
{date} DEBUG : HTTP/1.1 500 Internal Server Error
{date} DEBUG : HTTP/1.1 500 Internal Server Error
{date} DEBUG : DELETE /{webdav_path}/data/52/{sha256-file_1} HTTP/1.1
{date} DEBUG : HTTP/1.1 404 Not Found
{date} DEBUG : {sha256-file_1}: Received error: HTTP/1.1 500 Internal Server Error
{date} DEBUG : MKCOL /{webdav_path}/data/52/ HTTP/1.1
{date} DEBUG : HTTP/1.1 405 Unknown Error
{date} DEBUG : DELETE /{webdav_path}/data/52/{sha256-file_2} HTTP/1.1
{date} DEBUG : HTTP/1.1 404 Not Found
{date} DEBUG : {sha256-file_2}: Received error: HTTP/1.1 500 Internal Server Error
{date} DEBUG : MKCOL /{webdav_path}/data/52/ HTTP/1.1
{date} DEBUG : HTTP/1.1 405 Unknown Error
{date} DEBUG : PUT /{webdav_path}/data/52/{sha256-file_1} HTTP/1.1
{date} DEBUG : HTTP/1.1 500 Internal Server Error
{date} DEBUG : PUT /{webdav_path}/data/52/{sha256-file_2} HTTP/1.1
{date} DEBUG : HTTP/1.1 500 Internal Server Error

(identical config and same rclone version but megacmd 1.4.1 works like a charm)

I don't think that the problem is on my machine but if you have any hint to better debug this issue will be greatly appreciated.
Comment 9 Riccardo Torrini 2023-12-29 14:23:26 UTC
(In reply to Riccardo Torrini from comment #7)

Updated server to 13.2-p8, another test:

Dec 29 12:31:19 hydra pkg[71202]: cryptopp upgraded: 8.5.0_1 -> 8.9.0
Dec 29 12:31:53 hydra pkg[71202]: megacmd upgraded: 1.4.1 -> 1.6.3_2

Same error using webdav, I found this messages during rclone copy
(extracted from megacmdserver.log with a little edit for readability):

# failed using latest version
[API:err: 13:29:07] Failed to open('/home/riccardo/.megaCmd/httputfile.getxfer.3355.1.mega.home/index/{sha256-file_1}'): error 2: No such file or directory
[API:err: 13:29:07] Failed to open('/home/riccardo/.megaCmd/httputfile.getxfer.3355.2.mega.home/index/{sha256-file_2}'): error 2: No such file or directory
[API:err: 13:29:07] Failed to open('/home/riccardo/.megaCmd/httputfile.getxfer.3355.3.mega.home/index/{sha256-file_3}'): error 2: No such file or directory
[API:err: 13:29:07] Failed to open('/home/riccardo/.megaCmd/httputfile.getxfer.3355.4.mega.home/index/{sha256-file_4}'): error 2: No such file or directory

# good with 1.4.1 (found with "ls -lart" inside ~/.megaCmd)
... 16901704 Dec 29 14:54 httputfile.getxfer.3568.326.mega
... 16911823 Dec 29 14:55 httputfile.getxfer.3568.336.mega
... 16922232 Dec 29 14:55 httputfile.getxfer.3568.337.mega
... 16878039 Dec 29 14:55 httputfile.getxfer.3568.338.mega

I think that cache file could be "{fixed-string}.{pid}.{sequence_number}.mega" on 1.4.1 but on new version there is a ".{some_part_of_current_file_name_and_path}" added to the end. I say "some part" because of missing element between "home" and "index", namely {hostname_of_backupped_machine}.
Maybe error 500 is because of missing/mispelled local cached file?

Probably new mega-cmd is badly broken, is not a pkg issue :-(

-- 
Riccardo.
Comment 10 Riccardo Torrini 2023-12-29 22:00:34 UTC
changed title to better track current issue
also sent a detailed bug report to upstream
Comment 11 kikadf 2024-05-24 17:45:20 UTC
megacmd has a new release, 1.7.0. I updated the port, if you can, try that, maybe fixed your problem. bug #279273