Bug 235305

Summary: net/glusterfs: Memory leak in 3.11/3.12
Product: Ports & Packages Reporter: Vincent Milum Jr <freebsd>
Component: Individual Port(s)Assignee: freebsd-ports-bugs (Nobody) <ports-bugs>
Status: Open ---    
Severity: Affects Many People CC: craig001, daniel, flo, gnoma_86, iron.udjin, koobs, lapo
Priority: --- Keywords: needs-patch, needs-qa
Version: LatestFlags: koobs: maintainer-feedback? (daniel)
koobs: merge-quarterly?
Hardware: Any   
OS: Any   
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=1593884
https://bugzilla.redhat.com/show_bug.cgi?id=1613512
Attachments:
Description Flags
upgrade to 3.12.15 none

Description Vincent Milum Jr 2019-01-29 19:39:26 UTC
There are known memory leaks in upstream Gluster 3.11 and 3.12 which has since been resolved. Those patches have yet to hit FreeBSD, and I can easily confirm that I'm hitting the exact same memory leaks described in the upstream bugs. 

Working with a large number of files (in my case, writing 100k-1m files into Gluster) will eventually cause GlusterFS to consume all RAM and SWAP until the system becomes unresponsive due to OOM.

Upstream details: https://bugzilla.redhat.com/show_bug.cgi?id=1593884
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2019-01-30 03:04:09 UTC
Upstream "backport to 3.12.x" request: https://bugzilla.redhat.com/show_bug.cgi?id=1613512

"This is fixed with 3.12.13 now.
Comment 2 craig001 2019-02-14 14:42:44 UTC
please patch and change maintainer
Comment 3 Lapo Luchini 2019-06-14 13:10:24 UTC
I guess the previous comment could be read as a resignation from maintainership and, thus, this PR should not anymore be waiting on "maintainer feedback".

Hoping this will be fixed soon!
(and so soon as I have some time on hand I could try the backport myself)
Comment 4 Lapo Luchini 2019-06-14 13:21:02 UTC
I guess the previous comment could be read as a resignation from maintainership and, thus, this PR should not anymore be waiting on "maintainer feedback".

Hoping this will be fixed soon!
(and so soon as I have some time on hand I could try the backport myself)
Comment 5 Lapo Luchini 2019-07-03 07:54:16 UTC
Created attachment 205495 [details]
upgrade to 3.12.15

This is the easiest upgrade (with current patchset) from current version that contains a fix for this leak (was fixed upstream in 3.12.13).

Unfortunately, to my testing, it doesn't change much (if at all) as there are other leaks.

Upgrading to 3.13 or later might solve those, but that needs new patches that I didn't attempt.
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2019-07-03 09:08:18 UTC
(In reply to Lapo Luchini from comment #5)

Thank you for the patch Lapo. Could you reference from where this patch came?

Upstream issue/pr/commit references would be great to comment the patch with
Comment 7 Lapo Luchini 2019-07-03 10:12:09 UTC
It's a simple upgrade s/3.11.1/3.12.15/ but that had a compilation error, which I fixed by adding a missing "gf_" in a function name (as suggested by the compiler itself). (I don't understand how that even compiles upstream)

Truth said: I'm not *sure* what that function does and if my fix is strictly correct, but given the absence of "uuid_is_null()" in the sources, the presence of "gf_uuid_is_null()" and the fact that the variable is called "gfid" and the following line calls "gf_uuid_copy()" I'd say it's safe to assume that's the correct fix.
Comment 9 gnoma 2019-10-23 11:37:42 UTC
Hello, 

Any update on this? 
The package I see in the repository is still glusterfs-3.11.1_6. 


Thanks
Comment 10 iron.udjin 2019-12-15 22:15:00 UTC
Hello,

There is 3.13.2 available. But it doesn't compile on 12.1-STABLE:

--- glfs-fops.lo ---
glfs-fops.c:4894:13: warning: logical not is only applied to the left hand side of this bitwise operator [-Wlogical-not-parentheses]
        if (!xstat->flags_handled & GFAPI_XREADDIRP_STAT)
            ^                     ~
glfs-fops.c:4894:13: note: add parentheses after the '!' to evaluate the bitwise operator first
        if (!xstat->flags_handled & GFAPI_XREADDIRP_STAT)
            ^
             (                                          )
glfs-fops.c:4894:13: note: add parentheses around left hand side expression to silence this warning
        if (!xstat->flags_handled & GFAPI_XREADDIRP_STAT)
            ^
            (                    )
--- glfs-mgmt.lo ---
mv -f .deps/glfs-mgmt.Tpo .deps/glfs-mgmt.Plo
--- glfs-resolve.lo ---
mv -f .deps/glfs-resolve.Tpo .deps/glfs-resolve.Plo
--- glfs.lo ---
error: versioned symbol glfs_upcall_register@@GFAPI_3.13.0 must be defined
error: versioned symbol glfs_upcall_unregister@@GFAPI_3.13.0 must be defined
2 errors generated.
*** [glfs.lo] Error code 1

make[5]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2/api/src
--- glfs-handleops.lo ---
mv -f .deps/glfs-handleops.Tpo .deps/glfs-handleops.Plo
--- glfs-fops.lo ---
1 warning generated.
mv -f .deps/glfs-fops.Tpo .deps/glfs-fops.Plo
1 error

make[5]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2/api/src
*** [all-recursive] Error code 1

make[4]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2/api
1 error

make[4]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2/api
*** [all-recursive] Error code 1

make[3]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2
1 error

make[3]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2
*** [all] Error code 2

make[2]: stopped in /usr/ports/net/glusterfs/work/glusterfs-3.13.2
1 error

Is there way to fix it?
Comment 11 Florian Smeets freebsd_committer 2020-07-29 20:38:53 UTC
I'm still seeing memory leaks in what I think is the client part (the mounted gluster fs) after the Update to 8.0