Bug 256099 - lang/rust: Reduce memory usage/pressure
Summary: lang/rust: Reduce memory usage/pressure
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: FreeBSD Rust Team
Depends on:
Reported: 2021-05-23 11:54 UTC by Daniel Engberg
Modified: 2021-09-19 09:16 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (rust)


Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Engberg freebsd_committer 2021-05-23 11:54:28 UTC
During some build stages rust/cargo eats a lot of memory using its default settings making it more or less not viable for low/mid-rage systems.

Can we consider setting cogegen-units to 1 or add a toggle for it and also perhaps parallel compiling?
Ref: https://reviews.freebsd.org/D30099#677659
Comment 1 Tobias Kortkamp freebsd_committer 2021-06-04 12:04:08 UTC
Daniel, I'm sorry, not trying to ignore you but unsure of expectations.
Seems worthwhile but someone needs to run the builds, get some
before and after numbers, research downsides if any, and then I
guess we can set codegen-units=1 if it looks ok, sure.

As for some of the comments from the review:

> [...] uses LLVM from their package tree instead of bundled, perhaps that's worth looking into?

lang/rust had an LLVM_PORT option once, but that only works if
somebody feels responsible for supporting it and fixing any regressions
that might happen.  But nobody really did so we removed it.

> Unbundle libssh2?

It's bundled now I guess because we bundle libgit2 now too (because
devel/libgit2 was not updated fast enough again).  Since we update
the toolchain every 6 weeks it is all probably not worth the hassle.
Comment 2 Daniel Engberg freebsd_committer 2021-06-06 21:52:18 UTC

I mainly started to look into this as building Rust on my low-end server (specs below) failed despite having quite a bit of RAM and swap at disposal.

While this isn't ideal way of logging here's a graph of memory usage at the end of compiling rust (2 jobs) that succeeds.

This box is an old Dell T20 with a dual core Intel Pentium G3220 CPU, 12Gb of RAM and running ZFS for Poudriere but not on rootfs. It runs 12.2-RELEASE-p6 and building Rust in a 12.2 jail. It's lightly loaded and uses about 5-6Gb of RAM (incl ZFS) without any Poudriere job running. I have no specific ZFS tuning set, however from what I can tell ZFS cache seems to grow quite a bit compiling Rust.

Setting codegen reduces memory usage about 1-1.5Gb from what I can tell but memory usage is still quite high.

I also gave this a go on my RockPro64 (arm64) (4Gb of RAM) running 13-STABLE (stable/13-n245283-70a2e9a3d44), UFS only and while it took 14h+ hours (-j1) it did finish. During compiing it used about 2Gb (the job not the complete system) tops which is a lot less than what I'm seeing on my server.

I'll give this a go on another box running 13-STABLE (amd64) and see if that also consumes a lot of memory.

Thanks for replying about LLVM and libssh2, if it's too much of a hassle I understand the decision :-)
Comment 3 Daniel Engberg freebsd_committer 2021-06-07 01:08:15 UTC
Hmm... compiling & optimizing seems to use a bit more memory, I did see a few processes use more than 2.5Gb of memory. Wired memory is a lot more though, ~4.8G and peaked at 6.7G so I guess that's due to ZFS?
Comment 4 commit-hook freebsd_committer 2021-06-14 20:52:02 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=294f0c5c206d70e24b6bbc28766d962dc82f8b61

commit 294f0c5c206d70e24b6bbc28766d962dc82f8b61
Author:     Tobias Kortkamp <tobik@FreeBSD.org>
AuthorDate: 2021-06-14 18:50:33 +0000
Commit:     Tobias Kortkamp <tobik@FreeBSD.org>
CommitDate: 2021-06-14 20:51:11 +0000

    lang/rust-nightly: Try to reduce memory usage/pressure

    Try to reduce memory usage/pressure by only using one code generation

    "This flag [codegen-units] controls how many code generation units
    the crate is split into.  It takes an integer greater than 0.

    When a crate is split into multiple codegen units, LLVM is able to
    process them in parallel.  Increasing parallelism may speed up
    compile times, but may also produce slower code.  Setting this to
    1 may improve the performance of generated code, but may be slower
    to compile."


    PR:             256099
    Suggested by:   Daniel Engberg

 lang/rust/Makefile | 3 +++
 1 file changed, 3 insertions(+)
Comment 5 Guangyuan Yang freebsd_committer 2021-06-27 09:27:01 UTC
Just wanted to report here that, building Rust always get OOM'ed after ~6 hours on my low-end build box 10 times in a row, spec:

- Intel i5-6500T (4) @ 2.496GHz
- 16GB RAM, 2GB swap
- FreeBSD 13.0-RELEASE amd64
- ZFS on root

It does finish on my VPS which has similar spec but more swap, however, the memory/swap usage is very high. Spec:

- Intel Xeon Platinum 8171M (4) @ 2.095GHz
- 16GB RAM, 32GB swap
- FreeBSD 13.0-RELEASE amd64
- UFS on root, with ZFS enabled on datadisks

I haven't looked at it closely, so will report back if I noticed anything. Thanks!
Comment 6 Guangyuan Yang freebsd_committer 2021-06-30 01:33:57 UTC
Cannot get lang/rust-nightly to build on my build box with 16G memory with the code generation unit change. The CPU usage is down to one core, but the RAM pressure is still very high, and the whole process ended up getting OOM'ed.
Comment 7 commit-hook freebsd_committer 2021-09-19 09:16:18 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=6f1fefb50e755d727f471aeb75ebe4e28f876b4b

commit 6f1fefb50e755d727f471aeb75ebe4e28f876b4b
Author:     Tobias Kortkamp <tobik@FreeBSD.org>
AuthorDate: 2021-09-07 08:14:14 +0000
Commit:     Tobias Kortkamp <tobik@FreeBSD.org>
CommitDate: 2021-09-19 09:03:21 +0000

    lang/rust: Update to 1.55.0

    - Set codegen-units=1 [1]
    - Add hack to skip cargo update on git sources as a step towards solving [2]
    - Fix 'capacity overflow' panics on armv* [3]

    Changes:        https://blog.rust-lang.org/2021-09-09/Rust-1.55.0.html
    PR:             258337
    PR:             256099 [1]
    PR:             256581 [2]
    PR:             257419 [3]
    Reviewed by:    mikael, pkubaj
    Exp-run by:     antoine
    Differential Revision:  https://reviews.freebsd.org/D31872
    With hat:       rust

 Mk/Uses/cargo.mk                                   |   2 +-
 Mk/bsd.gecko.mk                                    |   2 +-
 lang/rust-bootstrap/Makefile                       |   8 +-
 lang/rust-bootstrap/distinfo                       |   6 +-
 lang/rust/Makefile                                 |  12 +--
 lang/rust/distinfo                                 | 114 ++++++++++-----------
 ...m-project_compiler-rt_lib_builtins_cpu__model.c |  21 ++--
 ...ols_cargo_src_cargo_sources_git_source.rs (new) |  45 ++++++++
 ...rc_tools_cargo_src_cargo_util_toml_mod.rs (new) |  22 ++++
 .../patch-vendor_openssl-sys_build_main.rs (gone)  |  19 ----
 ..._src_unix_bsd_freebsdlike_freebsd_mod.rs (gone) |  12 ---
 ..._unix_bsd_freebsdlike_freebsd_powerpc.rs (gone) |  50 ---------
 .../powerpc64-elfv1/patch-src_bootstrap_native.rs  |  10 +-
 ...h-compiler_rustc__target_src_spec_mod.rs (gone) |  10 --
 ...rc_spec_powerpc64le__unknown__freebsd.rs (gone) |  19 ----
 15 files changed, 154 insertions(+), 198 deletions(-)