Bug 263436 - www/firefox uses excessive amount of memory
Summary: www/firefox uses excessive amount of memory
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-gecko (Nobody)
URL:
Keywords: needs-qa, performance
Depends on:
Blocks:
 
Reported: 2022-04-20 13:19 UTC by Josef 'Jeff' Sipek
Modified: 2023-04-23 08:53 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (gecko)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Josef 'Jeff' Sipek 2022-04-20 13:19:17 UTC
Unfortunately, I don't have much hard data to present.  Ever since I upgraded to Firefox 98, my system's RAM (16G) and swap (2G) get nearly exhausted within a couple of days of starting Firefox.  Restarting Firefox releases all the memory as expected and all is well for 2-3 days before the memory consumption gets high enough to cause unnecessary paging.  Before the upgrade to v98, I used to run firefox for weeks without *any* issues like this.

Normally I use a couple of extensions, but I've reproduced it without any extensions enabled.


$ grep firefox /var/log/messages
Jan 20 14:23:29 satis pkg[1355]: firefox upgraded: 90.0.1,2 -> 95.0.2_2,2 
Mar 19 12:55:28 satis pkg[91791]: firefox upgraded: 95.0.2_2,2 -> 98.0.1,2 
Mar 28 22:30:31 satis kernel: pid 4282 (firefox), jid 0, uid 1000, was killed: out of swap space
Apr  3 09:31:13 satis pkg[27892]: firefox upgraded: 98.0.1,2 -> 99.0,2 
Apr  6 23:13:24 satis kernel: pid 10976 (firefox), jid 0, uid 1000, was killed: out of swap space
Apr 10 18:47:58 satis pkg[1460]: firefox upgraded: 99.0,2 -> 99.0_1,2 
Apr 16 17:27:15 satis pkg[81292]: firefox upgraded: 99.0_1,2 -> 99.0_2,2 
Apr 19 00:04:57 satis kernel: pid 6668 (firefox), jid 0, uid 1000, was killed: out of swap space
Apr 19 00:04:59 satis kernel: pid 82135 (firefox), jid 0, uid 1000, was killed: out of swap space
Apr 19 00:05:01 satis kernel: pid 7545 (firefox), jid 0, uid 1000, was killed: out of swap space


The firefox got killed because it ran the system out of memory on the 19th.  I restarted it, and after ~10 hours of mostly idling it was:

$ ps aux|egrep '(COMMAND|firefox)'
USER         PID  %CPU %MEM      VSZ    RSS TT  STAT STARTED        TIME COMMAND
jeffpc     12355   0.1  7.1 11646412 593476 v0  S    09:48       0:18.64 firefox
jeffpc     12357   0.0  1.6   287388 129120 v0  I    09:48       0:00.05 /usr/local/lib/firefox/firefox -content
jeffpc     12358   0.0  2.0  2532140 168012 v0  I    09:48       0:00.48 /usr/local/lib/firefox/firefox -content
jeffpc     12359   0.0  1.9  2522204 159644 v0  I    09:48       0:00.38 /usr/local/lib/firefox/firefox -content
jeffpc     12362   0.0  4.6  2956960 381616 v0  S    09:48       0:04.63 /usr/local/lib/firefox/firefox -content
jeffpc     12363   0.0  2.9  2794572 238748 v0  S    09:48       0:01.76 /usr/local/lib/firefox/firefox -content
jeffpc     12367   0.0  1.8  2506476 148620 v0  I    09:49       0:00.26 /usr/local/lib/firefox/firefox -content
jeffpc     12368   0.0  2.5  2567212 204548 v0  S    09:49       0:00.95 /usr/local/lib/firefox/firefox -content
jeffpc     12369   0.0  1.9  2516688 160500 v0  S    09:49       0:00.78 /usr/local/lib/firefox/firefox -content
jeffpc     12370   0.0  2.0  2526916 163660 v0  I    09:49       0:00.34 /usr/local/lib/firefox/firefox -content
jeffpc     12372   0.0  2.1  2533480 176008 v0  S    09:49       0:02.67 /usr/local/lib/firefox/firefox -content
jeffpc     12373   0.0  1.8  2503872 145752 v0  I    09:49       0:00.12 /usr/local/lib/firefox/firefox -content
jeffpc     12394   0.0  1.8  2503868 146488 v0  I    09:49       0:00.11 /usr/local/lib/firefox/firefox -content
jeffpc     12395   0.0  1.8  2503872 146452 v0  I    09:49       0:00.11 /usr/local/lib/firefox/firefox -content
jeffpc     12445   0.0  0.0    12868   2016  7  S+   09:53       0:00.00 egrep (COMMAND|firefox)

Another 24 hours later (with some evening/morning browsing and lots of idling during the day/night):

$ ps aux|egrep '(COMMAND|firefox)'
USER         PID  %CPU %MEM      VSZ     RSS TT  STAT STARTED        TIME COMMAND
jeffpc     12355   0.6 12.5 12928316 1039292 v0  S    09:48      20:39.08 firefox
jeffpc     17248   0.2  7.6  3500156  633440 v0  S    20:29       7:34.43 /usr/local/lib/firefox/firefox -conten
jeffpc     17436   0.2  6.5  3633244  541748 v0  S    20:54       8:38.38 /usr/local/lib/firefox/firefox -conten
jeffpc     16833   0.1  5.5  3447880  460024 v0  R    19:35       5:51.86 /usr/local/lib/firefox/firefox -conten
jeffpc     12357   0.0  1.1   287516   94092 v0  I    09:48       0:00.10 /usr/local/lib/firefox/firefox -conten
jeffpc     12358   0.0  1.7  2616232  143672 v0  I    09:48       0:06.67 /usr/local/lib/firefox/firefox -conten
jeffpc     12359   0.0  1.4  2529100  112988 v0  I    09:48       0:01.75 /usr/local/lib/firefox/firefox -conten
jeffpc     12362   0.0  6.5  3359912  537288 v0  S    09:48       4:46.52 /usr/local/lib/firefox/firefox -conten
jeffpc     12363   0.0  2.7  3064288  226496 v0  S    09:48       2:10.52 /usr/local/lib/firefox/firefox -conten
jeffpc     12367   0.0  1.4  2525440  113836 v0  S    09:49       0:04.57 /usr/local/lib/firefox/firefox -conten
...20 lines omitted...
jeffpc     23084   0.0  1.6  2523704  135192 v0  S    08:49       0:01.00 /usr/local/lib/firefox/firefox -conten
jeffpc     23085   0.0  1.5  2506320  125904 v0  S    08:49       0:00.56 /usr/local/lib/firefox/firefox -conten
jeffpc     23086   0.0  1.5  2505404  125068 v0  S    08:49       0:00.53 /usr/local/lib/firefox/firefox -conten
jeffpc     23087   0.0  1.7  2551240  140020 v0  S    08:49       0:00.89 /usr/local/lib/firefox/firefox -conten
jeffpc     23088   0.0  1.5  2505272  124888 v0  S    08:49       0:00.50 /usr/local/lib/firefox/firefox -conten
jeffpc     23099   0.0  4.8  2904628  396040 v0  S    08:50       0:35.20 /usr/local/lib/firefox/firefox -conten
jeffpc     23109   0.0  1.5  2503872  122596 v0  I    08:51       0:00.27 /usr/local/lib/firefox/firefox -conten
jeffpc     23205   0.0  1.5  2503872  126068 v0  S    08:59       0:00.27 /usr/local/lib/firefox/firefox -conten
jeffpc     23236   0.0  1.5  2503872  126860 v0  I    09:02       0:00.18 /usr/local/lib/firefox/firefox -conten
jeffpc     23253   0.0  0.0    12868    1932  7  S+   09:04       0:00.00 egrep (COMMAND|firefox)


What the ps(1) output doesn't show is that the virtual size of one of the processes seems to climb to as high as 25GB when actively browsing the web but quickly drop back down to "reasonable" single digit GB.

Let me know what other information I should collect or what I should try to get to the bottom of this.
Comment 1 Graham Perrin freebsd_committer freebsd_triage 2022-04-22 21:27:56 UTC
Packages from latest or quarterly? 

pkg -vv | grep -e url -e enabled

Which version of FreeBSD, exactly?

freebsd-version -kru ; uname -aKU

Is there discussion anywhere else?

(In reply to Josef 'Jeff' Sipek from comment #0)

> … reproduced it without any extensions enabled. …

With Firefox in safe mode, or by disabling the extensions in normal mode? 

> … virtual size of one of the processes seems to climb to as high as 25GB …

Aim to identify the site, or page, with which this occurs. 

about:memory
Comment 2 Josef 'Jeff' Sipek 2022-04-22 23:42:23 UTC
> Packages from latest or quarterly? 

Latest:

    url             : "pkg+http://pkg.FreeBSD.org/FreeBSD:13:amd64/latest",

> Which version of FreeBSD, exactly?

$ freebsd-version -kru ; uname -aKU
13.0-RELEASE-p11
13.0-RELEASE-p11
13.0-RELEASE-p11
FreeBSD satis 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr  5 18:54:35 UTC 2022     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 1300139 1300139

> Is there discussion anywhere else?

None that I'm aware of.

> (In reply to Josef 'Jeff' Sipek from comment #0)
> > … reproduced it without any extensions enabled. …
>
> With Firefox in safe mode, or by disabling the extensions in normal mode? 

Disabling in normal mode.

> > … virtual size of one of the processes seems to climb to as high as 25GB …
>
> Aim to identify the site, or page, with which this occurs. 

Pretty much every page make the virtual size spike up.  Not necessarily to 25GB, but it is relatively common.  Just now, the "main" process has 13GB virt / 1.1GB rss.  I opened a new tab, and the virtual size jumped to 19GB.  Closing the tab and opening a new tab again, didn't do it.  Waiting 20 minutes and opening a new tab did it again.

> about:memory

I have no idea how to read this or what to look for.

FWIW, it says:

 WARNING: the 'heap-allocated' memory reporter does not work for this platform and/or configuration. This means that 'heap-unclassified' is not shown and the 'explicit' tree shows less memory than it should.
Comment 3 Graham Perrin freebsd_committer freebsd_triage 2022-04-23 18:40:28 UTC
(In reply to Josef 'Jeff' Sipek from comment #2)

Despite the shortcomings, about:memory may help to identify the site, or page, that's causing excessive use of memory. 

When you sense excessive use, click each item under the process index. Is any one using an extraordinarily large amount of memory? 

Here, for example, I have one item that clearly uses around seven times as much as any other. Not a bug, in my case …

… in your case, maybe you'll find something that uses much more than 7x what's used by other things.
Comment 4 Josef 'Jeff' Sipek 2022-05-13 15:31:49 UTC
Sorry for the delay.  I about mid-way through my multi-day data "collection", I upgraded to firefox 100 and so I wanted to give it enough time to see what it behaves like.  Observations:

* Firefox 100 seems a little better than 99
* The memory usage growth seems to be a function of how many (not which) websites I visit.  "Vigorous" browsing will run up the process size much faster than more casual browsing.
* Having approximately 20 firefox ~150-380MB processes mostly idle along with the 1.4GB main process chews up about 1/3 of the system RAM.
* The main process is the major memory hog - the other processes, while not lightweight, tend to be more reasonable.
* The about:memory output doesn't show anything that jumps out at me (aside from the ridiculous vsize/resident and max vsize/resident values).

Is there a way to tell firefox that even though the machine has 16GB, I'd like to keep some memory for the ARC, kernel, and other processes?

Main process's about:memory headings & summary:

563.47 MB (100.0%) -- explicit
├──364.26 MB (64.65%) -- gfx
16.18 MB (100.0%) -- decommitted
6,172 (100.0%) -- event-counts
16 (100.0%) -- extensions
379.84 MB (100.0%) -- gfx
4.35 MB (100.0%) -- images
249 (100.0%) -- ipc-channels
285 (100.0%) -- ipc-channels-peak
502 (100.0%) -- js-component-loader
4 (100.0%) -- js-helper-threads
90.11 MB (100.0%) -- js-main-runtime
33.00 MB (100.0%) -- js-main-runtime-gc-heap-committed
22 (100.0%) -- js-main-runtime-realms
58 (100.0%) -- message-manager
1,973 (100.0%) -- observer-service
747 (100.0%) -- observer-service-suspect
674 (100.0%) -- preference-service
0 (100.0%) -- queued-ipc-messages
0.14 MB (100.0%) -- shared-string-bundles
7.77 MB (100.0%) -- window-objects

     1.50 MB ── font-list-shmem
     2.18 MB ── gfx-surface-image
     0.00 MB ── gfx-textures
     0.00 MB ── gfx-textures-peak
     0.00 MB ── gfx-tiles-waste
           0 ── ghost-windows
           3 ── imagelib-surface-cache-already-present-count
     0.58 MB ── imagelib-surface-cache-estimated-locked
     0.64 MB ── imagelib-surface-cache-estimated-total
         146 ── imagelib-surface-cache-image-count
         174 ── imagelib-surface-cache-image-surface-count
          94 ── imagelib-surface-cache-locked-image-count
         115 ── imagelib-surface-cache-locked-surfaces-count
           0 ── imagelib-surface-cache-overflow-count
           0 ── imagelib-surface-cache-table-failure-count
          59 ── imagelib-surface-cache-tracked-cost-count
          59 ── imagelib-surface-cache-tracked-expiry-count
           0 ── imagelib-surface-cache-tracking-failure-count
     2.06 MB ── js-main-runtime-temporary-peak
       4,992 ── page-faults-hard
  13,369,887 ── page-faults-soft
 1,710.35 MB ── private
 1,362.16 MB ── resident
 1,448.52 MB ── resident-peak
     2.04 MB ── shmem-allocated
   126.62 MB ── shmem-mapped
           0 ── unresolved-ipc-responses
20,886.48 MB ── vsize
 8,191.63 MB ── vsize-max-contiguous
     0.00 MB ── wasm-runtime
Comment 5 Lena 2022-05-19 18:35:41 UTC
You can try to start Firefox like this:

limits -m 4g firefox
Comment 6 Josef 'Jeff' Sipek 2022-05-21 23:42:19 UTC
(In reply to Lena from comment #5)

Interesting idea to use limits.

Setting the limit to 4GB works to keep the *sum* of all the firefox processes' resident sizes under 4GB, but the main process's resident size is slowly growing (as before).  So, while 2 days ago (shortly after restarting firefox) the main process accounted for ~750MB (~18% of 4GB) now it is 1560MB (38%).  As a result, firefox keeps fewer process alive as time goes by.

I don't know if it will stop growing at some point, but if it doesn't the main process will hit the 4GB limit on its own. :)

Oh, and the virtual size of the main process is still ridiculous 13GB, but I suppose that doesn't matter that much.  (Yes, I'm making a few assumptions about firefox's memory usage and FreeBSD kernel's handling of memory in that statement.)
Comment 7 Josef 'Jeff' Sipek 2022-06-14 01:44:06 UTC
(In reply to Josef 'Jeff' Sipek from comment #6)
> Setting the limit to 4GB works to keep the *sum* of all the firefox processes' resident sizes under 4GB, but the main process's resident size is slowly growing (as before).

I spoke too soon - it doesn't appear to be limiting the sum.  I started firefox with a 4GB limit and after about a week the 31 firefox processes that were running chewed up over 6GB RAM.  The resident sizes in MB: 1174 785 300 213 196 195 191 187 185 180 178 170 168 166 156 154 150 142 141 125 124 123 119 113 112 111 104 101 99 96 86
Comment 8 freebsd 2022-08-27 15:39:02 UTC
Maybe we can nudge the Importance up one level, as it not only affects you. I've seen this behaviour for quite some time, going into multiple years now.

Firefox always seems to consume 10s of GB after a few days, and I until now chucked it up to the many tabs being open (~100-200, split over 2-4 windows with ~100 in the main one), some with JS-heavy/media content + some addon(s) now behaving well like Treestyle Tabs, uMatrix, uBlock Origin and the like.
But I can't help but think that just a few years back I was able to keep that many tabs open without problems, so while it might be the more JS-intensive nature of the modern web, I'm still not sure this isn't some kind of memory leak or other inefficiency.

Invariantly, at some point, swap is exhausted and stuff gets oom-killed.

For comparison, my about:memory summary for the main process:

Main Process (pid 9174)

WARNING: the 'heap-allocated' memory reporter does not work for this platform and/or configuration. This means that 'heap-unclassified' is not shown and the 'explicit' tree shows less memory than it should.
Explicit Allocations

899.20 MB (100.0%) ++ explicit

Other Measurements

36.55 MB (100.0%) ++ decommitted

17,831 (100.0%) ++ event-counts

36 (100.0%) ++ extensions

481.56 MB (100.0%) ++ gfx

8.54 MB (100.0%) ++ images

691 (100.0%) ++ ipc-channels

774 (100.0%) ++ ipc-channels-peak

8 (100.0%) ++ js-helper-threads

119.21 MB (100.0%) ++ js-main-runtime

34.11 MB (100.0%) ++ js-main-runtime-gc-heap-committed

34 (100.0%) ++ js-main-runtime-realms

477 (100.0%) ++ js-module-loader

0.39 MB (100.0%) ++ memory-blob-urls

106 (100.0%) ++ message-manager

3,954 (100.0%) ++ observer-service

2,324 (100.0%) ++ observer-service-suspect

720 (100.0%) ++ preference-service

0 (100.0%) ++ queued-ipc-messages

0.14 MB (100.0%) ++ shared-string-bundles

22.09 MB (100.0%) ++ window-objects

     4.00 MB ── font-list-shmem
     6.53 MB ── gfx-surface-image
     0.00 MB ── gfx-textures
     0.00 MB ── gfx-textures-peak
     0.00 MB ── gfx-tiles-waste
           0 ── ghost-windows
           0 ── imagelib-surface-cache-already-present-count
     0.29 MB ── imagelib-surface-cache-estimated-locked
     0.43 MB ── imagelib-surface-cache-estimated-total
         445 ── imagelib-surface-cache-image-count
         302 ── imagelib-surface-cache-image-surface-count
         377 ── imagelib-surface-cache-locked-image-count
         218 ── imagelib-surface-cache-locked-surfaces-count
           0 ── imagelib-surface-cache-overflow-count
           0 ── imagelib-surface-cache-table-failure-count
          84 ── imagelib-surface-cache-tracked-cost-count
          84 ── imagelib-surface-cache-tracked-expiry-count
           0 ── imagelib-surface-cache-tracking-failure-count
     2.31 MB ── js-main-runtime-temporary-peak
         204 ── page-faults-hard
  24,126,094 ── page-faults-soft
10,129.50 MB ── private
 9,815.91 MB ── resident
 9,893.29 MB ── resident-peak
     2.10 MB ── shmem-allocated
    70.25 MB ── shmem-mapped
           1 ── unresolved-ipc-responses
46,885.31 MB ── vsize
 8,191.63 MB ── vsize-max-contiguous
     0.00 MB ── wasm-runtime

End of Main Process (pid 9174)
Comment 9 Graham Perrin freebsd_committer freebsd_triage 2022-08-28 18:20:33 UTC
about:config

fission.autostart

Try a change from true (the default) to false, quit, then run Firefox for a while to tell whether there's any significant difference to the excess. 

(In reply to Josef 'Jeff' Sipek from comment #6)

> … the virtual size of the main process is … 13GB, …

13.7 GB here, at the time of writing, measured by htop.

% pkg info -x firefox
firefox-104.0_2,2
% uname -aKU
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-CURRENT FreeBSD 14.0-CURRENT #19 main-n257401-82493ff7007a: Tue Aug 16 05:16:20 BST 2022     grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 1400065 1400065
%
Comment 10 Josef 'Jeff' Sipek 2022-09-02 12:32:36 UTC
The memory usage seems to grow especially fast when I'm looking around Mouser for components to buy.  I don't know if this is because of something the site does or the content types (e.g., I never look at PDFs in firefox otherwise), or if I just happen to visit significantly more pages/minute than during normal browsing of the web.

(In reply to Graham Perrin from comment #9)
> about:config
>
> fission.autostart
>
> Try a change from true (the default) to false, quit, then run Firefox for a while to tell whether there's any significant difference to the excess.

I can give it a try. This disables site isolation, correct?
Comment 11 Josef 'Jeff' Sipek 2022-09-07 01:47:48 UTC
Over the past few days I've been running with fission.autostart=false.  The memory usage crept up anyway.

Out of curiosity, how many processes am I supposed to see with fission off?  When I restart firefox, I get 7 processes with the main one using 11GB virtual and about 500MB resident.
Comment 12 Graham Perrin freebsd_committer freebsd_triage 2022-09-07 06:13:14 UTC
(In reply to Josef 'Jeff' Sipek from comment #11)

> ... When I restart firefox, I get 7 processes ...

The number probably relates (not equates) to the number of windows to be reopened.
Comment 13 Josef 'Jeff' Sipek 2022-09-11 12:09:26 UTC
I just had to restart firefox because it ate all the swap & RAM on the system.  So, fission.autostart=false doesn't fix it/work around it.

I don't know how much memory & swap firefox was using last night but I did *not* notice any memory-related lag when I went to bed.  This morning (9 hours later), everything was very laggy - so I checked memory usage and found the system with 90-something percent memory and swap used.  So, it isn't direct activity that causes this but rather something in the background.
Comment 14 Graham Perrin freebsd_committer freebsd_triage 2022-11-03 19:02:25 UTC
(In reply to Josef 'Jeff' Sipek from comment #2)

> 13.0-RELEASE-p11

Please tell whether symptoms are reproducible with this week's 13.1-RELEASE-p3.
Comment 15 Josef 'Jeff' Sipek 2022-11-04 01:50:45 UTC
> Please tell whether symptoms are reproducible with this week's 13.1-RELEASE-p3.

Sorry for not including that info in earlier comments, but I upgraded the box to 13.1-RELEASE about a month after it came out.  I'll update to -p3 and let you know how it goes.

(I've gotten much better at noticing the warning signs and restarting firefox before it is too late to avoid running out of memory and swap. :) )
Comment 16 Josef 'Jeff' Sipek 2022-12-01 15:04:16 UTC
> Please tell whether symptoms are reproducible with this week's 13.1-RELEASE-p3.

Yes, it still happens with Firefox 107.  It appears to get especially bad when I'm spending time on mouser(dot)com.  Both actively searching and viewing PDFs (using the built-in viewer), as well as when I let it idle over night.  (In the morning, lots of things are swapped out.)  I'm not sure if this is because on this website I end up clicking around more than usual, or if there is something about this site's content that doesn't sit well with firefox.

 $ freebsd-version -kru
 13.1-RELEASE-p3
 13.1-RELEASE-p3
 13.1-RELEASE-p4

 $ uname -aKU
 FreeBSD satis 13.1-RELEASE-p3 FreeBSD 13.1-RELEASE-p3 GENERIC amd64 1301000 1301000
Comment 17 Graham Perrin freebsd_committer freebsd_triage 2023-04-23 08:53:16 UTC
(In reply to Josef 'Jeff' Sipek from comment #16)

Please try 112.0.1_1,2 or greater with a clean (new) profile on FreeBSD 13.2-RELEASE. 

about:profiles