Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Description • 11 years ago

When user works with window-based workflow (using mostly/only windows and no tabs) Firefox tends to accumulate memory and not free it. I'm not sure if it happens with tabs, but actively browsing for a day raises memory usage to some 1.5GB. I took some memory dumps and it is clearly visible that detached/system principal (whatever that would mean) windows consume a considerable amount of memory.

	SthMan
	Comment 2 • 11 years ago

I have observed the same issue. 
Environment: Win7 fully patched, Intel i3 quad core @ 2.4Ghz, 8Gb RAM, FF 21.0, no FF extensions, only Flash 11.7.700.202 plug-in

Mode of Operation: "Open new windows in a new tab instead" unticked, "When I open a link to a new tab, switch to it immediately" ticked = all sites are open in browser windows, no tabs are used, tab bar is not operating.  Startup = "Show my windows and tabs from last time"

During the day as I work opening sites, clicking to other sites and closing browser windows the memory consumption ("Working Set" and "Memory" in Windows Task Manager) grows steadily until about 2Gb or more. My PC then starts to stall every 10 seconds or so. Closing all browser windows one-by-one until only "about:memory" is left does NOT result in any memory usage reduction - even after a couple of hours wait for GC to happen.

Only fix is a shutdown and re-start after which all windows are automatically restored (see Startup setting above), but after 1h or so of work the memory usage is back up to 2Gb or so.

My work involves typical browsing of GMail, technical web sites, Twitter, Facebook, etc nothing fancy, no games or video intensive work.

A FF browser expert looked at my about:memory and noted: "667.25 MB (31.13%) -- js/compartment([System Principal]about:blank), Meaning a blank page is using up almost 700 MB in js.  ...seems to be related to closing windows and opening new ones, which seems to cause "loose" blank windows and hanging chrome scripts"

I have tried the following:

1) moving all open windows into tabs: This seems to reduce the overall memory usage, but the symptoms are the same.

2) found some info (here: https://plus.google.com/104505198652368420143/posts/8gXqb3kG7bw and here: http://www.sevenforums.com/browsers-mail/159574-firefox-memory-leak-2.html) which suggested the following config changes:

javascript.options.mem.disable_explicit_compartment_gc = false (default = true)
javascript.options.mem.high_water_mark = 8 (default = 64)
javascript.options.mem.max = 12800 (default = -1)
and
browser.sessionhistory.max_entries = 5
browser.sessionhistory.max_total_viewers = 5
browser.sessionstore.max_tabs_undo = 5

This seems to slightly reduce the memory consumption, but the overall problem persists.

about;memory listing attached as FirefixMemProbs.txt

	SthMan
	Comment 4 • 11 years ago

I have now run a follow-on test in which I opened all browsing as new browser tabs rather than new browser windows. My typical use is to have 20-30 screens open at one time with regular opening and closing of screens as my work requires. I do not normally shut down my computer at night or otherwise.

Similar to what Simonas reports, the result observed has been much less growth in memory use and also much less "browser stalling" I had observed once the memory use grows beyond 2Gb.

From the above it seems that there is a "memory leak" when opening and closing new browser windows, which does not seem to happen (as much) when opening and closing new tabs.

Can somebody who knows the innards of FF have a look at this - from my Google searches a lot of people are frustrated by this unnecessary flaw in FF and fixing it would be a great plus (+1 maybe ;-) for FF!

Thank you!

	Mark Straver
	Comment 5 • 11 years ago

I think Bug 710282 might be a duplicate of this bug or at least related - both indicating extended, intensive use of the browser causing a steady increase in memory allocated but not freed.

	SthMan
	Comment 6 • 11 years ago

Mark, I think you are right when suggesting that https://bugzilla.mozilla.org/show_bug.cgi?id=710282 is similar or same.  But #710282 does not have much tech info and no updates since ~18 months...

	SthMan
	Comment 7 • 11 years ago

All,

I think I have excluded Flash / Silverlight as the culprits.  I have now run the browser for two weeks with the Flash and Silverlight plug-ins disabled.  The continual growth in memory usage is still there.

As mentioned above, the growth in memory usage appears to be more when opening new sites in windows rather than tabs.

So it would seem that the problem is with the FF code not releasing memory when newly opened windows are closed?

After a week's work in FF, the memory usage is now ~3Gb - which does not reduce when I close all but one window/tab.

Firefox coders, PLEASE HELP!

K

	Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Comment 8 • 11 years ago

I never even had Flash (or could even have Silverlight, I'm on Linux) installed, so those plugins most certainly are not a cause of this issue.

I also have tried stress testing and opening/closing a lot of about:blank windows and didn't notice memory usage growing too much (about:memory memory consumption is more variable than one about:blank window could leak).

	SthMan
	Comment 9 • 11 years ago

Simonas,

Simple stress testing (eg opening about:blank or Google.com windows) does not seem to cause the memory usage growth. When only opening about:blank or Google.com windows the grabage colleaction seems to reduce the memory usage back to a reasonable amount (eg 200K - 400k) after a few minutes.

It is only when I do actual work (eg GMail, technical web sites, Twitter, Facebook, etc nothing fancy, no games or video intensive stuff) does the memory growth problem occur. Then if I close all tabs/windows apart from one about:blank, the GC does not reduce the memory back to 200K - 400k.  Once the usage is over 1.5Gb, then the situation is hopeless - I have seen FF sit for hours with only about:blank open, and still consuming >2Gb - all that can be done is File->Exit FF and restart.  That is very disruptive to our workflow!

Klaus


PS: Greetings from Sydney to Vil­nius - I was in Lithuania in 2001 - Loved it! :-)

	Liviu Cirdei
	Comment 10 • 8 years ago

Is the issue still reproducible on latest Firefox versions? e.g. Firefox 43 (https://www.mozilla.org/en-US/firefox/new/) and Nightly 46 (https://nightly.mozilla.org/)

	Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Comment 11 • 8 years ago

Yes, it is still reproducible on 43.

At the time of writing (at about 1 day since last restart/OOM) I have 22 windows open and `about:memory` reports about 3GiB of memory usage. I reliably observe memory usage to grow over the course of few days until Firefox is killed by the OS due to OOM. 

I restarted Firefox and waited for all windows to load back up; about:memory  reports 1GiB, which is within reason.

	Nicholas Nethercote [inactive]
	Comment 12 • 8 years ago

Hi Simonas. Can you please collect memory reports again once memory usage gets high? Sorry for asking again, but the measurements in about:memory have changed a lot in the past three years, so looking at old reports is probably not that useful.

Also, can you please list any extensions you have enabled? They can greatly affect memory usage. Go to about:support and click on the "Copy text to clipboard" button. Thank you.

SthMan, if you still have problems can you please file a new, separate bug, with updated measurements, and put "[MemShrink]" in the whiteboard field? It's confusing to have multiple people commenting on a "high memory usage" bug because the root causes may be entirely different. Experience has shown that better results occur when each person having problems files a separate bug. Thank you.

	Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Comment 13 • 8 years ago

Okay, after you posted your request I’ve restarted my browser and begun working. At the time there were 6 windows open and the `top` was reporting ~1GB RES.

The report (reports ~1.8GB) was generated less than a day later. I made sure to close all windows other than the 6 I begun with. I clicked the “minimize memory usage” button before generating the report but it didn’t really do much.

My extensions:

NoScript	2.9.0.2	true	{73a6fe31-595d-460b-a920-fcc0f8843232}
Vimperator	3.10.1.1-let-fixed	true	vimperator@mozdev.org

	SthMan
	Comment 14 • 8 years ago

I can confirm Simonas' findings.  I have done similar tests, but with opening 30 windows. After 1-2 days working with these windows I would observe a memory use of ~ 2GB memory.  I would then close all but one window, run GC / 'minimize memory usage' but would not see a significant reduction in memory usage.

I have made two additional observations:

1: Opening 30 tabs creates less memory usage than opening 30 windows.
2: Completely disabling javascript with 'javascript.enabled' also reduces the memory usage

I hope this helps.

Klaus

	SthMan
	Comment 15 • 8 years ago

Additional Note on Extensions:

Add-ons:
Pagerank for Firefox 1.1.1.1 (disabled)

Appearance:
Default

Plugins:
Shockwave Flash 16.0.0235 (disabled)

	Nicholas Nethercote [inactive]
	Comment 16 • 8 years ago

In Simonas's memory reports, we have this:

> 1,895.11 MB (100.0%) -- explicit
> ├──1,230.40 MB (64.92%) -- window-objects
> │  ├────567.40 MB (29.94%) ++ top(none)/detached

The "detached" entry is indicative of some kind of leak. Simonas, can you try selectively disabling your extensions to see if one of them is at fault? (Extensions are often the cause of this kind of leak.) Thank you.

	Nicholas Nethercote [inactive]
	Comment 17 • 8 years ago

SthMan, as per comment 12, *please* put your data into a new bug. It's very likely your issues have a different root cause to Simonas's, and having comments about both in a single bug is confusing.

	Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Comment 18 • 8 years ago

Right, turns out vimperator is to blame in my case (they even have an issue https://github.com/vimperator/vimperator-labs/issues/253). There’s two questions then:

1. How does one debug this? (I lurk on #firefox and #developers as nagisa{,_,__} and would gladly discuss that)
2. An addon written in javascript (a fully managed runtime from the memory standpoint) is leaking memory – isn’t that the runtime’s (i.e. firefox’s) fault in the end?

	Nicholas Nethercote [inactive]
	Comment 19 • 8 years ago

(In reply to Simonas Kazlauskas [:simukis] from comment #18)
> Right, turns out vimperator is to blame in my case (they even have an issue
> https://github.com/vimperator/vimperator-labs/issues/253). There’s two
> questions then:
> 
> 1. How does one debug this? (I lurk on #firefox and #developers as
> nagisa{,_,__} and would gladly discuss that)

There are some tools for debugging this kind of leak, but they're not easy to use. mccr8, any chance you can take a look at this?


> 2. An addon written in javascript (a fully managed runtime from the memory
> standpoint) is leaking memory – isn’t that the runtime’s (i.e. firefox’s)
> fault in the end?

Possibly. It could just be a bug in Firefox. But it could also be a situation where both "the user (extension) shouldn't do that" and "the platform (Firefox) shouldn't allow that" arguments are reasonable. Without knowing the exact cause in this case it's hard to know where the blame should lie, nor whether laying blame is even useful.

In the past we have found ways to avoid entire classes of memory leaks in extensions. See 
https://blog.mozilla.org/nnethercote/2012/07/19/firefox-15-plugs-the-add-on-leaks/ for the most famous example.

	Andrew McCreight [:mccr8] Assignee
	Comment 20 • 8 years ago

Sure, I'll try to take a look this week. This sounds a little like bug 1174950.

	Andrew McCreight [:mccr8] Assignee
	Comment 21 • 8 years ago

I created a fresh profile, then disabled e10s and installed Vimperator and NoScript. I was unable to reproduce the leak. I tried opening new windows and closing them, or opening windows, loading a simple page, and then closing the window.

Simonas, can you reproduce the issue in a clean profile with Vimperator and NoScript installed? Do you have a more specific set of steps to reproduce this issue? Thanks.

	Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Comment 22 • 8 years ago

> Simonas, can you reproduce the issue in a clean profile with Vimperator and NoScript installed?

Yes. I did `firefox -ProfileManager`, created a new profile, installed only these two addons and I can observe the leak.

> Do you have a more specific set of steps to reproduce this issue? Thanks.

This is step by step of what I did:

1. firefox -ProfileManager;
2. create a new profile, run browser with the newly created profile;
3. install the plugins from mozilla addons website;
4. restart the browser;
5. open a new window using vimperator (w → reddit.com → Enter);
6. in the new window open another window using vimperator (w → reddit.com → Enter), repeat the step a few times;
7. close the windows created in previous steps, in the last open window open about:memory (o → about:memory → Enter);
8. click “minimize memory usage” and “measure” buttons;
9. Observe a few dozens of MB memory reported for the detached window(s).

	Andrew McCreight [:mccr8] Assignee
	Comment 23 • 8 years ago

For anybody else who is looking into this, it sounds like Vimperator is currently broken in Firefox 46 and later ( https://github.com/vimperator/vimperator-labs/issues/398 ) so you have to drop down to Beta or earlier to try to reproduce.

	Andrew McCreight [:mccr8] Assignee
	Comment 24 • 8 years ago

I can reproduce the issue. The nsGlobalWindows involved actually leak all the way to shutdown.

The shutdown path looks like this:

0x1269fe210 [nsXPCWrappedJS (nsINavBookmarkObserver)]
    --[mJSObj]--> 0x127edadc0 [JS Object (Object)]
[...bunch of JS...]
    --[proto]--> 0x11b1f4da0 [JS Object (Proxy)]
    --[private]--> 0x11c5b6100 [JS Object (Object)]
    --[reloadCommand]--> 0x127e3a700 [JS Object (XULElement)]
    --[UnwrapDOMObject(obj)]--> 0x11c3fe790 [FragmentOrElement (XUL) command id='Browser:Reload' chrome://browser/content/browser.xul]

Where the XUL Element is in the document in the window that is leaking. The only thing that seems to hold alive nsINavBookmarkObservers is nsINavBookmarksService, which has an observer-service like thing that can be potentially strong. If I force that to be weak, I get a bunch of errors, but the shutdown leak goes away. However, the non-shutdown leak (where the extra windows are sitting around in about:memory) persists. I've been looking into that but I haven't figured anything out yet.

	Andrew McCreight [:mccr8] Assignee
	Comment 25 • 8 years ago

I found a few reflectors that were marked black, even though I was using an All Traces CC. They are being held alive via paths that look like this:

via PersistentRooted<JSObject*> :
0x111090ec0 [BackstagePass 111472200]
    --[AddonManagerInternal]--> 0x1113c94a0 [Object <no private>]
    --[addonListeners]--> 0x11ca8bb80 [Array <no private>]
    --[objectElements[7]]--> 0x130c6b500 [Proxy <no private>]
    --[private]--> 0x192627740 [Object <no private>]
    --[modules]--> 0x1920be640 [Object <no private>]
    --[statusline]--> 0x1926273a0 [Object <no private>]
    --[_statuslineWidget]--> 0x192275b00 [Proxy <no private>]
    --[private]--> 0x1921d5d90 [XULElement <no private>]

(another had commandline to _completionList to _items to editor instead)

It looks like the addon registered some kind of listener thing.

I think this is where a XUL element gets added to _statuslineWidget:
https://github.com/vimperator/vimperator-labs/blob/c0ce5ca619d33c4c24c4d9ca316ee4e4fb40ba94/common/content/statusline.js#L74

Note the proxy near the end there. You can extend the hueyfix to work on all compartments, and amazingly the browser doesn't completely blow up. This also fixes the leak through _statuslineWidget shown above, but not the _completionList one that goes through nsEditor.

The path for that leak ends up looking like:

0x1867195b0 [JS Object (XPCWrappedNative_NoHelper)]
    --[js::GetObjectPrivate(obj)]--> 0x132adf460 [XPCWrappedNative]
    --[mIdentity]--> 0x12668ff00 [nsEditor]
    --[mTxnMgr]--> 0x132c94b00 [nsTransactionManager]
    --[transaction stack mDeque[i]]--> 0x111baee40 [nsTransactionItem]
    --[mTransaction]--> 0x12a08b900 [EditTxn]
    --[mChildren[i]]--> 0x11cebdc70 [EditTxn]
    --[mChildren[i]]--> 0x1111dbf60 [EditTxn]
    --[mRange]--> 0x127ab1c00 [nsRange]
    --[mOwner]--> 0x11037b000 [nsDocument normal (XUL) chrome://browser/content/browser.xul]

I'd guess there would be some way to purge out editor stuff like this when the window goes away, but I'm not familiar enough with editor to know. This strong path through editor escapes the SuperHueyFix(tm) because there are no CCWs.

Probably the best short-term fix is for Vimperator to do something on their side. I'll try to put together something more useful and post it in the leak Github issue.

	Eric Rahm [:erahm]
	Comment 26 • 8 years ago

Andrew's going to add a note to there github repo and we'll forward to evangelism.

	Andrew McCreight [:mccr8] Assignee
	Comment 27 • 8 years ago

I think the nsINavBookmarkObserver-related leak in comment 24 is because they create a JS book mark observer and then register it strongly:
  bookmarksService.addObserver(observer, false);

	Simonas Kazlauskas [:nagisa/:simukis] Reporter
	Comment 28 • 8 years ago

But then https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIObserverService#addObserver() states that in most cases observers should be registered strongly. Is that not true or is bookmarksService just an exception?

	Andrew McCreight [:mccr8] Assignee
	Comment 29 • 8 years ago

(In reply to Simonas Kazlauskas [:simukis] from comment #28)
> But then
> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/
> Interface/nsIObserverService#addObserver() states that in most cases
> observers should be registered strongly. Is that not true or is
> bookmarksService just an exception?

A strong reference would be okay if the addon was watching for when the window was being closed, and went through and removed all of the references. I don't know what the best practices are for this, really, I just am looking at why things leak or not. The bookmarks service is going to be the same as any other observer service.

	Andrew McCreight [:mccr8] Assignee
	Comment 30 • 8 years ago

I have posted in their issue related to these window leaks. Hopefully they will be able to fix them. It would be nice if we could deal with this in the browser itself, but there are so many ways that chrome can hold onto stuff I don't know if it can be done in general.