Xidorn Quan [:xidorn] UTC+11 Reporter
	Description • 7 years ago

I get a report from my friend who observed large growing memory usage on the main process after keeping Firefox open for a moderate duration (maybe ~10h?).

His memory report (from about:memory) shows that the heap-unclassified is particularly higher than normal.

And eventually, we found that this is extension-related. After disabling some extensions, the number on heap-unclassified drops to a normal level.

According to him, all extensions are WebExtensions. And the extensions which seem to have significant contribution to heap-unclassified includes:
* Ghostery
* Privacy Badger
* Smart HTTPS

He told me that some of the extensions also take lots of memory on Chrome, so taking memory itself is probably less concerned. But having their memory usage number inside heap-unclassified is not helpful anyway.

It should be easy to tell from memory report that what extension is using the memory.

	Eric Rahm [:erahm]
	Comment 1 • 7 years ago

Xidorn, can you get an about:memory report? Was the heap-unclassified in the web extension process or the main process?

Kris do you know who could look into improving the memory reporters on the WebExtension side? It seems like a DMD run could help here.

	Kris Maglione [:kmag]
	Comment 2 • 7 years ago

I'd have to have reliable steps to reproduce the issue first. Since bug 1382645 was fixed, we really don't use large objects that wouldn't already show up in an existing memory reporter.

The only exotic objects we really use at this point are:

- WebExtensionPolicy, WebExtensionContentScript, MatchPattern, MatchGlob: Simple policy objects that are mostly created at startup, and then stick around for the session. They're pretty tiny, too. I suppose it's possible that we could wind up accumulating a bunch of WebExtensionContentScript objects if extensions use `tabs.executeScript` a *lot*, and especially if they leak somehow.

- PrecompiledScript: A simple holder class for extension content scripts so they don't need to be re-compiled each time they're executed. These don't have memory reporters of their own, but the scripts they hold show up in the memory reporter for the compartment that created them.

- nsIPreloadededStyleSheet: Pre-loaded CSS stylesheets for extension content scripts. These should already be reported by the style loader's memory reporter.

- WebRequestService::ChannelEntry: Records of channels extensions are allowed to monitor the contents of. These have only been around for a few days, are tiny, and don't stick around for long.


Of the above, only nsIPreloadedStyleSheet and WebExtensionContentScript really seem like plausible contenders, and even then, only if we're doing something wrong.

	Xidorn Quan [:xidorn] UTC+11 Reporter
	Comment 3 • 7 years ago

The friend is using a release version of Firefox, so it is in the main process. I can attached the memory report if you think it is useful. But it just shows something like 800MB+ heap-unclassified in the main process, so I guess it isn't really that useful.

	Xidorn Quan [:xidorn] UTC+11 Reporter
	Comment 4 • 7 years ago

So given comment 2, I guess it could be something which is no longer a problem after 56.

	Kris Maglione [:kmag]
	Comment 5 • 7 years ago

(In reply to Xidorn Quan [:xidorn] UTC+10 from comment #4)
> So given comment 2, I guess it could be something which is no longer a
> problem after 56.

That seems fairly likely. We started using StructuredCloneHolder objects to transport extension messages in 55, and they had no memory reporters until 56.

Prior to bug 1389968 and bug 1391353, it was fairly easy for extensions to leak those objects until a page closed by promising to reply to a message and never actually doing so. You'd think that wouldn't be very common, but I've recently come across multiple extensions that not only do that, but do it for *a lot* of messages...

	Mathieu Marquer
	Comment 6 • 7 years ago

Hi, I'm also encountering large heap-unclassified usage (3 GB just a few seconds ago) on Firefox Nightly, should I open a new bug or do you think it can be related to this one? I have extensions, but none of those mentioned above.

	Mathieu Marquer
	Comment 7 • 7 years ago

The about:memory summarized:

3,308,185,577 B (100.0%) -- explicit
├──2,889,980,332 B (87.36%) ── heap-unclassified
├────166,988,088 B (05.05%) ++ window-objects
├────130,941,336 B (03.96%) ++ js-non-window
├─────30,746,080 B (00.93%) ++ storage
├─────16,143,408 B (00.49%) ++ heap-overhead
├─────13,539,968 B (00.41%) ++ workers/workers(chrome)
├─────12,833,488 B (00.39%) ++ dom
├──────7,204,192 B (00.22%) ++ xpconnect
├──────6,258,976 B (00.19%) ++ cycle-collector
├──────6,227,728 B (00.19%) ++ images
├──────5,747,824 B (00.17%) ++ atom-tables
├──────4,788,192 B (00.14%) ++ network
├──────4,331,625 B (00.13%) ++ startup-cache
├──────2,547,456 B (00.08%) ++ add-ons
├──────1,895,488 B (00.06%) ── profiler/profiler-state
├──────1,608,000 B (00.05%) ++ gfx
├──────1,522,256 B (00.05%) ── cookie-service
├──────1,334,352 B (00.04%) ++ layout
├──────1,028,192 B (00.03%) ── icu
├────────893,168 B (00.03%) ── xpti-working-set
├────────811,248 B (00.02%) ── preferences
├────────404,768 B (00.01%) ++ script-preloader
├────────254,448 B (00.01%) ++ xpcom
├─────────99,312 B (00.00%) ++ data-storage
├─────────38,612 B (00.00%) ── telemetry
├─────────11,584 B (00.00%) ++ url-preloader
├──────────4,864 B (00.00%) ── script-namespace-manager
├────────────592 B (00.00%) ── history-links-hashtable
├──────────────0 B (00.00%) ── media/libogg
└──────────────0 B (00.00%) ── spell-check

Reinstalling + fresh profile + Firefox Sync didn't help. Only happens on my Windows 7 computer from work, not on my Ubuntu Linux computer from home. When this happens, memory usage can go up by about 100 MB every second.

	Kris Maglione [:kmag]
	Comment 8 • 7 years ago

(In reply to Mathieu Marquer from comment #6)
> Hi, I'm also encountering large heap-unclassified usage (3 GB just a few
> seconds ago) on Firefox Nightly, should I open a new bug or do you think it
> can be related to this one? I have extensions, but none of those mentioned
> above.

(In reply to Mathieu Marquer from comment #7)
> Reinstalling + fresh profile + Firefox Sync didn't help. Only happens on my
> Windows 7 computer from work, not on my Ubuntu Linux computer from home.
> When this happens, memory usage can go up by about 100 MB every second.

That's pretty bad... What extensions do you have installed?

	Mathieu Marquer
	Comment 9 • 7 years ago

au-revoir-utm, Pushbullet, uBlock Origin.

	Andy McKay
	Comment 10 • 7 years ago

Dropping priority because we haven't been able to reproduce yet. Sounds like Krupa's team already had a look at this one.

	marius.santa
	Comment 11 • 7 years ago

Tested with all the mentioned webextensions installed on multiple stations, Windows 10 and Windows 7, 6+ hours and other sites opened, unable to reproduce, added the memory logs from my machine.

	Andy McKay
	Comment 12 • 7 years ago

I'm curious if this has changed in Firefox 57, a few of these extensions have released new versions. Sadly we haven't been able to reproduce this yet.

	Xidorn Quan [:xidorn] UTC+11 Reporter
	Comment 13 • 6 years ago

My friend told me this seems to have been fixed.

	Andy McKay
	Comment 14 • 6 years ago

Closing given the inability to reproduce and the original reporter saying that it seems to be have fixed. Thanks Xidorn.