Dan D (volunteer tester from the community) Reporter
	Description • 6 years ago

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0
Build ID: 20171226083017

Steps to reproduce:

My parent has 26 tabs open...


Actual results:

The computer slows down and is almost unusable.

I troubleshoot this exactly the same way each time. I restart the browser for my parent, by using the "Restart normally..." button in about:profiles, so that the tabs are unloaded from memory. This makes the computer work fine. Then my parent gradually has to access more of those 26 tabs to do their work, so more and more memory gets used. It eventually slows down again, so a week later I am called over to help once again.

I have tried to teach my parent to unload the tabs by restarting Firefox (easiest to do in "about:profiles")... but this is kind of a pain for them and it's looking like we may have to buy a new computer.

Here is the scenario in more depth: 

My parent often asks me to check on their computer, saying it has slowed to a crawl. (when I get there, it definitely has slowed to a crawl. Almost nothing works, the latency of trying to do tasks is several seconds between a typical click and the desired effect.)

Every time I check in on their computer, I first open Task Manager, and I see 100% disk usage and ~90% memory usage. In Task Manager, Firefox shows as using about 1,200 MB of my parent's 4GB installed ram. The next biggest RAM-using program might have about 120MB or as low as 60MB.

I think Windows is trying to swap memory out to disk, and/or compress memory to save itself from running out of usable memory. On this particular laptop, that slows everything down by a lot.

Again, restarting Firefox (effectively unloading all the tabs from memory) fixes this problem right up. Disk usage goes back comfortably below 100%. Memory usage plummets to around 50% (note: Windows always tries to have about 50% of memory occupied, even if you're not actively using it, by filling it with data you access often. This is called Superfetch, and it's Microsoft's take on prefetching data for speed of use.)

By the way, I have also tried the following:

I refreshed the profile my parent uses. (And restored their preferred homepage, preferred update setting (i.e. "check but notify before updating".) (Parent is on 57.0.3 anyway.)

I lowered the number of content processes to 1

I overrode automatic cache management, and set maximum web content cache to 128 MB.

I did these in sequence, one change at a time, and it doesn't seem to have helped or made any change to the slow-downs.

My parent's computer is a Dell laptop, intel T4500 processor, 4GB of RAM. The hard-drive probably isn't terribly fast I reckon. (Yes, this is a 7 year old computer, but we're environmentally conscious and don't want to upgrade more than we have to. In all respects, for my parent's use-cases, the computer is good enough other than for Firefox.)


Expected results:

Even though Firefox is straining this system's available memory, given the amount of tabs open, my hope is that my parent's computer should still function when Firefox is using lots of RAM. Their computer should not slow down so intensely.

(Again, Firefox is the only major RAM user I can find on this machine, and I've tried to clean up weird pre-installed or junk software where possible.)

(Thank you for your responses if any. I can work with you as much as is needed to help with this issue. I understand this would probably take a lot of work to address, but we've been through this enough times that I promised my parent I would file a bug report about it. Hope you are all having a good new year so far! Thanks again.)

	Dan D (volunteer tester from the community) Reporter
	Comment 1 • 6 years ago

Attachment shows very high disk usage by Firefox and "System" (which is the kernel, I believe) processes.

Lots of swapping to/from disk going on.

	Dan D (volunteer tester from the community) Reporter
	Comment 2 • 6 years ago

I was thinking of what might solve this.

1) The obvious solution: close some tabs. (I'll just say convincing my parent to close any of these tabs is quite hard.)

2) Some sort of smartphone-style memory management. Unload most of the data from background tabs as needed, clearing up RAM for the foreground activity. 

(Can something be ported over from the Android or iOS Firefox apps, so that the desktop Firefox can unload background content just like the Android/iOS apps do? It could even be behind a pref. That would be most helpful to folks like my parent, so long as non-technical users could access the setting. So I'd hope it would be accessible in the about:preferences screen somewhere. I personally would be able to set it in about:config for my parent, but they wouldn't know how to change the setting themselves without my writing a little tutorial for them.)

	Dan D (volunteer tester from the community) Reporter
	Comment 3 • 6 years ago

By the way, this seems to be a regression when compared to 52.5.3 ESR.

Parent has been switched over to Firefox 52 ESR for a few days now, and has reported no complaints. Parent says performance is much better, and the issue is gone.

I note that Firefox's max memory usage with all tabs loaded, in Firefox 52 ESR, was around 1,240 MB. (Compare: max memory usage in Firefox 57 was about 2,700MB). 

CPU usage in 52 ESR is consistently around 30-60% of what the T4500 CPU can do, even with the browser at "idle," which seems higher like higher than the average CPU usage for 57... But it's still comfortably below 100% most of the time. Firefox 52 ESR has minor slowdowns when loading a new tab, but compared to swapping to disk and compressing, I'd take the tiny and frequent performance hits in 52 over the massive and inevitable slowdown when running out of memory with 57. (Parent agrees!)

Anyway, thanks for looking into this. I hope the details I've provided will be helpful. (I'm a big fan of Firefox 57, but my parent's laptop is not, apparently.)

	YF (Yang)
	Comment 4 • 6 years ago

"Aw, snap! Google Chrome ran out of memory while trying to display this webpage".

	Eric Rahm [:erahm]
	Comment 5 • 6 years ago

If possible can you attach an about:memory report from 57/58 when they see this issue? It might help us track down what's going on. It would also be helpful to get a list of addons from about:support.

	Dan D (volunteer tester from the community) Reporter
	Comment 6 • 6 years ago

Hi.

Add-ons are only:

- Those enabled by default, specifically:

- WideVine CDM

- Open H.264

- Adobe Flash 28

- (No Extensions, just these plugins.)

(If you need technical specifics, I have the details and can post them, just let me know.)

(Profile has been refreshed a few times in the process of trying to reproduce this bug, so it's pretty stock-standard.)

I switched back to 57 to try to reproduce, but couldn't. Updated to 58, also couldn't reproduce.

Parent has been using 58 for three days so we could see if it was safe to go back to a recent FF version. Parent has said computer only slowed down momentarily once or twice (which is pretty normal, every computer slows down momentarily, we both agreed.)

Trying my best to push things to the limit, I was then able to reproduce the slowdowns by opening a heavy site parent had closed out of since initially running into the slowdowns... Allrecipes.com; I loaded the homepage of Allrecipes.com in a tab, and 3-to-5 recipes in their own tabs, in addition to the ~24 tabs parent had open, and the slowdowns happened again. I feel like the slowdown was less intense this time, easier to generally operate the computer throughout, but it got kinda sluggish at worst, and indeed disk usage showed as hovering around 94 to mostly 100% in Task Manager.

Going to attach some anonymized memory reports. (If non-anonymized would be helpful, I can send them via e-mail.)

	Dan D (volunteer tester from the community) Reporter
	Comment 9 • 6 years ago

To clarify, this is still *reliably* reproducible on this particular laptop, by loading all tabs Parent has been using, PLUS the Allrecipes.com homepage, PLUS three or more recipes at Allrecipes.com in their own tabs. That's roughly 26 tabs total.

Parent has closed heavy tabs at some point, and is no longer organically running into this issue, but when I artificially try to ramp up memory usage on purpose, (exactly as described above in this comment) I can get the whole-machine slowdowns to re-occur every time. 

(The slowdown is perhaps less intense than reported in comment #0. But I suppose it may be because I am not replicating how heavy my parent's inital tab set was. Or because 58 is improved somehow over 57. Or because my parent may have had a Windows update since then. Or etc... I'm not sure why, I can only observe and speculate.)

	Eric Rahm [:erahm]
	Comment 10 • 6 years ago

The first set of reports seems broken/incomplete. Looking at the second set (after minimize) we have 34GB of vsize in the main process:

>     34,839.05 MB ── vsize
>        207.69 MB ── resident-unique
>         10.07 MB ── shmem-allocated
>         10.19 MB ── shmem-mapped
>          7.69 MB ── system-heap-allocated

The rest of the numbers look okay. Given this is a 64-bit build, that's not the end of the world, but it's still pretty crazy. We can see how this breaks down:

> 134,217,727.94 MB (100.0%) -- address-space
> ├──134,182,884.82 MB (99.97%) ── free(segments=401)
> └───────34,843.12 MB (00.03%) -- (2 tiny)
>         ├──34,193.57 MB (00.03%) -- reserved
>         │  ├──34,172.02 MB (00.03%) ── private(segments=2219)
>         │  └──────21.55 MB (00.00%) ── mapped(segments=4)
>         └─────649.55 MB (00.00%) -- commit
>               ├──250.09 MB (00.00%) ++ image
>               ├──244.35 MB (00.00%) ++ private
>               └──155.11 MB (00.00%) ++ mapped

So all that said I don't have any great solutions, but it appears as if we have a potential *virtual* memory leak in the main process. It's also pretty clear the system was in a bad state when the initial memory report was taken which led to an incomplete set of reports.

There are a few rather large changes going from 52 to 57, so I don't think we can really call this a regression. Process-wise we went from a single main process, to a main process plus 4 content processes by default. We added a dedicated WebExtension process, this will be loaded if you have activity streams enabled (which ships by default). Additionally on Windows there is is a GPU process. Setting the content process count to 1 would still mean there are 4 processes running vs 52's single process.

A few things I might suggest:

#1 - A Firefox refresh, this will reset some settings among other things [1]. It's possible you already did this, but I wasn't 100% sure.
#2 - Try dropping the number of content processes to 2 (the T4500 is a dual core)
#3 - Try out tracking protection [2], this often speeds things up and reduces memory usage. Note: this may break some sites so it may not be the best solution for you.

Hopefully that finds a happy medium for your parents where they keep plenty of tabs open but not experience too many slowdowns. Unfortunately loading a lot of content heavy tabs can still slow things down, particularly on older hardware.

One thing that might also help is getting us a performance profile [3] when they experience a slow down.

[1] https://support.mozilla.org/kb/refresh-firefox-reset-add-ons-and-settings
[2] https://support.mozilla.org/kb/tracking-protection
[2] https://developer.mozilla.org/docs/Mozilla/Performance/Reporting_a_Performance_Problem

	Dan D (volunteer tester from the community) Reporter
	Comment 11 • 6 years ago

> Hopefully that finds a happy medium for your parents where they keep plenty of tabs open but not experience too many slowdowns.

We've found that "Just closing Allrecipes.com when you're done with the recipe" does a pretty good job of avoiding slowdowns. Heh.

Regarding the suggestions:

I have been doing a "Firefox refresh" many, many times.

If we can avoid Tracking Protection, I would like to. I know I personally don't like starving news sites of ad revenue, but I'd let my parent make the call whether that was worthwhile... Interesting suggestion.
 
Dropping content process count didn't seem to do much the first time around, but it's worth another try if nothing else works.

On another subject:

Can I just report that saving/opening memory profiles is really awkward on Windows?

When I hit save, the "save file" dialog gives me just "memory-report.json" as the suggested filename, even though the filetype filter of the save file dialog suggests I will end up with a "zipped JSON" file. Sure enough, the file saves what is truthfully a zipped archive of a JSON file as "memory-report.json", and Windows hides the extension, making it hard to fix it from ".json" to ".json.gz".

I ended up saving all my memory reports, putting them on a cloud storage app, downloading them onto Linux, and changing the file extension there. Then the actual ".json" file could be extracted from the archive. (Windows doesn't come with any means of opening a .gz file, though! So when trying to fix this on Windows, I had no way of knowing if I had done the right thing without jumping over to a Linux machine (or installing 7zip on my parent's computer.) And I think I tried ".json.zip" instead of ".json.gz", and that didn't work.)

	Nicholas Nethercote [inactive]
	Comment 12 • 6 years ago

> Can I just report that saving/opening memory profiles is really awkward on
> Windows?
> 
> When I hit save, the "save file" dialog gives me just "memory-report.json"
> as the suggested filename, even though the filetype filter of the save file
> dialog suggests I will end up with a "zipped JSON" file. Sure enough, the
> file saves what is truthfully a zipped archive of a JSON file as
> "memory-report.json", and Windows hides the extension, making it hard to fix
> it from ".json" to ".json.gz".

I just tried and there actually is a .gz extension -- it shows up in a Mingw shell as memory-report.json.gz. But you're right that Windows hides this well. If I right-click on the file icon and choose "Properties" I get a dialog that says the "Type of file" is "8 Zip archive (.gz)". (Furthermore, 8 Zip Lite is able to open it, though perhaps not everybody has that program.)

So this seems like more of a Windows issue than a Firefox issue, I think, given that the .json.gz suffix is there but Windows is hiding it?

	Dan D (volunteer tester from the community) Reporter
	Comment 13 • 6 years ago

Yes, it is a Windows issue in some sense, particularly if the Firefox team is wedded to using a .gz extension.

However, if it's possible to just save as a .zip on Windows, that would be MUCH less awkward. (If users have installed a third-party software like 7-Zip, PeaZip, unRar, etc. then it's no problem. But if you just have the default MS software it really feels like the about:memory page is "not for you" Windows users, in my opinion.)

Glad to know the extension was there all-along. But once again, if you haven't installed any third-party software, Windows doesn't know what to do with it. It can't be opened, and the File Explorer does show it as "[filename].json", which makes the situation into a real head-scratcher. (about:memory also refuses to load the .gz version; it errors out unless you extract the plain .json from the archive and load that. So windows users without third-party software can't view their own saved reports.)

I feel like saving as plain-old .zip, as a web browser, is not a hard problem, but for users who've never heard of "Gzip", opening a .gz on Windows is a comparatively hard problem.

	Nicholas Nethercote [inactive]
	Comment 14 • 6 years ago

> about:memory also refuses to load the .gz version; it errors out unless you extract the plain .json from the archive and load that. So windows users without third-party software can't view their own saved reports.

Hmm... if you accept the default name in the save dialog then the .gz suffix is added and you can load the file in about:memory. But if you choose a different name then the .gz suffix doesn't get added. That is definitely odd behaviour.

	Dan D (volunteer tester from the community) Reporter
	Comment 15 • 6 years ago

Here are some Perf Profiles:

Long (over 1 min):
https://perfht.ml/2nsWVes

Medium:
https://perfht.ml/2nrHkf8

Short:
https://perfht.ml/2nx7e1h

Really Short (~10 seconds):
https://perfht.ml/2nq1zKf

	Dan D (volunteer tester from the community) Reporter
	Comment 16 • 6 years ago

And here's another `about:memory` report showing [what I think is] a lot of virtual memory, again.

	Dan D (volunteer tester from the community) Reporter
	Comment 17 • 6 years ago

Here's another Task Manager screenshot that I thought was odd.

I'm not sure exactly what this means, but Task Manager shows some form of "memory" that is 8.4 Gigabytes used out of 10.1 Gigabytes.

(There is only 4GB of physical RAM on the laptop.)

This figure was a bit dynamic over time. It changed both in how much was on the left side of the slash, and on the right of the slash.

(Fluctuated from ~ 6.9 to ~8.6 on the left; Right side fluctuated between ~9 GB to ~10.4 GB over time)

	nivtwig
	Comment 18 • 6 years ago

allrecipes.com has a auto-playing videos on some of the recipes, maybe this is the cause of the problem, if there is a memory leak or unreleased memory in playing the videos.

Once you start watching a video of a recipe and finish watching it, another video loads and plays, and another, enldlessly in the background, even though the user might have switched to another tab .

	nivtwig
	Comment 19 • 6 years ago

(In reply to nivtwig from comment #18)
> allrecipes.com has a auto-playing videos on some of the recipes, maybe this
"a auto-playing" should be "auto-playing"

	Dan D (volunteer tester from the community) Reporter
	Comment 20 • 6 years ago

About those perf profiles:

Slowdown wasn't reproducible at first try. I first tried restarting Firefox, and slowly focusing (loading) each of ~24 open tabs one after the other. Then I additionally loaded the "Allrecipes.com" homepage, and 5 recipes from off of the homepage, each in their own tabs. The computer didn't slow down noticeably, and everything was fine (other than making it hard to reproduce this bug). Then I tried opening the "Amazon.com" homepage, the "Youtube.com" homepage, and probably a couple of other heavy-ish sites that I can't remember. For some reason it wasn't slowing down at all?

Then I switched to another open window, which happened to be Microsoft Word. Suddenly the disk activity of the computer skyrocketed, and everything got sluggish as reported in comment #0. To maximize this sudden increase in disk usage, I also switched to some other applications that had been open the whole time, namely Microsoft Outlook, Apple Software Update, and a couple of PDFs in Adobe Acrobat Reader DC. It was after doing all this that I took the perf profiles.

Note: Taking perf profiles when the disk is at constant 100% disk usage takes pretty much forever, so I had to close a lot of tabs after I had hit the "capture" button in the per-html extension. Otherwise, the captured profile would never load on my local machine, and I could never hope to upload it. That meant I had to close heavy tabs after hitting "capture," wait for profile to "symbolicate," upload the profile. To reproduce slowdowns again, I had to open heavy tabs again, switch amongst open applications for good measure (didn't seem to make a diffference at this point?) i.e.:(Outlook, Word, Apple Software Update, Acrobat Reader DC) and then I could start running another perf profile capture. The "Long" profile was taken first, then "Medium," then "Short," then "Really Short."

Speculating: So I think Windows had stowed away the other open programs' memory into a page file or something, to make room for Firefox. When I actually accessed those other programs, they needed their RAM space back in physical memory, and it was all being used by Firefox, so some mad and frantic disk activity was required to shuttle things to/from the page file.

Disclaimer: I'm not an engineer by training, so I'm just making educated guesses.

	nivtwig
	Comment 21 • 6 years ago

IMO Taking performance profiles here is pointless, since the basic problem is the huge memory use of Firefox, that triggers the use of the swap file for swapping memory to and from disk when there is not enough memory. 

Disk accesses are very slow vs memory by around 1:1000000 , and can slowdown the computer to a crawl when using the swap file heavily (around 10ms for random disk access, or only 100 times a second, vs several nanoseconds for memory)

	Jim Mathies [:jimm]
	Comment 22 • 6 years ago

Dan, can you post the about:support text for your parent's system? Also, do you know if they run any type of annti-virus software on that device?

	Dan D (volunteer tester from the community) Reporter
	Comment 23 • 6 years ago

Jim, I can provide this information, but I'd prefer to send it privately.

How can I do that?

	Dan D (volunteer tester from the community) Reporter
	Comment 24 • 6 years ago

Can I e-mail this info to someone on the Firefox team? I would not mind providing the info, but not attached at a publicly accessible website such as bugzilla.

	Jim Mathies [:jimm]
	Comment 25 • 6 years ago

(In reply to Dan D (Not an employee, just a tester) from comment #24)
> Can I e-mail this info to someone on the Firefox team? I would not mind
> providing the info, but not attached at a publicly accessible website such
> as bugzilla.

sure, you can email me: jmathies@mozilla.com. please reference this bug number in the subject.

	Dan D (volunteer tester from the community) Reporter
	Comment 26 • 6 years ago

Thanks. An e-mail was sent with the requested info.

If anything else is needed, just let me know and I'll do my best to get it.

	Dan D (volunteer tester from the community) Reporter
	Comment 27 • 6 years ago

I noticed just recently my Parent had 32-bit Anti-virus installed. Working on getting it replaced with a 64-bit AV program as soon as Parent is at a convenient place in their work to reboot the computer. Not sure if the 32-bit AV program has anything to do with this bug/issue. 

I am also working on convincing my Parent to allow a backup/re-install of Windows 10 to see if that fixes the occasional performance quirks of this machine.

The mega slowdowns are a "once every couple of weeks" problem still, but we are getting used to doing the workaround (close out or re-start Firefox).

	Jonathan Watt [:jwatt]
	Comment 28 • 6 years ago

Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)