Closed Bug 1436220 Opened 6 years ago Closed 6 years ago

Gigabytes of heap unclassified in main process

Categories

(Firefox :: Untriaged, defect)

58 Branch
defect
Not set
normal

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: likalex3, Unassigned)

References

Details

(Whiteboard: [MemShrink:P3])

Attachments

(3 files, 1 obsolete file)

Attached file memory-report.json.gz (obsolete) —
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0
Build ID: 20180128191252

Steps to reproduce:

To reproduce, you need to use Firefox (Version 58.0.1 64-bit) for around 3-4 hours. Open multiple tabs and have sites like YouTube or a stock ticker open (sites that aren't static) so you're continuously loading new data.


Actual results:

After about 3-4 hours Firefox's RAM and CPU usage hit 90-100%. The browser is no longer able to load new webpages or refresh existing ones. They will load indefinitely. This has affected all three of my Windows 10 computers using Firefox as well as all of my coworkers using Firefox as their main browser.


Expected results:

Firefox should not try to allocate all of my computer's resources.
Whiteboard: [MemShrink]
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0

I've tested this issue Windows 10 x64 with the latest Firefox release (58.0.2) and the latest Nightly (60.0a1-20180209005628) and haven't managed to reproduce the issue.
After loading multiple tabs (20) youtube, vimeo pages and also other webpages like facebook, twiiter, etc. The RAM memory and CPU usage was below 20%. No hangs, latency or other issues occured.

Can you please specify the number of tabs you have had loaded?

Could you also please retest this using the latest Firefox release and latest Nightly build and report back the results? When doing this, please use a new clean Firefox profile, maybe even safe mode, to eliminate custom settings as a possible cause (https://goo.gl/AR5o9d).
Thanks Emil. The number of tabs varies. It's generally 10+

I'm testing on safe mode now. I'll report back as soon as I know if this fixes it. If I do run into this again, is there any specific info you would need?
Flags: needinfo?(likalex3)
Attached image RAMandCPU.png
Running in safe mode prevents FF from freezing, but it still gradually eats through my RAM and CPU until it hits 100%. This is in safe mode. I tried on my home desktop and laptop with the same results.
Flags: needinfo?(likalex3)
(In reply to Arseny from comment #5)
> I reported this in another branch already but want to repeat it here: I am
> having the same problem with 58.0.2 I am on Win 7 64bit. At different times
> I have 30-50 tabs opened (mostly text pages with minimum graphic content)
> and adblock origin active (no other addons, default profile etc). Leaving
> this overnight leads to massive amounts of RAM taken by firefox (one process
> being a gig or few gigs, others ~400-300mb each). If left open it sooner or
> later eats all RAM available slowing PC to a crawl. If I close firefox and
> then reopen it restoring all tabs it only takes ~600mb total (250mb one
> firefox process and others are about 100mb each).

Please file a separate bug.
The memory report seems to be broken. Can you take a memory report before you see issues, wait until things get bad and take another memory report we'll be able to diff them and see what's going on. Also if you feel comfortable can you provide a link to the stock ticker you are using?
Flags: needinfo?(likalex3)
Yep, not a problem.

Just a quick update to go along with my memory report. I decided to switch to an another laptop. I performed a factory restore of Windows 10 and a clean install of FF x64. The CPU and RAM issue returned immediately, despite the clean install. I've used safe mode and regular browsing mode.

Each time FF eats up all my CPU and RAM I'm browsing a different set of sites. I haven't used the stock ticker since Monday, just to see what happens.

The only constant that comes to mind is that I'm using FF x64. I have not tried the x86 version yet.

The stock ticker I use is Yahoo! Finance
finance.yahoo.com

Please let me know if you need anything else.
Flags: needinfo?(likalex3)
Attached file memory-report.json.gz
Also feel free to shoot over any troubleshooting steps you think might help. I used to work in IT, so nothing is too technical.
Hy Alex, thanks for providing a new memory report in comment 9. 

However, I see that Eric is currently away until 3/16, so in the meantime could you also please provide a performance report using the https://perf-html.io/ tool.

Just in case further information regarding the memory leak and cpu usage could show up more easily in this kind of performance report.
Flags: needinfo?(erahm)
Flags: needinfo?(likalex3)
Since Eric is not around. Mike, can you please take a look and see if there is anything actionable here?
Flags: needinfo?(mconley)
I see a ton of heap unclassified in the parent process in that memory report... I think in normal circumstances, the next step is to try to get data from a dark-matter-detection build, but I'm not super sure how that works. I'm going to redirect to mccr8.
Flags: needinfo?(mconley) → needinfo?(continuation)
Attachment #8948849 - Attachment is obsolete: true
Flags: needinfo?(continuation)
Summary: Memory Leak 58.0.1 (64-bit) → Gigabytes of heap unclassified in main process
Yeah, there's gigabytes of heap-unclassified in the main process, which is really bad. On Nightly, enabling WebRender can cause that much memory to leak, but you can't enable that on release, so that can't be the issue.
Flags: needinfo?(erahm)
Whiteboard: [MemShrink] → [MemShrink:P3]
Flags: needinfo?(mconley)
Hey mccr8, what's the easiest way for Alex to get a hold of a DMD build? Do we have to spin up one from try?
Flags: needinfo?(mconley) → needinfo?(continuation)
I don't know, sorry.
Flags: needinfo?(continuation)
Last time I asked Eric, I think he said there's no really good solution, because you have to symbolicate the report locally, so I'm not sure a try build will do the trick. Well, maybe a try build will have the dmd.py script, and the reporter could run it? Maybe that would work.
Flags: needinfo?(mconley)
The build would have dmd.py, they probably wouldn't be able to symbolicate but we could do that on our end.
Hi Alex, are you still following this bug and able to reproduce this issue? If so, we have some debugging steps for you to try.
Flags: needinfo?(mconley)
Considering the fact that I cannot reproduce this and the fact that the reporter did not answered to the needinfo request until now, I will mark this as Resolved-Incomplete.

If anyone can still reproduce it, feel free to reopen the issue and provide more information.
Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → INCOMPLETE

While I can't speak for the original poster, I have seen this issue for over a year and have had to sideline Firefox support and use of Firefox Developer Edition of my companies Single Page Application.

To kick this ticket back off, some details that I can provide about the application are:

  • Uses WebSocket
  • Uses WebGL (via TensorFlowJS)
  • Uses React / Redux (latest versions)

Symptoms:

  • Open a tab to above app
  • Refresh tab a few times
  • Memory slowly grows with each refresh
  • On Ubuntu Linux, the eventual high memory usage crashes the entire computer to the point where I have to do a hard reset

I would be happy to try the debugging steps and get this resolved or at least shed some light on whats happening.

Given that Cameron M seems to have a reliable way of reproducing this, is there a way we can get him an instrumented build to help us determine what this heap unclassified stuff is? Do we have any recent DMD builds around that he could use?

Flags: needinfo?(continuation)

I think Nightly builds are all DMD now, so it is just a matter of passing the right argument when running it.

Flags: needinfo?(continuation)

Also, we do have some frequent intermittent leaks on TreeHerder involving websockets, so maybe that's related...

Is Firefox Developer Edition similar to a Nightly build?

Hmm I guess I'm not sure how the symbolizing works in that case.

Eric, is there some easy way for a user to use DMD if they are experiencing issues, or is that not practical? Thanks.

Flags: needinfo?(erahm)

(In reply to Andrew McCreight [:mccr8] from comment #27)

Eric, is there some easy way for a user to use DMD if they are experiencing issues, or is that not practical? Thanks.

devedition is roughly beta but with a darker theme and a separate profile, it doesn't have DMD as an option. You'd need to run Nightly. If you're on Ubuntu this is probably the easiest way to do that programmatically:

  1. Install nightly and run it with DMD enabled
# setup a temporary environment for testing
cd /tmp
mkdir repro && cd repro
virtualenv test-nightly && source test-nightly/bin/activate

# Use helpers to download and install a nightly
pip install mozdownload mozinstall mozrunner
mozdownload -t daily && mozinstall *.bz2

# Run nightly with DMD enabled in a clean profile
DMD=1 mozrunner -b firefox/firefox
  1. Do whatever causes high heap-unclassified -- check in about:memory and note what the PID is
  2. From about:memory under Save DMD output click Save
  3. You should now have several dmd-...-json.gz files under /tmp
  4. Post-process the DMD file and attach the dmd.txt file here
# Run the dmd.py script from the firefox install, this will attempt to cleanup stacks so it might take a while
firefox/dmd.py -f24 -a -o dmd.txt /tmp/dmd-*-<PID>.json.gz
Flags: needinfo?(erahm)

Thank you Eric, I will try to run this today and post the text file that is generated.

Attached file cameron-ubuntu-dmd.txt

Here is the dmd.txt file from the session.

I have some more data which is more useful - when the Developer Tools were closed, the issue did not occur, and occasional spikes in memory were cleaned up in less than 15 seconds back to a typical baseline throughout my application.

However, when the Developer Tools were open, I saw the unclassified heap consistently grow and never release memory. This is the typical use case for my usage of the Developer Edition and vanilla Firefox, so it makes sense. I should note that since this was using a clean nightly build with a new profile, no developer extensions where enabled.

Thanks for the report. It looks like a lot of the unreported memory is allocated under WebGLShader::CompileShader(). Jeff, can you take a look please? Thanks.

Cameron, if you go to about:memory and click on Minimize Memory Usage, does that help? (It is possible that some garbage is hanging around that is keeping all of these WebGL data structures alive, because the GC isn't running because it isn't aware of the information being retained.)

Flags: needinfo?(jgilbert)

This should probably get filed as a new bug. Piling in on an existing bug just makes it confusing for people to figure out what is going on.

Flags: needinfo?(likalex3)
See Also: → 1319426

Moving this discussion to the seemingly-related bug 1319426.

Flags: needinfo?(jgilbert)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: