Ben Kelly [:bkelly, not reviewing] Reporter
	Description • 7 years ago

While we've fixed a number of memory issues recently, I'm still seeing elevated "private bytes" in the parent process.  For example:

          0.00 MB ── gfx-d2d-vram-draw-target
          0.00 MB ── gfx-d2d-vram-source-surface
          5.76 MB ── gfx-surface-win32
          0.00 MB ── gfx-textures
          0.00 MB ── gfx-textures-peak
          0.00 MB ── gfx-tiles-waste
                0 ── ghost-windows
          0.00 MB ── gpu-committed
          0.00 MB ── gpu-dedicated
          0.00 MB ── gpu-shared
        167.69 MB ── heap-allocated
          1.00 MB ── heap-chunksize
        318.00 MB ── heap-mapped
              114 ── host-object-urls
          8.44 MB ── imagelib-surface-cache-estimated-locked
          8.44 MB ── imagelib-surface-cache-estimated-total
                0 ── imagelib-surface-cache-overflow-count
          1.75 MB ── js-main-runtime-temporary-peak
      1,324.19 MB ── private
      1,481.65 MB ── resident
      1,319.93 MB ── resident-unique
         99.09 MB ── shmem-allocated
         99.15 MB ── shmem-mapped
          2.98 MB ── system-heap-allocated
      3,299.46 MB ── vsize
132,179,771.69 MB ── vsize-max-contiguous

Our entire heap is 318MB, but the private bytes are and unique RSS are over a GB.

From what I have read private bytes is memory that has been reserved, but may not actually be used.  For example, mmap a large file or executable and its not paged in.

This may not be a direct problem for us then, but its troubling because its the memory footprint that shows up in the windows Task Manager.

Is there any way to find out what is producing the large private bytes usage?  I don't think DMD would help at all since its not heap.

I thought maybe it was a large sqlite file that was being held open, but the largest sqlite in my profile is 20MB.  Nowhere near 1GB.

I wonder if we are leaking a mmap() or something.

Note, this does not happen on initial browser launch.  Only after running for a while (like overnight).  I've only seen it in nightly so far.

	Andrew McCreight [:mccr8]
	Comment 1 • 7 years ago

system-heap-allocated is also very low (unlike another similar case I saw recently).

	Ben Kelly [:bkelly, not reviewing] Reporter
	Comment 2 • 7 years ago

I tried using sysinternals to inspect the vmmap.  Nothing jumped out at me, but I also don't really know how to use the tool either.

I restarted my browser now, though, so I can't look at that particular session any more.

	Eric Rahm [:erahm]
	Comment 3 • 7 years ago

(In reply to Ben Kelly [not reviewing due to deadline][:bkelly] from comment #0)
>         167.69 MB ── heap-allocated
>         318.00 MB ── heap-mapped
>       1,324.19 MB ── private
>       1,481.65 MB ── resident
>       1,319.93 MB ── resident-unique
>          99.09 MB ── shmem-allocated
>          99.15 MB ── shmem-mapped
>           2.98 MB ── system-heap-allocated
>       3,299.46 MB ── vsize
> 132,179,771.69 MB ── vsize-max-contiguous
> 
> Our entire heap is 318MB, but the private bytes are and unique RSS are over
> a GB.
> 
> From what I have read private bytes is memory that has been reserved, but
> may not actually be used.  For example, mmap a large file or executable and
> its not paged in.
> 

This is different from other issues we've seen where private bytes is high, but resident and resident-unique are low. What was your explicit measurement? Was it in line with |heap-mapped|?

I agree that it feels like we're leaking mmapped memory. I suppose it's also possible jemmaloc is misbehaving on the latest Win10 release.

	Ben Kelly [:bkelly, not reviewing] Reporter
	Comment 4 • 7 years ago

(In reply to Eric Rahm [:erahm] from comment #3)
> This is different from other issues we've seen where private bytes is high,
> but resident and resident-unique are low. What was your explicit
> measurement? Was it in line with |heap-mapped|?

Yes, explicit does not show the elevated private bytes value.  It was close to heap-mapped.

I haven't reproduced it then, but a lot has changed in my setup.  I switched to a new machine, new install, stopped using an addon, etc.  I'm keeping an eye out for it.

	Danial Horton
	Comment 5 • 7 years ago

Perhaps bug 1298905 and bug 1357225 should be closed as duplicates of this.

	Danial Horton
	Comment 7 • 7 years ago

in my memory report, and looking at the process with vmmap, a significant amount of ReadWrites make up the Private bytes of the Parent process.

│  │  ├──1,032.35 MB (100.0%) ── readwrite(segments=2633) [+]  << Some hours after.
│  │  ├──-213.48 MB (100.0%) ── readwrite(segments=1222) [-]   << Fresh start

The worst aspect of this bug though, it does not require any interaction from the user.

I have left a clean profile running for a day and returned home to a significantly higher private set, with little increase in explicit size.

	Danial Horton
	Comment 8 • 7 years ago

also, it is definitely not isolated to windows 10. I am Running Windows 7 sp1 here myself.

I jumped from Firefox 44 beta to 55 because of the return to unloaded tabs in content processes, so was slightly irritated when the build i jumped to had comparatively worse session longetivity.


Comparison

Firefox 53
4,095.94 MB (100.0%) -- address-space
├──3,150.09 MB (76.91%) ── free(segments=393)
├────479.96 MB (11.72%) -- commit
│    ├──216.25 MB (05.28%) -- private
│    │  ├──211.75 MB (05.17%) ── readwrite(segments=599)

Firefox 54
4,095.94 MB (100.0%) -- address-space
├──3,183.71 MB (77.73%) ── free(segments=387)
├────482.28 MB (11.77%) -- commit
│    ├──212.28 MB (05.18%) -- private
│    │  ├──207.44 MB (05.06%) ── readwrite(segments=590)

Nightly
4,095.94 MB (100.0%) -- address-space
├──3,029.58 MB (73.97%) ── free(segments=383)
├────605.02 MB (14.77%) -- commit
│    ├──275.10 MB (06.72%) -- private
│    │  ├──261.67 MB (06.39%) ── readwrite(segments=600)

Now from what I have seen, the segments increase and decrease from anything as low as 250, to upwards of 8000+ segments any time the measurement is refreshed.  So it seems like Freeing is occurring, but the MB usage of readwrites will grow more and more and this is where the bloated value of Private bytes seems to be.

For example, this instance of nightly i am comment from displays

4,095.94 MB (100.0%) -- address-space
├──2,611.48 MB (63.76%) ── free(segments=400)
├──1,076.94 MB (26.29%) -- commit
│  ├────707.68 MB (17.28%) -- private
│  │    ├──691.27 MB (16.88%) ── readwrite(segments=1687)

with
  385.23 MB (100.0%) ++ explicit
  722.73 MB ── private
  819.43 MB ── resident
  725.36 MB ── resident-unique

Whatever the problem is, Nightly is not allocating in memory that appears to be free, and this is resulting in a massive amount of Private byte fragmentation.

	Danial Horton
	Comment 11 • 7 years ago

either this was caused by jsgc poisoning, (and the variable needed a reboot to take effect)
or the latest nightly build has resolved this.

	Eric Rahm [:erahm]
	Comment 13 • 7 years ago

Bug 1371221, comment 0 might have actionable STR. I think primarily it's just let the browser sit for a day.

	Danial Horton
	Comment 14 • 7 years ago

I was wrong, it isn't fixed.  The issue has changed from the working set visibly increasing to the Commit set doing so instead, this change in behavior seems to have followed bug 1365194

In vmmap, this would look like the Size and Committed fields of private data ballooning over time.

	Danial Horton
	Comment 15 • 7 years ago

This has been fixed by bug 1377738