Jukka Jylänki Reporter
	Description • 8 years ago

Looking at a memory report of an Emscripten asm.js page built with Unity3D, there is a suspiciously large amount of memory blocks reported under heap-unclassified by our DMD memory reporting tool, coming from ANGLE shader validator. Assuming there should be no reason that the shader validator would be alive(?) after all shaders have already been compiled, these blocks look like leaked memory.

On a tested page we are trying to squeeze to run on a lower memory mobile phone, the log finds 1706 individual blocks leaked of size 8192 bytes each, and 50 blocks of size 4093 bytes, for a total of 14180202 bytes.

The callstack for those allocations commonly reads like

    #01: malloc_impl (d:\gecko-dev\memory\build\replace_malloc.c:153)
    #02: moz_xmalloc (d:\gecko-dev\memory\mozalloc\mozalloc.cpp:83)
    #03: graphite2::gralloc<char> (d:\gecko-dev\gfx\graphite2\src\inc\main.h:89)
    #04: TPoolAllocator::allocate (d:\gecko-dev\gfx\angle\src\compiler\translator\poolalloc.cpp:272)
    #05: pool_allocator<std::_Tree_node<std::pair<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > const ,TSymbol * __ptr64>,void * __ptr64> >::allocate (d:\gecko-dev\gfx\angle\src\compiler\translator\poolalloc.h:272)
    #06: std::_Wrap_alloc<pool_allocator<std::_Tree_node<std::pair<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > const ,TSymbol * __ptr64>,void * __ptr64> > >::allocate (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xmemory0:901)
    #07: std::_Tree_comp_alloc<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,pool_allocator<char> >,TSymbol * __ptr64,std::less<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > >,pool_allocator<std::pair<std::basic_string<char (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xtree:857)
    #08: std::_Tree_comp_alloc<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,pool_allocator<char> >,TSymbol * __ptr64,std::less<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > >,pool_allocator<std::pair<std::basic_string<char (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xtree:888)
    #09: std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,pool_allocator<char> >,TSymbol * __ptr64,std::less<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > >,pool_allocator<std::pair<std::basic_string<char,std::char_ (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xtree:1665)
    #10: std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,pool_allocator<char> >,TSymbol * __ptr64,std::less<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > >,pool_allocator<std::pair<std::basic_string<char,std::char_ (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xtree:1861)
    #11: std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,pool_allocator<char> >,TSymbol * __ptr64,std::less<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > >,pool_allocator<std::pair<std::basic_string<char,std::char_ (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xtree:1837)
    #12: std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,pool_allocator<char> >,TSymbol * __ptr64,std::less<std::basic_string<char,std::char_traits<char>,pool_allocator<char> > >,pool_allocator<std::pair<std::basic_string<char,std::char_ (c:\program files (x86)\microsoft visual studio 14.0\vc\include\xtree:1078)
    #13: TSymbolTableLevel::insert (d:\gecko-dev\gfx\angle\src\compiler\translator\symboltable.cpp:59)
    #14: TSymbolTable::insert (d:\gecko-dev\gfx\angle\src\compiler\translator\symboltable.h:373)
    #15: TSymbolTable::insertBuiltIn (d:\gecko-dev\gfx\angle\src\compiler\translator\symboltable.cpp:271)
    #16: TSymbolTable::insertBuiltIn (d:\gecko-dev\gfx\angle\src\compiler\translator\symboltable.cpp:234)
    #17: TSymbolTable::insertBuiltIn (d:\gecko-dev\gfx\angle\src\compiler\translator\symboltable.h:420)
    #18: InsertBuiltInFunctions (d:\gecko-dev\gfx\angle\src\compiler\translator\initialize.cpp:138)
    #19: TCompiler::InitBuiltInSymbolTable (d:\gecko-dev\gfx\angle\src\compiler\translator\compiler.cpp:452)
    #20: TCompiler::Init (d:\gecko-dev\gfx\angle\src\compiler\translator\compiler.cpp:177)
    #21: ShConstructCompiler (d:\gecko-dev\gfx\angle\src\compiler\translator\shaderlang.cpp:197)
    #22: mozilla::webgl::ShaderValidator::Create (d:\gecko-dev\dom\canvas\webglshadervalidator.cpp:193)
    #23: mozilla::WebGLContext::CreateShaderValidator (d:\gecko-dev\dom\canvas\webglshadervalidator.cpp:182)
    #24: mozilla::WebGLShader::CompileShader (d:\gecko-dev\dom\canvas\webglshader.cpp:213)

See the attached log for the full DMD report.

	Jukka Jylänki Reporter
	Comment 1 • 8 years ago

Examining this further by running all demos in the emunittest suite, then closing all the tabs, opening about:memory, tapping CC/GC/Minimize memory usage buttons, and then taking a DMD log, there are about 20000 such blocks of size 8192 coming from this call site, which suggests this is an actual memory leak.

	Andrew McCreight [:mccr8]
	Comment 2 • 8 years ago

The ANGLE shader validator uses a pool allocator. It looks like the pool grows as large as it needs to, and then remains at that size until the ANGLE Compiler is destroyed in ~Context. I don't know what the lifetime of Context is, but it sounds like it is reasonably long. Maybe the pool could be purged at some earlier point, or split into separate pools with shorter lifetimes. Somebody who is more familiar with this code would have to look into this.