Darkspirit Reporter
	Description • 6 years ago

Seen on Socorro. bug 1446588 didn't stop this crash reason.

[@ static void core::option::expect_failed | static union core::result::Result<T> webrender::resource_cache::ResourceCache::get_cached_image ]
bp-a5506d84-c971-4af5-aa1e-4c7ed0180420 20180418230818 Win10
> Didn't find a cached resource with that ID!

[@ static void std::panicking::rust_panic_with_hook ]
bp-c51d97f8-b0da-4554-80e7-ea5640180418 20180417100054 Win10
> Didn't find a cached resource with that ID!

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 1 • 6 years ago

I'm seeing this crash pretty much every time I shut down or update WR-enabled nightly on macOS.

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 2 • 6 years ago

This is the #2 WR-specific topcrash. We should address this before turning on WR in nightly. I seem to be able to reproduce it so I'll take a look.

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 3 • 6 years ago

This might actually be #1 after combining the signatures.

	:gerard-majax
	Comment 4 • 6 years ago

I have seen the crash 5 times within the last 60 minutes, linux/ubuntu 18.04 with Intel GPU. Even just moving away from my laptop.

	:gerard-majax
	Comment 5 • 6 years ago

And just to be clear, this started just after I upgraded from 20180530220105 to 20180531101452.

	:gerard-majax
	Comment 6 • 6 years ago

I'm wondering if there's any link with those kernel messages:
> Jun  1 11:39:44 portable-alex kernel: [1329561.799780] [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
> Jun  1 11:40:03 portable-alex kernel: [1329581.395141] [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.

At least, it looks like each time there's one, I get a crash very soon after (< 60s). I've already set the maximum video memory in the BIOS (512MB).

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 7 • 6 years ago

I haven't been able to reproduce it on a local build yet, so I haven't been able to confirm. But my theory is this: when a tab is closed, we run through WRBP::ClearResources. This sends a transaction to clear out the display list and remove the pipeline at [1]. This transaction gets sent to the scene builder thread where it may take a bit of time.

Meanwhile, a few lines down, at [2], we release the WebRenderAPI clone for that tab, which should trigger the ClearNamespace at [3]. This happens in a message sent directly to the RenderBackend thread, and so can jump in front of the DL/pipeline removal, and put WR into a state where it still has the DL/pipeline, but cleared all the cached images. If, in that state, it tries to do a render (there might be one inflight already) then it might result in this error.

[1] https://searchfox.org/mozilla-central/rev/38bcf897f1fa19c1eba441a611cf309482e0d6e5/gfx/layers/wr/WebRenderBridgeParent.cpp#1607
[2] https://searchfox.org/mozilla-central/rev/38bcf897f1fa19c1eba441a611cf309482e0d6e5/gfx/layers/wr/WebRenderBridgeParent.cpp#1621
[3] https://searchfox.org/mozilla-central/rev/38bcf897f1fa19c1eba441a611cf309482e0d6e5/gfx/webrender_api/src/api.rs#988

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 8 • 6 years ago

I made a try push with logging [1] and I was able to reproduce on that. Indeed, it seems like the namespace is being cleared and then we request the image after that. Will try to put together a fix.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=0919751458bc2db546a9a3dc2aac55143e7d23a2

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 9 • 6 years ago

This might fix it: https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd2323486caab76edd458c4b46c444f551d819c5

	:gerard-majax
	Comment 11 • 6 years ago

(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #9)
> This might fix it:
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=fd2323486caab76edd458c4b46c444f551d819c5

Thanks. So far, toggling the pref |gfx.webrender.async-scene-build| seems to have helped a lot: nearly three hours without any crash! I'm going to test with your binaries out of TaskCluster :)

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 13 • 6 years ago

Hm, for some reason the try push seems to indicate the windows debug build is hanging on startup. Not sure what's going on there, will investigate.

	:gerard-majax
	Comment 14 • 6 years ago

(In reply to Alexandre LISSY :gerard-majax from comment #11)
> (In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #9)
> > This might fix it:
> > https://treeherder.mozilla.org/#/
> > jobs?repo=try&revision=fd2323486caab76edd458c4b46c444f551d819c5
> 
> Thanks. So far, toggling the pref |gfx.webrender.async-scene-build| seems to
> have helped a lot: nearly three hours without any crash! I'm going to test
> with your binaries out of TaskCluster :)

So, this has been rock stable since my comment. Looks good, :kats !

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 15 • 6 years ago

Shoot, the patch introduces a deadlock situation because it blocks on WR while holding the indirect layer trees lock at https://searchfox.org/mozilla-central/rev/38bcf897f1fa19c1eba441a611cf309482e0d6e5/gfx/layers/ipc/CompositorBridgeParent.cpp#504

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 16 • 6 years ago

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a93a88b934822534d54155e73acf22cd81e1067a should be better

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Assignee
	Comment 18 • 6 years ago

That's looking good :)

	Sotaro Ikeda [:sotaro]
	Comment 20 • 6 years ago
mozreview-review

Comment on attachment 8982595 [details]
Bug 1455597 - Flush the transaction to remove the pipeline before shutting down the WebRenderAPI.

https://reviewboard.mozilla.org/r/248570/#review254944

Loos good!

	Pulsebot
	Comment 21 • 6 years ago

Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bcebfd54b4e1
Flush the transaction to remove the pipeline before shutting down the WebRenderAPI. r=sotaro

	Dorel Luca [:dluca]
	Comment 22 • 6 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/bcebfd54b4e1