Jim Mathies [:jimm] Reporter
	Description • 9 years ago

This is an outstanding a11y KillHard abort. Looks like the api must be async since the failures happen in during a random mix of unrelated e10s api calls.

Bug 1116884 KillHard child signature breakdown:
----------------------------------------------------
23	51.1%	SendRpcMessage(..)
7	15.6%	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(..)
3	6.7%	SendSyncMessage(..)
3	6.7%	mozilla::dom::PContentChild::SendGetBlocklistState(..)
2	4.4%	mozilla::net::PCookieServiceChild::SendGetCookieString(..)
2	4.4%	mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,unsigned int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *)
2	4.4%	mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *)
1	2.2%	mozilla::dom::PContentChild::SendClipboardHasType(nsTArray<nsCString> const &,int const &,bool *)
1	2.2%	mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const&, unsigned long const&, mozilla::layers::TargetConfig const&, nsTArray<mozilla::layers::PluginWindowData> const&, bool const&, bool const&, unsigned int const&, bool const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::EditReply>*)
1	2.2%	mozilla::dom::PBrowserChild::SendGetInputContext(..)

IPC error breakdown per KillHard signature:
----------------------------------------------------
SendRpcMessage(..):
  3 - '(msgtype=0x3C0002,name=???) Route error: message sent to unknown actor ID'
  3 - '(msgtype=0x200012,name=???) Message not allowed: cannot be sent/recvd in this state'
  6 - 'HangMonitor'
  8 - '(msgtype=0x200004,name=PBrowser::Msg_PDocAccessibleConstructor) Processing error: message was deserialized, but the handler returned false (indicating failure)'
  4 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(..):
  1 - '(msgtype=0xA20006,name=???) Message not allowed: cannot be sent/recvd in this state'
  7 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

SendSyncMessage(..):
  3 - 'HangMonitor'
  1 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PContentChild::SendGetBlocklistState(..):
  4 - '(msgtype=0x3A004E,name=PContent::Msg_GetBlocklistState) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::net::PCookieServiceChild::SendGetCookieString(..):
  2 - '(msgtype=0x3C0002,name=???) Route error: message sent to unknown actor ID'
  1 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,unsigned int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *):
  3 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *):
  2 - 'HangMonitor'
  1 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PContentChild::SendClipboardHasType(nsTArray<nsCString> const &,int const &,bool *):
  2 - '(msgtype=0x3A00A2,name=???) Message not allowed: cannot be sent/recvd in this state'

mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const&, unsigned long const&, mozilla::layers::TargetConfig const&, nsTArray<mozilla::layers::PluginWindowData> const&, bool const&, bool const&, unsigned int const&, bool const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::EditReply>*):
  2 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PBrowserChild::SendGetInputContext(..):
  2 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

	Trevor Saunders (:tbsaunde) Assignee
	Comment 1 • 9 years ago

see my last comment in bug 1185726 I guess we should just change it to assert and return true until we find the bug in the child.

	Trevor Saunders (:tbsaunde) Assignee
	Comment 4 • 9 years ago

Having one reorder event that superseeds another does not mean that the
dependant mutation events for the first reorder are obsolete.

	alexander :surkov (:asurkov)
	Comment 5 • 9 years ago

Comment on attachment 8648170 [details] [diff] [review]
only coalesce reorder events when a previous one for the same target is obsolete

Review of attachment 8648170 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with comments addressed

::: accessible/tests/mochitest/treeupdate/test_bug1189277.html
@@ +21,5 @@
> +      this.eventSeq = [];
> +      this.containerNode = getNode("outerDiv");
> +
> +      this.eventSeq.push(new invokerChecker(EVENT_REORDER, this.containerNode));
> +      this.eventSeq.push(new invokerChecker(EVENT_SHOW, "newChildDoc"));

pls add testing for two HIDE events as well

nit, I would prefer styling
this.eventSeq = [
  new invokerChecker(),
  new invokerChecker()
];

@@ +38,5 @@
> +      }
> +
> +      this.finalCheck = function runTest_finalCheck()
> +      {
> +      ok(true, "dummy test");

you don't have to have finalCheck iirc and that fake ok()

	Pulsebot
	Comment 6 • 9 years ago

https://hg.mozilla.org/integration/mozilla-inbound/rev/4d2723bc0bcb

	Ryan VanderMeulen [:RyanVM]
	Comment 7 • 9 years ago

https://hg.mozilla.org/mozilla-central/rev/4d2723bc0bcb

	Jim Mathies [:jimm] Reporter
	Comment 8 • 9 years ago

Definitely fixed, can we uplift to aurora? 42.0a2 has the same problem.

	Trevor Saunders (:tbsaunde) Assignee
	Comment 9 • 9 years ago

(In reply to Jim Mathies [:jimm] from comment #8)
> Definitely fixed, can we uplift to aurora? 42.0a2 has the same problem.

its not e10s only, and I'm a little concerned about fall out, but I guess its kind of early.  surkov thoughts?

	alexander :surkov (:asurkov)
	Comment 10 • 9 years ago

(In reply to Trevor Saunders (:tbsaunde) from comment #9)
> (In reply to Jim Mathies [:jimm] from comment #8)
> > Definitely fixed, can we uplift to aurora? 42.0a2 has the same problem.
> 
> its not e10s only, and I'm a little concerned about fall out, but I guess
> its kind of early.  surkov thoughts?

same worries, but taking into account that no known regressions so far and major of testing doesn't happen on nightly then it's probably ok for uplifting.

	Jim Mathies [:jimm] Reporter
	Comment 11 • 9 years ago

Comment on attachment 8648170 [details] [diff] [review]
only coalesce reorder events when a previous one for the same target is obsolete

Approval Request Comment
[Feature/regressing bug #]:
a11y e10s support
[User impact if declined]:
crashes, bug 1116884 incident rates stay elevated
[Describe test coverage new/current, TreeHerder]:
landed on mc on the 19th, no issues this far.
[Risks and why]:
could regress something undetected.
[String/UUID change made/needed]:
none

	Sylvestre Ledru [:Sylvestre]
	Comment 12 • 9 years ago

Comment on attachment 8648170 [details] [diff] [review]
only coalesce reorder events when a previous one for the same target is obsolete

We want to fix a11y

	Ryan VanderMeulen [:RyanVM]
	Comment 13 • 9 years ago

https://hg.mozilla.org/releases/mozilla-aurora/rev/6209bf0c49e5

	Jim Mathies [:jimm] Reporter
	Comment 14 • 9 years ago

This still shows up on aurora. Opening this back up so we can track on it.

	David Bolter [:davidb] (NeedInfo me for attention)
	Comment 15 • 9 years ago

Jim is there a place I can look to see if this killhard still happens? (on nightly)

	Jim Mathies [:jimm] Reporter
	Comment 16 • 9 years ago

(In reply to David Bolter [:davidb] from comment #15)
> Jim is there a place I can look to see if this killhard still happens? (on
> nightly)

Currently isn't around - 

http://www.mathies.com/mozilla/client-abort-report-nightly.txt
http://www.mathies.com/mozilla/client-abort-report-aurora.txt
http://www.mathies.com/mozilla/client-abort-report-beta.txt

	Trevor Saunders (:tbsaunde) Assignee
	Comment 17 • 9 years ago

(In reply to Jim Mathies [:jimm] from comment #16)
> (In reply to David Bolter [:davidb] from comment #15)
> > Jim is there a place I can look to see if this killhard still happens? (on
> > nightly)
> 
> Currently isn't around - 

any reason to believe it isn't fixed?

	Guilherme Lima
	Comment 18 • 9 years ago

e10s is now disabled when acessibility is on, that's probably the reason these crashes are gone.

Before that I got 2 crashes while browsing and opening pages from www.9gag.com, and both have
ipc_channel_error (msgtype=0x560008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)

	David Bolter [:davidb] (NeedInfo me for attention)
	Comment 20 • 8 years ago

Still happening in nightly.

	BugBot [:suhaib / :marco/ :calixte]
	Comment 21 • 8 years ago

Crash volume for signature 'IPCError-browser | (msgtype=0x560009,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized,':
 - nightly (version 50): 16 crashes from 2016-06-06.
 - aurora  (version 49): 106 crashes from 2016-06-07.
 - beta    (version 48): 0 crash from 2016-06-06.
 - release (version 47): 0 crash from 2016-05-31.
 - esr     (version 45): 0 crash from 2016-04-07.

Crash volume on the last weeks:
             Week N-1   Week N-2   Week N-3   Week N-4   Week N-5   Week N-6   Week N-7
 - nightly          0          0          0          0          0          3         13
 - aurora          13          9         17         27         10         24          1
 - beta             0          0          0          0          0          0          0
 - release          0          0          0          0          0          0          0
 - esr              0          0          0          0          0          0          0

Affected platforms: Windows, Mac OS X, Linux

	Jim Mathies [:jimm] Reporter
	Comment 24 • 8 years ago

Looks like bug 1270916 may have addressed this.

	Ben Kelly [:bkelly, not reviewing]
	Comment 25 • 8 years ago

I've been hitting this once a day on FF51 beta.  Jim, any ideas why I would be triggering this on beta?

https://crash-stats.mozilla.com/report/index/bp-2e3d625c-820b-4ccb-b140-18b032161121
https://crash-stats.mozilla.com/report/index/bp-b8ba88cc-9187-46d7-b96f-7e0992161122

Note, I briefly enabled the windows emoji keyboard the other day, but I have it disabled now.  Did that put me into some kind of accessibility mode?  How do I disable that?  (I have e10s forced on.)

	Jim Mathies [:jimm] Reporter
	Comment 26 • 8 years ago

(In reply to Ben Kelly [:bkelly] from comment #25)
> I've been hitting this once a day on FF51 beta.  Jim, any ideas why I would
> be triggering this on beta?
> 
> https://crash-stats.mozilla.com/report/index/bp-2e3d625c-820b-4ccb-b140-
> 18b032161121
> https://crash-stats.mozilla.com/report/index/bp-b8ba88cc-9187-46d7-b96f-
> 7e0992161122
> 
> Note, I briefly enabled the windows emoji keyboard the other day, but I have
> it disabled now.  Did that put me into some kind of accessibility mode?  How
> do I disable that?  (I have e10s forced on.)

Check to see if you have a11y enabled through about:support. You may hit this if you have e10s forced on and you have some sort of touch screen laptop or soft keyboard active.

	Ciprian Georgiu, Desktop QA
	Comment 27 • 8 years ago

I ran into this issue with latest Aurora 52.0a2 (2016-12-14) while testing touch events in APZ. The crash happened on mashable.com using Dell XPS 12, Windows 10 x64. 
 - https://crash-stats.mozilla.com/report/index/cf796ca5-d569-46e9-8397-bcba02161214

It seems that are still some recent crashes on 52.0a2 from the stats signatures.

	Jacob W. Klein
	Comment 28 • 7 years ago

Just wanted to chime in to say that I have another "common scenario" that causes this error, I think.
- Using Windows 10 Insider Preview Build 14986
- Using Firefox 50.0.1.0 x86

Sometimes, if I
- Go to https://www.neowin.net
- Hover over the "Facebook Twitter Google+" image in the header
- POOF. "Bad news first: This tab has crashed"

It doesn't happen all the time, and I can't pin down why. I think it has something to do with how many Flash objects I've previously loaded in the session. Not sure.

Regards,
Jacob

	Jacob W. Klein
	Comment 29 • 7 years ago

Note: I have e10s forced on, with a few extensions installed.

	(No longer employed by Mozilla) Aaron Klotz
	Comment 30 • 7 years ago

(In reply to Jacob W. Klein from comment #29)
> Note: I have e10s forced on, with a few extensions installed.

That's the price of force-enabling e10s, I'm afraid. We are currently focusing on 52-53 for stabilizing touch and a11y on e10s.