Geoff Brown [:gbrown] Reporter
	Description • 9 years ago

In bug 1189784, I am trying to enable dom/media mochitests on Android 4.3. Those tests have always been skipped on Android 4.3, since most of those tests fail intermittently on that platform. Many of the test failures are timeouts; I have tried increasing the timeout thresholds, but this has not helped much -- it seems tests continue to hang. I suspect there is not enough cpu available to run the emulator + test + media support.

However, if I change the aws instance type to c3.xlarge, then most of the dom/media tests pass reliably.

Can we run Android 4.3 Opt mochitests (M1 .. M16) on c3.xlarge?

There are other limitations preventing us from running dom/media mochitests on Android 2.3 so I am not suggesting any changes for 2.3.

Android 4.3 Debug mochitests already run on c3.xlarge so I am not suggesting any changes for 4.3 Debug.

(fwiw, I am starting to think all emulator tests should run on c3.xlarge, since we seem to keep running into problems like this. The m1.medium seems to be just barely capable of running the emulator -- anything extra and we run into trouble. I don't have a specific need beyond dom/media on 4.3 Opt though.)

	Kim Moir [:kmoir] ET Assignee
	Comment 4 • 9 years ago

Comment on attachment 8650083 [details] [diff] [review]
bug1195893.patch

the builder diff is just the same jobs but in a different order.  No actual job names are being changed, just the underlying instance type

	Justin Wood (:Callek)
	Comment 5 • 9 years ago

Comment on attachment 8650083 [details] [diff] [review]
bug1195893.patch

Review of attachment 8650083 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/mobile_config.py
@@ +2391,5 @@
>  for suite in ANDROID_4_3_MOZHARNESS_DICT: 
>      if suite[0].startswith('plain-reftest'):
>          ANDROID_4_3_C3_DICT['opt_unittest_suites'].append(suite)
>          ANDROID_4_3_C3_DICT['debug_unittest_suites'].append(suite)
> +    elif re.match(r"^mochitest-\d", suite[0]):

I'm hating our configs more and more when I see stuff like this (regex's based on suite name, is scary)

@@ +2395,5 @@
> +    elif re.match(r"^mochitest-\d", suite[0]):
> +        ANDROID_4_3_C3_DICT['opt_unittest_suites'].append(suite)
> +        ANDROID_4_3_C3_DICT['debug_unittest_suites'].append(suite)
> +        ANDROID_4_3_C3_TRUNK_DICT['debug_unittest_suites'].append(suite)
> +    elif suite[0].startswith('mochitest') and not (re.match(r"^mochitest-\d", suite[0])):

can save some minor computational time and omit |and not...| since we'll never get here if it did match (due to the condition your adding above)

	Kim Moir [:kmoir] ET Assignee
	Comment 6 • 9 years ago

patch with fixes from review

	Kim Moir [:kmoir] ET Assignee
	Comment 7 • 9 years ago

verified that it's working on treeherder

will keep an eye on emulator64 pool and see if the size needs to be increased

	Kim Moir [:kmoir] ET Assignee
	Comment 8 • 9 years ago

Note to future self
run
for f in `find . -name '*.json'` ./configs/watch_pending.cfg; do python -m json.tool $f; done
on the watch_pending.cfg before submitting patch

because this caused watch pending to fail and new spot instances not to be created

Fix
https://github.com/mozilla/build-cloud-tools/commit/5f23650624fc4a34b051d8b045f7363e3b728355