Evan Tseng [:evanxd] Assignee
	Description • 10 years ago

We would like to make sure the tests are stable enough.
So we let the tests run many times continually for a pull request.

	Evan Tseng [:evanxd] Assignee
	Comment 2 • 10 years ago

Comment on attachment 8375307 [details]
WIP PR

>https://github.com/evanxd/gaia/commit/2bd7bb516b109e1ac19ec0957888841c12ee94e3

	Evan Tseng [:evanxd] Assignee
	Comment 3 • 10 years ago

This patch is for current build script architecture.
https://github.com/evanxd/gaia/commit/6a0ad528fd0d0fb5db6d7e82f74082b6fdf2a881

	Evan Tseng [:evanxd] Assignee
	Comment 4 • 10 years ago

Hi George,

Please help me to review the patch.
You could check-pick the it and change any marionette test to trigger 30 times running in CI_ACTION=marionette_js_check on Travis.

Thanks. :)

	Evan Tseng [:evanxd] Assignee
	Comment 5 • 10 years ago

The https://github.com/mozilla-b2g/gaia/pull/16244 pull request cannot be accessed now, GitHub seems have some issues now.

	Evan Tseng [:evanxd] Assignee
	Comment 6 • 10 years ago

Yeah, GitHub is back now. :D

	(Inactive after June) George Duan [:gduan] [:喬智]
	Comment 7 • 10 years ago

Comment on attachment 8376689 [details] [review]
Pull request

Hi Evan,
I've updated my comments in github. We can discuss it on Mon.

1. can't we just use git command to find the changed files of last commit and run the test? So that we can run it remotely and locally by simply a |make pull-request-marionette-test|
2. is it a must-pass test from now?
3. some nit, please check.

	Evan Tseng [:evanxd] Assignee
	Comment 8 • 10 years ago

Hi George,

Thanks for the comments.

For 1, there might be more than one commit in a pull request.
For 2, we could let the action in `allowed failures` area first. After we make sure the action is stable, then we could move the `must pass` area.
For 3, already updated in GitHub page.

Thanks.

	Evan Tseng [:evanxd] Assignee
	Comment 9 • 10 years ago

Comment on attachment 8376689 [details] [review]
Pull request

Hi George,

I already updated the patch.
Please help me to review the patch again.
Thanks. :)

	(Inactive after June) George Duan [:gduan] [:喬智]
	Comment 10 • 10 years ago

Comment on attachment 8376689 [details] [review]
Pull request

Thanks, Evan. It's a very helpful tool 
r=gduan

Hi Yuren,
would you mind to take a look at this patch ?
Thanks.

	Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)
	Comment 11 • 10 years ago

Comment on attachment 8376689 [details] [review]
Pull request

Evan, you should handle exit value of `RUN_SPECIFIC_TESTS` or we will always get green state of travis-ci.

	Evan Tseng [:evanxd] Assignee
	Comment 12 • 10 years ago

(In reply to Yuren [:yurenju][:小朱] from comment #11)
> Comment on attachment 8376689 [details] [review]
> Pull request
> 
> Evan, you should handle exit value of `RUN_SPECIFIC_TESTS` or we will always
> get green state of travis-ci.

Hi Yuren,

I already fixed that.
Thanks for the tips.

	Evan Tseng [:evanxd] Assignee
	Comment 13 • 10 years ago

master: https://github.com/mozilla-b2g/gaia/commit/9ea16d757a290de5b374d809e43dc1e99f4f802d

	Gareth Aye [:gaye] (back from PTO)
	Comment 14 • 10 years ago

Hey Evan,

I think it might have made sense to have more discussion about whether we want to be using travis resources for this. Don't get me wrong - it's awesome that you took initiative to do this, but there may have been other ways to accomplish this using fewer resources. In the future, please run patches like this one by another marionettejs owner/peer (see https://wiki.mozilla.org/Modules/Core#JS_Marionette_Harness_and_Tools).

	Evan Tseng [:evanxd] Assignee
	Comment 15 • 10 years ago

Hi Gareth,

Got it.
I will follow the way.

Thanks.