Closed
Bug 1166431
Opened 9 years ago
Closed 6 years ago
Create distinct log files for each call to BaseMarionetteTestRunner.run_tests
Categories
(Remote Protocol :: Marionette, defect, P3)
Remote Protocol
Marionette
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: whimboo, Unassigned, Mentored)
Details
(Keywords: pi-marionette-runner, Whiteboard: [lang=Python])
As I have seen right now while running our update tests in Jenkins, the CI marked the job as unstable. The reason for it is that we are generating invalid log files. This is happening because we are starting Marionette twice - first for direct updates and a second time for fallback updates. In this case Marionette does not replace the log files but simply appends more data at the end. Especially for the xunit log files it will end up with multiple "<xml>" root elements. David, any idea of how we could get this easily fixed? One possibility would be if we check for those command line options and if set, generate a backup of the log files. Or simply numerate them up. I think anything else is deeper in the Marionette client and not that important to get fixed.
Flags: needinfo?(dburns)
Comment 1•9 years ago
|
||
I keep thinking I have answered this and noticing I havent so I am sorry it has taken me so long to reply. I suspect the issue is that somewhere we are creating a new reporter object when starting up the browser/session and that is why we are getting ourselves into bad place. I think it is just because restarts was never thought of in this respect and hopefully we can have a "global" report that we all use during a test run.
Flags: needinfo?(dburns)
Reporter | ||
Comment 2•9 years ago
|
||
Maja, with all of your latest work is this still a problem for e.g. xunit tests? If yes, maybe it's something we can still fix in Marionette proper in one of the next releases?
Flags: needinfo?(mjzffr)
It's still a problem. Let's fix it.
Assignee: hskupin → mjzffr
Blocks: 1210874
Component: Firefox UI Tests → Marionette
Flags: needinfo?(mjzffr)
Keywords: ateam-marionette-runner
Product: Mozilla QA → Testing
QA Contact: hskupin
Summary: Firefox UI update tests are generating invalid log files → Create distinct log files for each call to BaseMarionetteTestRunner.run_tests
:whimboo and I have concluded that this is no longer a high-priority change since firefox-update-tests are now run by a mozharness script so the mozharness logs are examined instead. As for the xml log, that was only used by a dashboard for mozmill-ci which is also falling out of use. In any case, I also discussed possible approaches to fix the issue. The approach I favour now is swap out the logger's handlers every time BaseMarionetteTestRunner.run_tests is called. In other words, the `logger` object has a list `handlers`. In this list, the StreamHandler objects are responsible for writing to files or stdout. We could add some logic to BaseMarionetteTestRunner.run_tests to flush and replace the appropriate StreamHandler objects whenever we want logging to go to new file(s). All the file names could have a date-stamp. This behaviour should be hidden behind a command-line option so that the default behaviour remains the same, that is, the default file names are reused for all log files so that we don't fill up the local workspace.
Assignee: mjzffr → nobody
No longer blocks: 1210874
Mentor: mjzffr
Status: ASSIGNED → NEW
Whiteboard: [lang=Python]
Comment 6•6 years ago
|
||
I’m going to close this bug since the last update two years ago said it was no longer a high priority. It is likely never going to become a high priority.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
Reporter | ||
Comment 7•6 years ago
|
||
It was a nice to have thing for the Firefox UI update tests. But those won't exist that long anymore.
Comment 8•6 years ago
|
||
We should make sure that when (if?) we move to a new test harness based on pytest, it uses separate log files.
Updated•1 year ago
|
Product: Testing → Remote Protocol
You need to log in
before you can comment on or make changes to this bug.
Description
•