Closed Bug 1243399 Opened 8 years ago Closed 7 years ago

Automate MDN data scraping

Categories

(developer.mozilla.org Graveyard :: BrowserCompat, defect)

defect
Not set
major

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: Jeremie, Unassigned)

References

Details

(Keywords: in-triage, Whiteboard: [bc:infra][bc:kuma][bc:milestone=scooter])

What problem would this feature solve?
======================================
Currently, the importer is run manually, to make sure we will be able to do a smooth transition to the Browser Compat Data we need to make sure the importer run automatically on a regular basis.

Who has this problem?
=====================
All contributors to MDN

How do you know that the users identified above have this problem?
==================================================================
When we will switch displaying data from the BC data base on MDN, the data will be still contributed inside the Wiki content until we are done with the whole C&M GUI.

How are the users identified above solving this problem now?
============================================================
By scraping the data automatically, we prevent users contributing data on MDN to go to the browsercompat.org web site to ensure the new data are taken into account.

Do you have any suggestions for solving the problem? Please explain in detail.
==============================================================================
At minimum, having a daily automated scraping of all the data would be nice. At best, having the importer being notice of page change on MDN and scrap data on demand would be top notch.

Is there anything else we should know?
======================================
Doing this early in the process will help identifying the most common contribution mistake that break the importer. It will help us define what fix are needed for the importer and what delay is acceptable between the release of tables display and the release of the C&M GUI.
Blocks: 996570
Severity: enhancement → major
Keywords: in-triage
OS: Other → All
Whiteboard: [specification][type:feature] → [bc:infra][bc:kuma]bc:milestone=scooter]
Whiteboard: [bc:infra][bc:kuma]bc:milestone=scooter] → [bc:infra][bc:kuma][bc:milestone=scooter]
Depends on: 1247974
The BrowserCompat project is canceled.  See https://github.com/mdn/browsercompat for current effort. Bulk status change includes the random word TEMPOTHRONE.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Product: developer.mozilla.org → developer.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.