Will Kahn-Greene [:willkg] ET needinfo? me Assignee
	Description • 6 years ago

Over a number of bugs (this one blocks on them), we've been moving ownership of database data from alembic/sqlalchemy to Django. In most of those cases, we've left the original tables in place in case we needed to go back to them or find something in them.

This bug covers deleting those tables:

* os_names
* os_names_matche
* bug_associations
* crontabber
* crontabber_log
* graphics_device
* signatures

	Will Kahn-Greene [:willkg] ET needinfo? me Assignee
	Comment 1 • 6 years ago

We're pretty close to not needing the product_versions table. Once we no longer need that, we can remove sqlalchemy, alembic, all the stored procedures, all the tables managed with those things, and all the related code.

I'm going to rescope this bug to cover that.

	[github robot]
	Comment 3 • 6 years ago

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/9615da2448ff39c938cf54a77519d4869bffbe63
fix bug 1503383: remove alembic, sqlalchemy, and non-Django things

This is a huge change. It does the following:

* Remove ftpscraper and featured_versions crontabber apps. We no longer
  need these since we no longer use product_versions and friends tables.
* Remove ProductVersionsMiddleware and API endpoint.
* Remove alembic infrastructure and migrations.
* Remove sqlalchemy infrastructure and models.
* Remove all the stored procedures and non-Django tables.
* Add a new "socorro-cmd db" subcommand to create and drop the database.
* Remove setupdb app and rework tests and Makefile scripts to use
  Django to create tables.
* Update a bunch of documentation.

After this, the database is managed by Django and there are no non-Django
bits. The only thing left of the postgresql crashstorage stuff is the
ConnectionContext because it's used by crontabber and the BetaVersionRule
in the processor.

https://github.com/mozilla-services/socorro/commit/561e8368223a72cbe1fed25a838b922cf6a5cf20
bug 1503383: move dbapi2_util to socorro/lib/dbutil.py

https://github.com/mozilla-services/socorro/commit/8cf512af7e26e0f9a84fedf51058aaa762357854
bug 1503383: remove ConnectionFactory

https://github.com/mozilla-services/socorro/commit/c674cffe56151e532f050268d1d8bdf3d2bba029
Merge pull request #4723 from willkg/1503383-pg

fix bug 1503383: remove alembic, sqlalchemy, and non-Django things

	[github robot]
	Comment 4 • 6 years ago

Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/61bc780a469464a52af81e13f11aee156e40857e
fix bug 1503383: fix crontabber

Crontabber isn't running because the "main()" bit was taken out of the
Python module. Need to run it via socorro-cmd now.

https://github.com/mozilla-services/socorro/commit/a004922cfe03e0ef15d5bf692b883b4546f2e55f
Merge pull request #4724 from willkg/1503383-crontabber

fix bug 1503383: fix crontabber

	Will Kahn-Greene [:willkg] ET needinfo? me Assignee
	Comment 6 • 6 years ago

We ran the script in stage and it hit issues because there was an index that depended on one of the functions. I adjusted the script. The gist and history is here:

https://gist.github.com/willkg/ef8fa69973652fb6b819434ed019e732

We ran that in stage and prod and everything looks good.