We have a lot of configuration keys in consulate, and that is hard to parse and hard to maintain. We also have a few configuration files in our socorro-infra repo [0], and those are loaded in boxes when we build them. So, we want to clean that up a bit, doing the following: * remove useless options or options that are the same as the application defaults * move options with the same values between stage and prod into the socorro-infra config files [0] https://github.com/mozilla/socorro-infra/tree/master/socorro-config

First step: the webapp namespace. This pull request reorganizes the webapp-django.conf file, removes unneeded values and adds a few options that have the same values in stage and prod.

Changes to apply through consulate on stage after previous PR is merged.

Changes to apply through consulate on prod after previous PR is merged and deployed.

Commits pushed to master at https://github.com/mozilla/socorro-infra https://github.com/mozilla/socorro-infra/commit/4f37041fa2153d9feb0db2060d00d02cfb04880d Bug 1296238 - Refactored basic webapp configuration. https://github.com/mozilla/socorro-infra/commit/6d19a7cf4016bba3d7fc3ca95d4c3705ab1834fb Merge pull request #251 from adngdb/1296238-refactor-webapp-config Bug 1296238 - Refactored basic webapp configuration.

Commits pushed to master at https://github.com/mozilla/socorro-infra https://github.com/mozilla/socorro-infra/commit/9dc16f5f42c2847c811f467e4494475c89d15b28 Revert "Bug 1296238 - Refactored basic webapp configuration." https://github.com/mozilla/socorro-infra/commit/c421b8856f510dba5782a4c846761bed5888d5df Merge pull request #253 from mozilla/revert-251-1296238-refactor-webapp-config Revert "Bug 1296238 - Refactored basic webapp configuration."

Comment on attachment 8782361 [details] changes-stage-webapp.md Yes. I checked the "# Useless keys" part. They'are all useless. I don't know why I'm really struggling to understand what it means "# Already defined in config files" I thought we had entirely done away with config file?! Are they still lurking? If so, we should delete them all and not use them. The only sane places for config is consul or defaults in .py files. Where are these config files?

Comment on attachment 8782362 [details] changes-prod-webapp.md Same point I made on the changes-stage-webapp.md file. 1) The useless keys definitely are useless. 2) I don't understand why these values are in config files (and not in consul or source code)

I think I do want to remove keys that are no longer used, but I want to know what the process is for determining what is a key that's no longer used. Some of them are obvious. Others not so much. I don't think I want to move configuration between consul and the config files in socorro-infra. I'd much rather have all the configuration in one place since it makes it easier to reason about what's going on without having to look in multiple places. As a side note, I discovered some differences between -stage and -prod. Those differences would not have been alleviated by this work. We should do an audit between the two environments and fix the differences.

I agree with Will, at least temporarily. Let's make a decision on this at or after the work week.

What is the status of this bug? Can all secrets be set via environment variables?

Consul is loosely coupled through environment variables via envconsul, everywhere

@miles -- this has a confusing title. We have conflicting definitions of configuration within the project. This bug is not 'remove consul entirely' but instead to remove any config values from consul that are in agreement with application defaults, and then move common configuration between stage+prod out of consul. Consul would still provide attributes about the environment like the address and creds for accessing data stores.

Two things: 1. Consul is going away with the new docker-based environment. 2. The docker-based dev environment is a -prod like environment. We can use that to test the kinds of changes in this bug in the comfort and safety of our throw-away local dev environments. Given that, I'm going to grab this and work on it after I have the dev environment more functional.

I did a pass on docker/config/local_dev.env . This file holds all the security and environment-specific configuration for all the Socorro components running in a local dev environment. This file will be used to figure out the equivalent file for the -stage and -prod server environments. Thus it behooves us to have a minimal file in which we know what all the things are. We should do passes on the other configuration files as well.

Commit pushed to master at https://github.com/mozilla-services/socorro https://github.com/mozilla-services/socorro/commit/f44d09d7385a756459d4d8e1e7ed768b1109d754 bug 1296238 - reduce vars in local_dev.env (#4005) This removes a bunch of unneeded vars from local_dev.env and moves a few others to better files.

Mike's PR moves behavior configuration into Python code: https://github.com/mozilla-services/socorro/pull/4307 I think that'll radically reduce the amount of configuration we have floating around. We should wait until that lands and then see if there's anything left to do here. Unassigning myself.

Mike's PR has landed. We also have bug #1439546 which removes a ton of dead configuration. Once that lands, I think we can call this bug done, too.

I marked the last outstanding bug as FIXED. Given that, I'm going to mark this FIXED, too. Yay!