Configuration as a code

4 min readMar 12, 2021

Foreword
This article’s auditory
Use case examples
Idea overview
ConfigurationPolicy in depth
ConfigurationPolicyComponent in depth
(! TL/DR jump here) Practical example
Appendix A- Bonuses in the code

Foreword

Did you ever had a nightmare where you trigger production deployment with staging values? Having a guilty conscience you insert “last frontier protection“ to your deployment code:

if environment_name == “production”: check_protected_values()

But you want your code to be declarative! You wanted the same code to run in all environments with a single difference- input. This article presents the best solution I’ve found (for now) to eat the cake and leave it whole.

This article’s auditory

You must remember what Dev in DevOps stands for!

You have multiple environments/modules sharing common configuration- see “Use Case Examples”.

I implement the solution in Python. But it can be implemented in any language of your choice.

Use Case Examples

Multi environments’ configurations

The same code must run in separate but similar environments: [prod, stg, qa] or [prod_site_1, prod_site_2, prod_site_N]

API/Protocol validation

AWS Lambda receiving arguments can be triggered from the code (CI/CD), by a user (AWS console) and hybrid (Jenkins job).

We use “configuration” as a single validation mechanism- the same validation code used on all client sides’ flavors and on the lambda side.

“Onion” configuration

Using “levels” of configuration values population:

At first the default values being set.
Next- environment grade (stg/prod) values override the default.
Finally- externally passed values (Jenkins job/Script arguments/Terraform) override all previous.

“Configuration” protects each “level” of the values population.

e.g: Protecting the environment grade (LOCAL<QA<STG<PROD) from decreasing : once set STG, can’t be reset to QA or LOCAL.

Configuration parts reuse

Using single “Source Of Truth” file to save ALL constants is not a scalable solution. Better approach- splitting the constants per task/component and organizing them in folders, repositories, S3 buckets, AWS Systems etc. And then using same “configuration rules” to compose those constants.

e.g. Jenkins related configurations- used for infrastructure deployment (infra type [ECS/K8s/EC2], hostname, ports…). On the other hand overlapping part of these configs being used to access Jenkins API in a CI/CD code (hostname, port to access, username, password)

Idea overview

The solution is based on 2 concepts:

ConfigurationPolicy- is a smart validation schema. In simple words: this is a single place in your code to manage the configuration. ConfigurationPolicies used in all tasks to access values. For example- Jenkins configuration can be used both for triggering remote jobs (“narrow context”- few values being used). As well as part of Jenkins deployment and provisioning (“wide context”- many values). ConfigurationPolicies can implement inheritance. For example global configuration: EnvironmentNamesConfigurationPolicy can be used in many more specific ConfigurationPolicies: build, deploy, monitoring, IPC...
ConfigurationPolicyComponent- Single rule. Building block of the ConfigurationPolicy. Used to restrict single value. i.e. Any smallest piece of information- hostname, port, environment name, password etc.

ConfigurationPolicy in depth

ConfigurationPolicy is a class. Inherits from a base class which provides small set of common methods: init_from_json_file, init_from_console, init_from_dictionary etc. These methods provide all the needed options to receive, validate and populate values into the instance.
Policy inheritance and policy-composition. It’s better to use several policies in array than making unnecessary dependencies (Flat is better than nested (c)). e.g. DeploymentConfigurationPolicy should not inherit from BackendConfigurationPolicy and FrontendConfigurationPolicy. On the other hand valid example: Multiple policies should inherit from EnvironmentGradePolicy because grade (local, qa, stg, prod) is a common component in many cases.

ConfigurationPolicyComponent in depth

Each component is simply an attribute with getter and setter which control its behavior and value restrictions. “@property” in Python. Examples:

port- int in range 0-65535
instance_count- int ≥ 0
environment_grad- enum with values: LOCAL, QA, STG, PROD

Components may interact with each other. Examples:

dns_prefix- <env_grade>.<corporate_domain>.com
db_hostname- <env_grade_mongo_<host_index>

Practical example

Taken from https://github.com/AlexeyBeley/horey/tree/main/configuration_policy/tests

To run the tests (Linux/Mac): clone the repo, cd into the repo and run: “make test-configuration_policy”

Set of rules we want to implement

Grade- enum (LOCAL/QA/STG/PROD)
Grade- can be increased but can not be decreased. For example if Jenkins sets deployment grade STG, user can not decrease it to QA.
Jenkins hostname- used to deploy the service.
Jenkins hostname- in QA, STG and PROD environments must be automatically generated in a format “jenkins-<grade>”. And must be protected from explicit suppression (static value).
Jenkins hostname- in LOCAL environments can be explicitly set (dynamic value).

Implementation

We use GradeConfigurationPolicy to set rules (ConfigurationPolicyComponents) over grade. Code location: configuration_policy/tests/configuration_policies/grade_configuration_policy.py

Testing grade rules. Code location: configuration_policy/tests/test_grade_configuration.py

We use JenkinsConfigurationPolicy to set rules (ConfigurationPolicyComponents) over Jenkins hostname. Code location: configuration_policy/tests/configuration_policies/jenkins_configuration_policy.py