Chapter 8: Terraform Directory Strategies

"Managing multiple environments with Terraform" is a debated topic in the Terraform community, as there is no definitive answer. It largely depends on the specific requirements and needs of your environment. Ultimately, we all use similar Terraform code to create resources such as Kubernetes clusters and virtual machines, but some people prefer to use resources while others find modules more beneficial. A key question to consider is how to apply the same code in different environments. Here are a few known methods:

1. Terraform Workspaces

2. Git Branch for each environment

3. Environment variables for each environment

4. Variable file inputs per environment

5. Terragrunt

According to the official Terraform documentation, one solution for managing multiple environments is using Terraform Workspaces. This allows you to use a unified code across different environments and easily switch between workspaces using the "terraform apply" command. However, using workspaces can reduce the flexibility of your code, as you may need to use many Terraform "if-else" statements to handle differences between environments. For example, you may want to use smaller virtual machines in the development environment and larger machines in production. This could result in code similar to the following for each infrastructure component:

resource "azurerm_linux_virtual_machine" "onur" {
  # ... other output omitted for brevity
  name = var.name
  size = terraform.workspace == "prod" ? "Standard_D4_v4" : "Standard_D2_v4"
  # ... other output omitted for brevity}

One disadvantage of using Terraform workspaces is that you can only have one remote backend for all of the workspaces. This can be problematic if your organization has regulations requiring production data (including the state file) to be stored in a different backend, or if you have different subscriptions for different environments. It can also be easy to forget to switch to the correct workspace when testing things locally, leading to the risk of pushing changes to the wrong environment. These limitations are what personally keep me from using Terraform workspaces.

Another solution is to use different branches for each environment. With this approach, you can have completely different Terraform code and variable files for each environment, as well as a dedicated remote state for each environment. However, this approach has a significant drawback: the main Terraform code in different branches will eventually diverge, leading to completely different environments. For this reason, I do not consider this option to be a valid one.

Another solution is to pass different environment variables from the command line for each environment, but this has the disadvantage of requiring you to pass a large number of variables, even if you can automate this process to some extent.

Another solution is using Terragrunt, which is a promising alternative. It provides a number of advantages, as listed on their website, and has proven to be superior to the other methods described above in my personal experience. However, one potential downside is the need to depend on another tool (although it is a wrapper around Terraform) rather than using Terraform directly. It is important to consider the availability of this tool within your organization and the potential impact on your dependency management.

The method I personally prefer is simple to implement and provides a lot of flexibility. I use the same .tf files for all my environments to ensure that the infrastructure I create in one environment will at least use the same Terraform code. Then, I create a terraform.tfvars file for each environment, as follows:

.
|--- main.tf
|--- outputs.tf
|--- variables.tf
.
.
.
|--- environment
|   |--- dev
|   |   |--- terraform.tfvars
|   |   |--- backend.hcl
|   |--- prod
|   |   |--- terraform.tfvars
|   |   |--- backend.hcl
|   |--- staging
|       |--- terraform.tfvars
|       |--- backend.hcl

To allow each environment to use its own dedicated remote state file, you can initialize Terraform (typically as part of your CI/CD pipelines). This will provide isolation, so that failures or losses in one remote state file will not impact the others. Once initialized, you can run your Terraform "plan/apply/destroy" commands for the dev environment like this:

terraform apply -var-file=environment/dev/terraform.tfvars

Using this approach, you can have a simpler main.tf file because you don't need to compare which environment you are running your code against. Instead, you pass a completely different variable file for each environment. Another advantage is that you can keep your .tf files and .tfvars files in different versions if you are uploading your artifacts to an artifact repository. Some people consider this to be a best practice, as it keeps source code and configuration in separate artifacts.

While using multiple Terraform configurations with different backends can seem like a simple solution and an alternative to Terraform workspaces, it does have some downsides. One downside is that if you want to keep infrastructure resources with persistent storage in a different remote state file, this approach will not allow you to do so. To overcome this limitation, you may need to divide your code into multiple resources and create each resource in a dedicated main.tf file to isolate the state files. However, doing so can be contrary to the philosophy of Terraform, which is to declare the entire infrastructure in a single configuration.

To address this issue, you may want to consider creating infrastructure stacks to separate your stateful and stateless infrastructure and create them in parallel or sequentially as needed. Terragrunt is a tool that can be used to achieve this kind of flexibility with minimal complexity. In this approach, you can define your infrastructure stacks as modules and create them as separate Terraform configurations that are managed by Terragrunt. Each stack has its own state file, which makes it easy to manage stateful resources separately from stateless ones. By doing this, you can achieve the benefits of a modular approach while still adhering to the principles of Terraform.

Ultimately, there is no one-size-fits-all approach to managing multiple environments with Terraform. The best solution will depend on your organizational requirements and personal preferences. My personal preference is the method I described last.