Chapter 6: Terraform Modules

What is Terraform Module?

A Terraform module is a self-contained package of Terraform configurations that are managed as a group. Using modules has several advantages. It can be considered as encapsulating your Terraform resource definitions under a versioned software package, which gives the consumers of the module the ability to edit the exposed parameters as needed. Using modules offers several benefits. It allows you to reuse and share code across multiple configurations, which can save time and reduce errors. Additionally, modules can help you to better organize and maintain your infrastructure as code, making it easier to understand and modify. By encapsulating infrastructure resources and their associated configurations, modules can also make it easier to test and version your code, ensuring that changes are tracked and managed effectively over time.

For example, imagine you need to create an Azure Kubernetes Cluster for your project, but you also anticipate needing to create another cluster for a different project in the future. While you can certainly define an Azure Kubernetes Service resource and apply it to your current project, it will not be easy to distribute what you have written or make modifications in a versioned manner using the Terraform resource approach.

However, with the Terraform module approach, you can distribute your code, fix any possible issues, and increment the version by following the semantic versioning approach. This makes it easier to manage and share infrastructure components across projects and teams, and ensures that changes can be tracked and managed effectively over time.

To define a module, you use the module block, which can take input variables and return output values. Modules can be used in the root module of a Terraform configuration or within other modules. They can be sourced from local files, directories, a version control system, or a Terraform Registry.

Terraform Module Structure

Hashicorp has a well-documented module structure on their website. I strongly recommend reviewing the details of the structure on the HashiCorp website, which you can find here. Here is a summary of the structure to get you started:

|--- README.md
|--- main.tf
|--- variables.tf
|--- outputs.tf
|--- ...
|--- modules/
|   |--- module1/
|   |   |--- README.md
|   |   |--- variables.tf
|   |   |--- main.tf
|   |   |--- outputs.tf
|   |--- module2/
|   |--- .../
|--- examples/
|   |--- example1/
|   |   |--- main.tf
|   |--- example2/
|   |--- .../

Let's quickly describe the components before we proceed with a simple real world example:

• root: This is the root directory of the module. It should contain the main Terraform configuration files for the module, including main.tf, variables.tf, and outputs.tf.

• main.tf: This file should contain the main Terraform configuration for the module, including the resource blocks that define the infrastructure that the module creates.

• variables.tf: This file should define the input variables that the module accepts.

• outputs.tf: This file should define the output values that the module returns.

• README.md: This file should contain documentation for the module, including information about its purpose, how to use it, and any relevant examples.

• examples/: This directory should contain examples of how to use the module. These can be in the form of standalone Terraform configuration files or scripts that demonstrate how to use the module.

• modules/: This directory should contain any child modules that the root module depends on.

I personally prefer to create a dedicated versions.tf file in the module directory so that users can easily see the required version of the module.

How to write a basic module?

Writing a module is not different than writing a regular Terraform resource definition. Something to keep in mind while writing a module is that a module is not intended to create the entire infrastructure. Rather, they are building blocks to compose your environment, so you should divide your code into smaller modules as much as possible. One module should serve only one purpose. For example, the main.tf file that we have used in previous chapters created a resource group, Azure Service Plan, and Azure Linux Web App. These should not all be created as a single module unless you have a valid reason. Ideally, you should have one module for the Azure Service Plan and one module for the Azure Linux Web App.

You can certainly create a module for a resource group, but it is such a small piece of the resource block that I personally think it would be over-engineering to create a module for it.

First, let's create a new directory called basic-infra-with-modules. Then, let's create two more directories as follows:

.
|--- modules
    |--- linux-web-app
    |--- service-plan

Let's start by creating the module for Azure Service Plan. We will use the official documentation as our guideline as much as possible. The suggested Terraform module structure suggests that we create some files and directories. Let's create the bare minimum to start with as follows:

> tree service-plan
service-plan
|--- README.md
|--- examples
|   |--- basic
|       |--- main.tf
|       |--- terraform.tfvars
|       |--- variables.tf
|--- main.tf
|--- outputs.tf
|--- variables.tf
|--- versions.tf

Remember that Terraform modules accept arguments. If a module resource needs to be placed in a resource group, I suggest you provide the name of the resource group in the module as a variable, so that users of the module can have more flexibility to use their own naming conventions.

Let's start with module's main.tf file:

# ./main.tf
provider "azurerm" {
  features {}
}

locals {
  # Define SKUs
  isolated_skus    = ["I1", "I2", "I3", "I1v2", "I2v2", "I3v2"]
  elastic_skus     = ["EP1", "EP2", "EP3"]
  consumption_skus = ["Y1"]
}
resource "azurerm_service_plan" "asp" {
  # Required Arguments
  name                = var.name
  resource_group_name = var.resource_group_name
  location            = var.location
  os_type             = "Linux"
  sku_name            = var.sku_name

  # Optional Arguments
  worker_count                 = contains(local.consumption_skus, var.sku_name) ? null : var.worker_count
  maximum_elastic_worker_count = contains(local.elastic_skus, var.sku_name) ? null : var.maximum_elastic_worker_count
  app_service_environment_id   = var.app_service_environment_id
  per_site_scaling_enabled     = var.per_site_scaling_enabled
  zone_balancing_enabled       = var.zone_balancing_enabled
  tags                         = var.tags

  # Lifecycle check for SKUs
  lifecycle {
    precondition {
      condition     = var.app_service_environment_id == null && !contains(local.isolated_skus, var.sku_name)
      error_message = "Isolated SKUs (I1, I2, I3, I1v2, I2v2, and I3v2) can only be used with App Service Environments"
    }
  }
}

We defined some local variables in the main.tf file to be used in our comparison. The Azure Service Plan has some limitations against some of the SKUs and we will check them. Let's check the condition on line 21. Here, we are saying that the worker_count will be set to null (unset) if the sku_name variable is one of the available consumption SKUs. Otherwise, it will be set to the worker_count value. We also use a similar logic on line 22. We use a different method between lines 29 and 34 if we want to use one of the isolated SKUs. Isolated SKUs can only be used with App Service Environments and we are checking if the app_service_environment_id variable is set while using an Isolated SKU. This condition is checked as a precondition and Terraform will not let us provision the resource (or change it) if we do not meet the condition. An example error will be as follows:

> terraform plan

| Error: Resource precondition failed
|
|   on ../../main.tf line 30, in resource "azurerm_service_plan" "asp":
|   30:       condition     = var.app_service_environment_id == null && !contains(local.isolated_skus, var.sku_name)
|     |-----------------
|     | local.isolated_skus is tuple with 6 elements
|     | var.app_service_environment_id is null
|     | var.sku_name is "I2v2"
|
| Isolated SKUs (I1, I2, I3, I1v2, I2v2, and I3v2) can only be used with App Service Environments

Here, it is clear that we provided an SKU with the "I2v2" value, but have not provided any value for app_service_environment_id.

Please note that on line 17, we have hard-coded the os_type parameter to Linux, which means that consumers of the module cannot change it. It may be necessary or desirable to hard-code certain parameters if they must always be fixed or if they are required to meet organizational or regulatory requirements.

Let's now define the variables.tf file:

# ./variables.tf
variable "name" {
  type        = string
  description = "The name which should be used for this Service Plan."
}

variable "location" {
  type        = string
  description = "The Azure Region where the Service Plan should exist."
}

variable "resource_group_name" {
  type        = string
  description = "The name of the Resource Group where the AppService should exist."
}

variable "sku_name" {
  type        = string
  description = "The SKU for the plan."
  default     = "F1"

  validation {
    condition     = try(contains(["B1", "B2", "B3", "D1", "F1", "FREE", "I1", "I2", "I3", "I1v2", "I2v2", "I3v2", "P1v2", "P2v2", "P3v2", "P1v3", "P2v3", "P3v3", "S1", "S2", "S3", "SHARED", "Y1", "EP1", "EP2", "EP3", "WS1", "WS2", "WS3"], var.sku_name), true)
    error_message = "Invalid sku_name. Possible values include B1, B2, B3, D1, F1, I1, I2, I3, I1v2, I2v2, I3v2, P1v2, P2v2, P3v2, P1v3, P2v3, P3v3, S1, S2, S3, SHARED, EP1, EP2, EP3, WS1, WS2, WS3, and Y1"
  }
}

variable "app_service_environment_id" {
  type        = string
  description = "The ID of the App Service Environment to create this Service Plan in."
  default     = null
}

variable "maximum_elastic_worker_count" {
  type        = string
  description = "The maximum number of workers to use in an Elastic SKU Plan. Cannot be set unless using an Elastic SKU."
  default     = null
}

variable "worker_count" {
  type        = number
  description = "The number of Workers (instances) to be allocated."
  default     = 1
}

variable "per_site_scaling_enabled" {
  type        = bool
  description = "Should Per Site Scaling be enabled."
  default     = false
}

variable "zone_balancing_enabled" {
  type        = bool
  description = "Should the Service Plan balance across Availability Zones in the region."
  default     = false
}

variable "tags" {
  type        = map(string)
  description = "A mapping of tags which should be assigned to the AppService"
  default     = {}
}

We have defined every variable here and set the default values for some of the variables. This is especially important to keep the module clean if you expect some of the variables to be fixed in many cases.

Then it is time to define the outputs.tf file. Here, we expose the outputs of our module which can also be referenced by another resource(s) or module(s) if any input is required. For example, the Azure Linux Web App will require the Azure Service Plan ID of the respective Service Plan, so it is a must to export that variable for others to consume.

# ./outputs.tf
output "service_plan_id" {
  description = "ID of the Service Plan"
  value       = azurerm_service_plan.asp.id
}

output "service_plan_name" {
  description = "Name of the Service Plan"
  value       = azurerm_service_plan.asp.name
}

If the value you are exporting is sensitive, you can add sensitive = true parameter to the output block.

The versions.tf file is no surprise. You can view it in the GitLab repository.

Let's now provide a basic example of how to use our module. You should provide at least one example under the examples directory, but more are also welcome. Here, we have provided a basic usage example as follows:

# examples/basic/main.tf
provider "azurerm" {
  features {}
}

terraform {
  required_version = "~> 1.3"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.37"
    }
  }
}

resource "azurerm_resource_group" "example" {
  name     = var.resource_group_name
  location = var.location
}

module "example_asp" {
  # Source the module locally
  source = "../.."

  # Test with required arguments only
  name                = var.name
  resource_group_name = azurerm_resource_group.example.name
  location            = var.location
  sku_name            = var.sku_name
}

output "asp_rg_name" {
  value = azurerm_resource_group.example.name
}

output "asp_id" {
  value = module.example_asp.service_plan_id
}

Please note how we exported the Azure Service Plan ID. We used the attribute which is exposed by the module itself.

We should also define the variables that we provided as input to our module under variables.tf, as we do for a regular resource definition.

# examples/basic/variables.tf
variable "name" {
  type        = string
  description = "The name which should be used for this Service Plan."
}

variable "location" {
  type        = string
  description = "The Azure Region where the Service Plan should exist."
}

variable "resource_group_name" {
  type        = string
  description = "The name of the Resource Group where the AppService should exist."
}

variable "sku_name" {
  type        = string
  description = "The SKU for the plan."
  default     = "F1"

  validation {
    condition     = try(contains(["B1", "B2", "B3", "D1", "F1", "FREE", "I1", "I2", "I3", "I1v2", "I2v2", "I3v2", "P1v2", "P2v2", "P3v2", "P1v3", "P2v3", "P3v3", "S1", "S2", "S3", "SHARED", "Y1", "EP1", "EP2", "EP3", "WS1", "WS2", "WS3"], var.sku_name), true)
    error_message = "Invalid sku_name. Possible values include B1, B2, B3, D1, F1, I1, I2, I3, I1v2, I2v2, I3v2, P1v2, P2v2, P3v2, P1v3, P2v3, P3v3, S1, S2, S3, SHARED, EP1, EP2, EP3, WS1, WS2, WS3, and Y1"
  }
}

Then we need to provide what we set for non-default variables under the terraform.tfvars file:

# examples/basic/terraform.tfvars
name                = "example-asp-eus"
resource_group_name = "rg-example-asp"
location            = "East US"
sku_name            = "F1"

Let's test our example to see how it works. We can run terraform init, terraform apply, and terraform destroy against the example.

Please note that the Terraform authentication environment variables have to be exported before running any Terraform command. We will not use the Terraform remote backend for the example.

> terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_resource_group.example will be created
  + resource "azurerm_resource_group" "example" {
      + id       = (known after apply)
      + location = "eastus"
      + name     = "rg-example-asp"
    }

  # module.example_asp.azurerm_service_plan.asp will be created
  + resource "azurerm_service_plan" "asp" {
      + id                           = (known after apply)
      + kind                         = (known after apply)
      + location                     = "eastus"
      + maximum_elastic_worker_count = (known after apply)
      + name                         = "example-asp-eus"
      + os_type                      = "Linux"
      + per_site_scaling_enabled     = false
      + reserved                     = (known after apply)
      + resource_group_name          = "rg-example-asp"
      + sku_name                     = "F1"
      + worker_count                 = 1
      + zone_balancing_enabled       = false
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + asp_id      = (known after apply)
  + asp_rg_name = "rg-example-asp"

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

The only missing piece is to provide README documentation for our module. I would suggest taking a look at the terraform-docs repository to learn how to generate basic documentation with minimal effort. The basic README documentation should include:

• The definition of the module

• Usage instructions

• The versions of Terraform and the provider that were tested

• Variable types and definitions

Now our module is ready to be used. You can even upload it to a public or private repository with an initial version and call it as follows in any of your Terraform code:

module "service-plan" {
  source  = "yasarlaro/service-plan/azurerm"
  version = "1.0.0"
}

Please note that the module name, source location, and version can be different in your case.

I am leaving the creation of the Azure Linux Web App module to you. You should try to start with a basic version and then add more checks and controls over time.

How to replace a Terraform Resource with Terraform Module?

Now we have a module, and we can try using it in our example. Let's go to the root directory and create a main.tf file. We can actually just copy everything from the previous example. The directory structure will be like this:

.
|--- main.tf
|--- backend.tf
|--- outputs.tf
|--- variables.tf
|--- versions.tf
|--- environment
|   |--- DEV
|   |   |--- backend.hcl
|   |   |--- terraform.tfvars
|   |--- PROD
|       |--- backend.hcl
|       |--- terraform.tfvars
|--- modules
|   |--- linux-web-app
|   |--- service-plan
|       |--- README.md
|       |--- examples
|       |   |--- basic
|       |       |--- main.tf
|       |       |--- terraform.tfstate
|       |       |--- terraform.tfstate.backup
|       |       |--- terraform.tfvars
|       |       |--- variables.tf
|       |--- main.tf
|       |--- outputs.tf
|       |--- variables.tf
|       |--- versions.tf

Please initialize Terraform for the DEV environment and then run terraform plan to ensure that the infrastructure is up to date and ready for us to begin.

> terraform init -backend-config=environment/DEV/backend.hcl --reconfigure

Initializing the backend...

# ... other output omitted for brevity

> terraform plan -var-file=environment/DEV/terraform.tfvars

# ... other output omitted for brevity

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Please open the main.tf file and make the following modifications:

# ./main.tf

# Create a resource group
resource "azurerm_resource_group" "basic-infra" {
  name     = var.resource_group_name
  location = var.location
}

module "basic-infra" {
  # Local module sourcing
  source = "./modules/service-plan"

  name                = var.service_plan_name
  resource_group_name = azurerm_resource_group.basic-infra.name
  location            = azurerm_resource_group.basic-infra.location
  os_type             ="Linux"
  sku_name            = var.sp_sku_name
}

# Create a Linux Web App
resource "azurerm_linux_web_app" "basic-app" {
  name                = var.webapp_name
  resource_group_name = azurerm_resource_group.basic-infra.name
  location            = azurerm_resource_group.basic-infra.location
  service_plan_id     = module.basic-infra.service_plan_id

  site_config {
    always_on = var.webapp_always_on
  }

  lifecycle {
    precondition {
      # always_on must be explicitly set to false when using Free, F1, D1, or Shared Service Plans.
      condition     = !var.webapp_always_on && contains(["Free", "F1", "D1", "Shared"], var.sp_sku_name)
      error_message = "webapp_always_on must be explicitly set to false when using Free, F1, D1, or Shared Service Plans"
    }
  }
}

We have replaced the resource "azurerm_service_plan" "basic-infra" with the module definition on line 9. We also needed to edit line 25 to provide the Service Plan ID from the exported output of the module. Please note that the names of the Azure resources have not been changed and there should not be any changes detected on the Azure side.

We also had a reference to the azurerm_service_plan resource in the outputs.tf file, so let's also change it before running any Terraform command. It should be replaced as follows:

output "sp_name" {
  description = "Name of the Azure service plan"
  value       = module.basic-infra.service_plan_name
}

Please run terraform plan again:

> terraform plan -var-file=environment/DEV/terraform.tfvars

| Error: Module not installed
|
|   on main.tf line 7:
|    7: module "basic-infra" {
|
| This module is not yet installed. Run "terraform init" to install all modules required by this configuration.

As you can see, Terraform tried to use the module but it failed because the module is not locally downloaded for Terraform to use under the .terraform directory. Please initialize Terraform again using the -reconfigure option to be safe, and then run terraform plan:

> terraform plan -var-file=environment/DEV/terraform.tfvars

# ... other output omitted for brevity

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
  - destroy

Terraform will perform the following actions:

  # azurerm_linux_web_app.basic-app will be updated in-place
  ~ resource "azurerm_linux_web_app" "basic-app" {
    # ... other output omitted for brevity
      ~ service_plan_id                   = "/subscriptions/xxx/resourceGroups/rg-dev-basic-infra-neu/providers/Microsoft.Web/serverfarms/dev-asp-001" -> (known after apply)
        tags                              = {}
        # (17 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # azurerm_service_plan.basic-infra will be destroyed
  # (because azurerm_service_plan.basic-infra is not in configuration)
  - resource "azurerm_service_plan" "basic-infra" {
      - id                           = "/subscriptions/xxx/resourceGroups/rg-dev-basic-infra-neu/providers/Microsoft.Web/serverfarms/dev-asp-001" -> null
      # ... other output omitted for brevity
    }

  # module.basic-infra.azurerm_service_plan.asp will be created
  + resource "azurerm_service_plan" "asp" {
      + id                           = (known after apply)
      # ... other output omitted for brevity
    }

Plan: 1 to add, 1 to change, 1 to destroy.

As you can see above, Terraform wanted to destroy the existing infrastructure and create it again with exactly the same settings. The reason is that Terraform stores the infrastructure state in a state file with objects, and our service plan was kept under an azurerm_service_plan object, which has been replaced by a module in our Terraform code. Terraform compares the desired state and does not find a Service Plan with a module object type, so it tries to create one. Similarly, it checks its state file and notices that the azurerm_service_plan resource no longer exists in the desired state and decides to remove it. This is an expected behavior of Terraform, but it is not the desired behavior for us. However, we can fix this issue by adding a move block in the main.tf file as follows:

moved {
  from = azurerm_service_plan.basic-infra
  to   =  module.basic-infra.azurerm_service_plan.asp
}

If we run the plan command again:

> terraform plan -var-file=environment/DEV/terraform.tfvars
# ... other output omitted for brevity

Terraform will perform the following actions:

  # azurerm_service_plan.basic-infra has moved to module.basic-infra.azurerm_service_plan.asp
    resource "azurerm_service_plan" "asp" {
        id                           = "/subscriptions/xxx/resourceGroups/rg-dev-basic-infra-neu/providers/Microsoft.Web/serverfarms/dev-asp-001"
        name                         = "dev-asp-001"
        tags                         = {}
        # (10 unchanged attributes hidden)
    }

Plan: 0 to add, 0 to change, 0 to destroy.

Now we can safely run the terraform apply command for all impacted environments.

You may want to delete the move block after applying it to all environments, or keep it for historical reasons.