Bringing Existing Azure Infrastructure Under Terraform Management: A Practical Guide

As cloud environments grow, managing resources manually or through the Azure Portal can quickly become unwieldy. Infrastructure as Code (IaC) with Terraform offers a powerful solution for consistent, repeatable, and version-controlled deployments. But what if you already have a substantial Azure footprint not managed by Terraform? This blog post will guide you through the process of “onboarding” your existing Azure infrastructure into Terraform, focusing on generating configuration and managing state.

The Challenge of Existing Infrastructure

Traditionally, bringing existing resources under Terraform management involved a painstaking manual process:

1. Manually writing HCL: You’d have to write Terraform configuration files (HCL) that precisely matched your existing resources in Azure.
2. Importing into state: Then, for each resource, you’d use the terraform import command to link the HCL configuration to the live resource’s state in Azure. This ensures Terraform recognizes the resource as managed. This method is prone to errors, especially for complex environments, and can be incredibly time-consuming.

Introducing terraform plan -generate-config-out (Experimental!)

HashiCorp has been actively developing features to simplify this process. One such experimental feature is the terraform plan -generate-config-out="generate.tf" command. This command aims to automate the generation of HCL configuration for resources that Terraform detects in your state but don’t yet have corresponding configuration files.

Important Disclaimer: It’s crucial to understand that -generate-config-out is an experimental feature. This means its behavior might change, it might not cover all resource types or configurations perfectly, and the generated output will likely require manual review and modification. Do not use this in production environments without extensive testing and validation.

Step-by-Step Onboarding with Generated Configuration

Let’s walk through the process:

Prerequisites:

1. Azure CLI: Ensure you have the Azure CLI installed and authenticated to your Azure subscription (az login).
2. Terraform CLI: Have Terraform CLI installed (version 1.5 or newer is recommended for this feature).
3. Terraform Project Directory: Create an empty directory for your new Terraform project.

1. Define your Provider and Backend

First, set up your main.tf (or providers.tf) with your Azure provider configuration and, importantly, your remote backend for state management. Storing your state remotely (e.g., in an Azure Storage Account) is a best practice for team collaboration and state durability.

Terraform

# provider.tf

Note: You’ll need to create the your-terraform-state-rg resource group, yourtfstatesa storage account, and tfstate container in Azure manually before running terraform init.

2. Initialize Terraform

Navigate to your project directory in the terminal and run:

Bash

This command initializes your working directory, downloads the Azure provider, and configures your remote backend.

3. Import Existing Resources into State

Before you can generate configuration, Terraform needs to know about the existing resources. This is where the import block (Terraform 1.5+) comes into play. For each resource you want to onboard, you’ll add an import block to your .tf files.

Let’s say you have an existing Azure Resource Group named my-existing-rg and a Virtual Network named my-existing-vnet within it.

Terraform

# import.tf (or append to main.tf)

How to get the Azure Resource ID:

• Azure Portal: Navigate to the resource, go to its “Properties” or “JSON view,” and copy the Resource ID.
• Azure CLI: Use az resource show --name <resource-name> --resource-group <rg-name> --resource-type <resource-type> --query id --output tsv

After defining your import blocks, run terraform plan:

Bash

Terraform will detect the import blocks and attempt to reconcile them with your state. If successful, it will show a plan to import these resources. Now, apply the plan to import them into your state:

Bash

This step will populate your Terraform state file (terraform.tfstate) with the details of your existing Azure resources. However, you still don’t have the HCL configuration for them.

4. Generate Configuration from State

This is where the experimental generate-config-out flag comes in. After importing resources into your state, run:

Bash

terraform plan -generate-config-out=

AdvertisementPrivacy Settings

Terraform will read your state file, identify resources that are managed but lack HCL definitions in your current working directory, and attempt to generate the corresponding configuration into the specified file (generated.tf in this case).

5. Review and Rationalize the Generated Configuration

Open generated.tf. You will find HCL code representing your imported Azure resources.

Expectations and Necessary Modifications:

• Literal Representation: The generated configuration is often very literal. It will include every attribute and its explicit value as it exists in Azure. This might mean:
  • Hardcoded values where you might prefer variables or local values.
  • Explicit null values for optional attributes that you might normally omit.
  • No logical grouping or modularization.
• Experimental Nature: As an experimental feature, it might not generate perfect or complete configurations for all resource types or complex dependencies. Some attributes might be missing or incorrectly represented.
• References vs. IDs: It might hardcode resource IDs instead of creating logical Terraform references (e.g., resource_group_name = "my-rg" instead of resource_group_name = azurerm_resource_group.example_rg.name). This is a key area for rationalization.

Your Tasks After Generation:

1. Validation: Immediately after generation, run terraform plan again without the -generate-config-out flag. This will show you any differences between the generated HCL and the actual state. Bash terraform plan You will likely see some “changes” or “drift.” These indicate parts of the generated HCL that don’t perfectly match the live resource or where Terraform needs more explicit instruction.
2. Refinement:
   • Replace Hardcoded IDs with References: This is crucial for maintainability. Instead of /subscriptions/.../resourceGroups/my-rg, replace it with azurerm_resource_group.example_rg.id or azurerm_resource_group.example_rg.name.
   • Introduce Variables: Identify repeatable values (e.g., location, environment names) and move them into variables.tf.
   • Create Locals: Use locals for derived values or to simplify complex expressions.
   • Modularize: For larger infrastructure, break down the generated configuration into reusable modules (e.g., a network module, a compute module).
   • Remove Redundancies: Delete attributes that are null or default and don’t need to be explicitly set.
   • Add Comments: Document the purpose of resources and any non-obvious configurations.
   • Review Lifecycle Hooks: Consider if lifecycle blocks (e.g., ignore_changes) are necessary for any specific resources where manual changes are expected.
3. Iterative Planning and Validation: After each set of modifications, run terraform plan to ensure your changes are aligned with the desired state and don’t introduce unintended drift. The goal is to achieve a “No changes. Your infrastructure matches the configuration.” message.

6. Remove Import Blocks (Optional but Recommended)

Once you have successfully imported the resources and generated / rationalized their HCL configuration such that terraform plan shows no changes, the import blocks in your .tf files are no longer strictly necessary. You can remove them to keep your configuration clean. Terraform will now manage these resources solely based on their presence in the state file and the HCL configuration.

Conclusion

Onboarding existing Azure infrastructure into Terraform is a significant step towards robust cloud management. While the terraform plan -generate-config-out command offers an exciting experimental shortcut for generating initial HCL, it’s not a magic bullet. It provides a valuable starting point, but the generated output requires careful review, validation, and rationalization to become truly production-ready, maintainable Infrastructure as Code. Embrace the iterative process of generating, validating, and refining, and you’ll soon have your existing Azure environment fully managed by Terraform!