Terraform

A provider is a plugin that translates Terraform's resource model into calls against a specific API. The AWS provider knows how to create S3 buckets, IAM roles, and VPCs. The Kubernetes provider knows how to manage namespaces, deployments, and services. Without a provider, Terraform has no way to speak to a platform. Providers are distributed separately from Terraform itself and are downloaded during terraform init based on the required_providers block in your configuration.

Provider versioning is a critical operational practice. Pinning providers to a specific version or a constrained range in the required_providers block ensures that the team, CI pipelines, and automated applies all use identical provider behaviour. An unpinned provider means that running terraform init on a new machine or a fresh CI runner may pull a newer provider version with different defaults, changed behaviour, or new required fields that silently break existing configurations.

Provider configuration typically includes authentication credentials and a target region or endpoint. In production pipelines, authentication should come from environment variables or a workload-identity mechanism rather than hardcoded credentials in the provider block. Storing cloud credentials inside Terraform configuration files defeats the purpose of secret management and makes rotation difficult.

A common mistake is treating the provider as a fixed background detail and upgrading it infrequently. Major provider version upgrades often rename arguments, remove deprecated resources, or change how existing resources are interpreted by the state. Upgrading providers should be treated like upgrading any dependency. Done intentionally, tested in a non-production environment first, and reviewed against the provider changelog for breaking changes.

A resource block is the fundamental unit of Terraform configuration. It declares a single infrastructure object of a specific type, such as aws_s3_bucket, google_compute_instance, or kubernetes_deployment, and specifies the attributes that define that object's configuration. Terraform maps each resource block to exactly one real object in the target system, tracking the relationship in the state file.

Good resource definitions are narrow, focused, and self-descriptive. A resource that configures ten distinct behaviours in a single block is difficult to review and makes the scope of a change hard to understand. Splitting logically separate concerns into distinct resources, even when a single API call could configure them together, makes each change smaller, easier to review, and easier to test in isolation.

Immutable versus mutable resources is an important distinction when reviewing plans. Some resource attributes can be updated in place. Changing a description, a tag, or a replica count applies without replacing the resource. Other attributes are immutable. Changing the name of an S3 bucket, the availability zone of an RDS instance, or the CIDR of a subnet forces Terraform to destroy the old resource and create a new one. The plan output marks these as resource replacement (-/+), and reviewers should treat any replacement of a stateful resource with extreme caution.

Resource dependencies in Terraform are usually inferred automatically. If resource B references an attribute of resource A, Terraform creates A before B and destroys B before A. Explicit depends_on is available when a dependency exists at a higher level than Terraform can detect from configuration alone, but it should be used sparingly. Over-use of depends_on makes the dependency graph opaque and slows plan and apply by serialising steps that could otherwise run in parallel.

Variables parameterise a Terraform configuration so the same code can produce different outcomes across environments, regions, or use cases without duplicating logic. A variable has a name, an optional type constraint, an optional description, and an optional default value. Callers provide values through variable files, environment variables prefixed with TF_VAR_, or interactive prompts. Modules receive variable values from the calling configuration.

Type constraints make variable interfaces self-documenting and catch mistakes early. Declaring a variable as type = string, type = number, or type = object({...}) gives Terraform the information to validate input before any plan runs. Without types, a misconfigured variable value may not surface as an error until Terraform tries to use it in an API call, making the failure harder to diagnose.

Sensitive variables should be marked with sensitive = true. This prevents Terraform from printing the value in plan or apply output and in CLI logs. However, sensitive = true does not prevent the value from appearing in the state file. Variables that contain credentials, connection strings, or private keys should not be passed to Terraform directly as variable values if the state file is not adequately protected. The better pattern is to load the value from a secrets manager at runtime rather than passing it as a Terraform variable.

A common mistake is accumulating too many optional variables with complex defaults, turning a module into a configuration maze. The goal of variables is to expose only the decisions the caller needs to make. If a variable is almost always left at its default and changing it breaks other assumptions, it is probably an implementation detail that should be internal to the module rather than a public interface.

Output values expose selected attributes from a Terraform configuration so that other configurations, scripts, or team members can consume them. A module that creates a VPC might output the VPC ID, the subnet IDs, and the security group IDs. The calling configuration uses these outputs to wire other resources to the network without needing to know the internal implementation of the VPC module.

Outputs are part of a module's public contract. Changing an output name or type is a breaking change for anything that depends on it. For widely used internal modules, outputs should be versioned and changed deliberately, with the same care given to changing an API endpoint in application code. Removing an output that downstream configurations depend on will cause plan failures and deployment disruptions.

Remote state data sources use outputs as their mechanism for cross-configuration sharing. A configuration can declare a terraform_remote_state data source that reads the state file of another configuration and exposes its outputs as attributes. This is powerful but creates a tight coupling between the two configurations. The consuming configuration depends on the producing configuration's state being present and consistent. The coupling should be intentional and the interface kept as narrow as possible.

Sensitive outputs should be marked sensitive = true for the same reasons as sensitive variables. However, a common misunderstanding is that sensitive outputs are protected in state. They are not. The sensitive flag prevents values from being printed in CLI output but the value is still present in the state file in plaintext. Protecting sensitive output values requires protecting the state backend with appropriate access controls and encryption.

The Terraform state file is the record that connects every resource block in your configuration to the actual object it manages in the cloud or platform. It stores resource IDs, all known attribute values, metadata about provider versions, and the dependency graph. Without the state file, Terraform cannot determine which real objects it manages, so every plan would attempt to create everything from scratch, and every apply would produce duplicate resources.

State files contain sensitive information that is often underappreciated. Resource ARNs, IP addresses, database connection strings, and sometimes actual secret values passed as resource attributes all appear in the state file in plaintext. Anyone with read access to the state file has a detailed inventory of the infrastructure and potentially access to credentials. State must be stored in a secure remote backend, such as an S3 bucket with server-side encryption and strict bucket policies, Terraform Cloud, or Google Cloud Storage with IAM-controlled access. Storing state in a local file on a developer's laptop or committing it to version control are serious security mistakes.

State locking prevents concurrent Terraform operations from corrupting the state file. If two operators or two CI jobs run terraform apply simultaneously against the same state, they can produce conflicting changes that leave the state inconsistent. Remote backends that support locking acquire an exclusive lock before any operation that modifies state. S3 combined with DynamoDB provides locking for AWS-based workflows. Terraform Cloud and Terraform Enterprise provide built-in locking. A stale lock from a crashed operation can block future runs and may require manual intervention to clear.

Manual state manipulation commands (terraform state mv, terraform state rm, terraform import) are powerful but dangerous. They modify the state file directly in ways that bypass normal plan-and-apply validation. A mistake in a state mv command can disassociate a real resource from Terraform management, causing the next apply to create a duplicate. Any manual state operation should be preceded by a state backup using terraform state pull, performed carefully with the exact resource addresses confirmed, and followed immediately by a terraform plan to verify the state is consistent before any further automated applies.

Remote state allows one Terraform configuration to read the outputs of another configuration stored in a shared backend. The terraform_remote_state data source reads the state file of a target configuration and exposes its outputs as attributes. This is the standard pattern for connecting independent configurations, such as a network configuration that creates VPCs and subnets and an application configuration that places workloads into those networks.

Remote state creates coupling between configurations. The consuming configuration cannot successfully plan or apply if the producing configuration's state is absent, locked, or has removed an output the consumer depends on. This coupling is sometimes appropriate, such as when the network team owns the VPC and the application team owns the workloads, and there is a clear ownership boundary between them. It becomes problematic when remote state is used casually to share arbitrary values between configurations without that clear boundary.

The security implications of remote state deserve explicit attention. The backend credentials used to read a remote state file must have access to that state file, which may contain sensitive information about the infrastructure it describes. A configuration that reads another team's state implicitly gains read access to everything in that state, including resource attributes that may be sensitive. The interface between configurations via remote state should be as narrow as possible. Expose only the outputs that the consumer genuinely needs.

An alternative to remote state for sharing values between configurations is to use purpose-built shared services, such as AWS SSM Parameter Store or HashiCorp Consul, to pass values between teams and systems. This decouples the producer and consumer configurations at the cost of an additional dependency. For stable, infrequently changing values like VPC IDs and subnet IDs, this approach is often more maintainable over time than remote state coupling.

A module is a container for a group of related Terraform resources that can be called from other configurations. Every Terraform configuration is technically a module, but the term usually refers to reusable units called from a root configuration. Modules are the primary mechanism for building a standardised internal library of infrastructure patterns that teams can consume without needing to understand all the implementation details.

Module versioning is essential for maintaining a stable internal library. Modules published to an internal artifact store or the public Terraform Registry should follow semantic versioning. Callers should pin to a specific version constraint rather than using a floating reference. An unpinned module reference means that running terraform init can pull a newer module version with different behaviour, different required variables, or removed outputs, potentially breaking existing configurations silently on the next init.

The interface design principle for modules is to expose only what the caller must control and hide everything else. A module with fifty input variables is not providing useful abstraction, it is just reorganising where the complexity lives. A well-designed module has a small, stable interface that answers a single clear question, handles all the internal resource wiring, tagging, encryption defaults, and access policy choices internally. Security-relevant defaults should be secure by default in the module, with exceptions requiring an explicit override rather than being left to the caller.

Testing modules is necessary for modules used across many teams. Terratest provides a Go-based integration testing framework that can apply a module to a real cloud environment, make assertions about the resulting infrastructure, and tear it down. Checkov and OPA can provide faster, lower-cost policy tests that validate module outputs against security rules without deploying to a real environment. Testing investment pays back quickly in large organisations where a defect in a widely used module creates work for every team that uses it.

The plan step asks Terraform to calculate and display every change that would result from applying the current configuration against the current state. The output shows resources to be created (marked +), destroyed (marked -), and updated in place (marked ~). For updates, it lists the exact attribute values changing from old to new. For resource replacements, it uses -/+ to signal that the old resource will be deleted and a new one created. This output is the primary tool for human review before any change touches live infrastructure.

Reading a plan output is a skill. Reviewers should look beyond line counts and spot. Resource replacements in stateful services (a database or a load balancer being replaced is almost always high-risk), large numbers of unexpected resources appearing or disappearing, changes to security-sensitive attributes such as IAM policies, encryption settings, and public access flags, and changes to production that should have been validated in a staging environment first. The -/+ symbol on a production database is the single most important thing to catch before approving a plan.

In automated pipelines, terraform plan runs on every pull request and the output is posted as a comment for reviewers. The apply step runs only after the PR is approved and merged to the protected branch. This two-step model enforces review before apply even in a fully automated workflow. Tools like Atlantis and Terraform Cloud implement this model with additional features such as state locking during apply, audit trails for every plan and apply, and environment-specific approval rules.

Applying without reviewing the plan is one of the most dangerous practices in Terraform workflows. It is easy to assume that a small code change has a predictable and narrow effect, but Terraform resource lifecycles can produce large cascading changes from small configuration edits. Terraform Cloud supports a plan-only mode where the plan is saved and must be explicitly approved before apply proceeds, making accidental unreviewed applies impossible in automated contexts. Teams that run terraform apply directly without reviewing the plan output first are operating without the primary safety mechanism the tool provides.