Terraform Level 1, Task 24: Managing Secrets with AWS Secrets Manager¶

Today's task was a crucial lesson in cloud security best practices. I used Terraform to create a secret in AWS Secrets Manager. This is the standard, secure way to handle sensitive data like database credentials, API keys, and other application secrets.

This was a fantastic exercise because it taught me to stop thinking about secrets as simple strings and start thinking about them as managed, versioned resources. I learned how to create a structured secret (with a username and password) and define it as code. This document is my very detailed, first-person guide to that entire process, with a deep dive into the code and the core concepts of secrets management.

Table of Contents¶

Terraform Level 1, Task 24: Managing Secrets with AWS Secrets Manager
Table of Contents

The Task¶

My objective was to use Terraform to create a new secret in AWS Secrets Manager. The specific requirements were: 1. All code had to be in a single main.tf file. 2. The secret's name had to be devops-secret. 3. The secret's value had to be a structured key-value pair: username: admin and password: Namin123. 4. The resource had to be created in the us-east-1 region.

My Step-by-Step Solution¶

The process involved writing a Terraform file that defined two resources: one for the secret "container" and one for the secret's actual value.

Phase 1: Writing the Code¶

In the /home/bob/terraform directory, I created my main.tf file. I wrote the following declarative code.

# 1. Configure the AWS Provider
provider "aws" {
  region = "us-east-1"
}

# 2. Define the Secrets Manager Secret Resource
# This resource creates the "shell" or container for the secret.
resource "aws_secretsmanager_secret" "devops_secret_resource" {
  name = "devops-secret"
  tags = {
    Name = "devops-secret"
  }
}

# 3. Define the Secret Version Resource
# This resource populates the secret with its first version of the actual value.
resource "aws_secretsmanager_secret_version" "devops_secret_version" {
  secret_id = aws_secretsmanager_secret.devops_secret_resource.id

  # The 'secret_string' is the actual sensitive data.
  # We use jsonencode to create a well-formatted JSON string from a map.
  secret_string = jsonencode({
    username = "admin"
    password = "Namin123"
  })
}

Phase 2: The Terraform Workflow¶

From my terminal in the same directory, I executed the three core commands. 1. Initialize: terraform init. 2. Plan: terraform plan. The output showed me that Terraform would create two resources: aws_secretsmanager_secret and aws_secretsmanager_secret_version. 3. Apply: terraform apply. After I confirmed with yes, Terraform created the secret in my AWS account. The success message confirmed the task was done.

Why Did I Do This? (The "What & Why")¶

- AWS Secrets Manager: This is a fully managed service specifically designed to help you protect and manage secrets needed to access your applications, services, and IT resources. - The Problem it Solves (Decoupling and Security): The absolute worst practice is to hardcode secrets (like passwords or API keys) directly in your application code or configuration files. This is a huge security risk. Secrets Manager solves this by: 1. Centralizing Secrets: It provides a single, secure place to store all your sensitive data. 2. Decoupling: My application code no longer contains the secret. Instead, it contains code that queries the Secrets Manager API at runtime to fetch the secret it needs. 3. Encryption: Secrets Manager automatically encrypts secrets at rest using AWS Key Management Service (KMS). 4. Fine-Grained Access Control: I can use IAM policies to control exactly which users or servers are allowed to retrieve which secrets. 5. Automatic Rotation: It has built-in capabilities to automatically rotate credentials for services like RDS databases, which is a powerful security feature. - Structured Secrets: My task required storing a username and a password. Instead of creating two separate secrets, the best practice is to store related data together. The standard way to do this is to store the secret value as a JSON string, which is exactly what the jsonencode function helped me do.

Deep Dive: A Line-by-Line Explanation of My `main.tf` Script¶

This script showed me that creating a secret is a two-step process in Terraform: creating the secret object, then creating its first version.

[Image of an application fetching a secret from AWS Secrets Manager]

# Standard provider configuration block.
provider "aws" {
  region = "us-east-1"
}

# This is the first resource block, which defines the Secret "container" itself.
# "aws_secretsmanager_secret" is the Resource TYPE.
# "devops_secret_resource" is the local NAME I use to refer to this secret.
resource "aws_secretsmanager_secret" "devops_secret_resource" {

  # The 'name' argument sets the unique name for the secret.
  name = "devops-secret"
  tags = {
    Name = "devops-secret"
  }
}

# This is the second resource block, which defines the actual VALUE of the secret.
# Every time I update this resource, Secrets Manager creates a new "version" of the secret.
resource "aws_secretsmanager_secret_version" "devops_secret_version" {

  # This is the most important line: The Dependency Link.
  # The 'secret_id' argument tells this version which secret it belongs to.
  # I am providing the ID by referencing an attribute from the resource I defined above.
  secret_id = aws_secretsmanager_secret.devops_secret_resource.id

  # 'secret_string' is where I put the sensitive data.
  # The 'jsonencode' function is a built-in Terraform function that takes a Terraform
  # map and converts it into a properly formatted JSON string. This is the standard
  # way to store structured data like a username/password pair.
  secret_string = jsonencode({
    username = "admin"
    password = "Namin123"
  })
}

Common Pitfalls¶

- Hardcoding Secrets in Code: The biggest anti-pattern. Even though the secret value is visible in my main.tf file, it is considered a security improvement because the value will be encrypted in the Terraform state file and in AWS. In a real production setup, I would use a more advanced method to inject this value from a secure vault at runtime, rather than committing it to Git. - Forgetting the secret_version resource: Just creating the aws_secretsmanager_secret resource is not enough. This only creates an empty secret with no value. The aws_secretsmanager_secret_version resource is mandatory to actually store any data. - IAM Permissions: The IAM user or role running Terraform needs secretsmanager:CreateSecret permission. The application that needs to read the secret at runtime needs secretsmanager:GetSecretValue permission (and kms:Decrypt if it's a default encrypted secret).

Exploring the Essential Terraform Commands¶

- terraform init: Prepared my working directory by downloading the aws provider plugin. - terraform validate: Checks the syntax of Terraform files. - terraform fmt: Auto-formats code to the standard style. - terraform plan: Showed me a "dry run" plan of the resources to be created. - terraform apply: Executed the plan and created the secret after I confirmed with yes. - terraform show: Shows the current state of my managed infrastructure. - terraform state list: Lists all the resources that Terraform is currently managing. -- terraform destroy: The opposite of apply. It creates a plan to destroy all the infrastructure managed by the current configuration.