Skip to content

Terraform Level 1, Task 12: Creating a Public S3 Bucket (The Modern Way)

Today's task was a masterclass in how cloud infrastructure and the tools that manage it are constantly evolving. My objective was to create a publicly accessible S3 bucket. While this seems simple, I discovered that recent AWS security enhancements have made the "old" way of doing this with Terraform obsolete, leading to a complex troubleshooting journey.

This document is my detailed, first-person account of that journey. I'll cover the initial failures, the diagnosis of the root cause (default "Block Public Access" settings), and the final, modern, multi-resource solution that is now required to correctly create a public S3 bucket with Terraform.

Table of Contents

The Task

My objective was to use Terraform to create a new, publicly accessible AWS S3 bucket. The requirements were: 1. All code had to be in a main.tf file. 2. The bucket's name had to be datacenter-s3-xxxx. 3. The bucket needed to be created in the us-east-1 region. 4. It had to be made publicly accessible using an ACL.

My Step-by-Step Solution (The One That Worked)

The solution required a multi-resource approach to first create the bucket, then disable the default security blocks, and finally apply the public ACL.

Phase 1: Writing the Code

In the /home/bob/terraform directory, I created my main.tf file with the following complete and correct code. 

provider "aws" {
  region = "us-east-1"
  # This line, found in provider.tf, was critical for the lab environment
  s3_use_path_style = true 
}

# 1. Define the bucket itself, without any ACL.
resource "aws_s3_bucket" "datacenter_bucket_resource" {
  bucket = "datacenter-s3-3563" # My specific bucket name for this attempt
  tags = {
    Name = "datacenter-s3-3563"
  }
}

# 2. Add a resource to explicitly disable the "Block Public Access" settings.
resource "aws_s3_bucket_public_access_block" "public_access_block" {
  bucket = aws_s3_bucket.datacenter_bucket_resource.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

# 3. Add a resource to manage ownership controls, a prerequisite for ACLs.
resource "aws_s3_bucket_ownership_controls" "ownership_controls" {
  bucket = aws_s3_bucket.datacenter_bucket_resource.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

# 4. Finally, apply the public ACL, but only after the other resources are created.
resource "aws_s3_bucket_acl" "bucket_acl" {
  depends_on = [
    aws_s3_bucket_public_access_block.public_access_block,
    aws_s3_bucket_ownership_controls.ownership_controls,
  ]

  bucket = aws_s3_bucket.datacenter_bucket_resource.id
  acl    = "public-read"
}

Phase 2: The Terraform Workflow

From my terminal in the same directory, I executed the standard workflow. 1. Initialize: terraform init 2. Plan: terraform plan (This showed a plan to create 4 new resources). 3. Apply: terraform apply After I confirmed with yes, Terraform created all four resources in the correct order, and the process completed successfully.

My Troubleshooting Journey: A Two-Part Failure

This was a complex problem that I had to solve in two stages.

  • Failure 1: The DNS Lookup Error

    • Symptom: My very first terraform apply failed with a no such host error when trying to connect to a URL like http://datacenter-s3-5847.aws:4566/.
    • Diagnosis: This was a DNS error specific to the lab's AWS simulator. The provider was trying to use "virtual-hosted-style" S3 URLs, which the simulator's network couldn't resolve.
    • Solution: I had to add s3_use_path_style = true to the provider "aws" block (which was correctly placed in provider.tf). This forced Terraform to use "path-style" URLs (http://aws:4566/datacenter-s3-5847/), which the simulator could handle.

  • Failure 2: The Validation Error

    • Symptom: After fixing the DNS issue, terraform apply ran successfully! However, the lab's validation script failed, saying the public s3 bucket doesn't exist.
    • Diagnosis: This was the most confusing part. My code had acl = "public-read". Why wasn't the bucket public? I learned that recent versions of the AWS provider align with new AWS security defaults. All new buckets have "Block all public access" enabled by default. This new setting overrides the old acl argument, so my bucket was being created but remained private. The validation script saw it wasn't public and reported that the "public bucket" didn't exist.
    • Solution: The fix was to adopt the new, modern Terraform approach: use the aws_s3_bucket_public_access_block resource to explicitly turn off these protections, and then use the separate aws_s3_bucket_acl resource to apply the public-read setting.

Why Did I Do This? (The "What & Why")

- S3 (Simple Storage Service): This is AWS's object storage service, designed for storing and retrieving any amount of data. A bucket is the root-level container for your objects (files). - S3 Public Access: Making an S3 bucket public is a powerful feature, often used for hosting static websites, but it's also a significant security consideration. Because of past security incidents, AWS has made it intentionally harder to make buckets public. The "Block all public access" feature is a global safety switch that is now on by default. - ACL (Access Control List): This is one of the methods for controlling access to S3 buckets. The public-read ACL is a "canned" policy that grants read-only access to anyone on the internet.

Deep Dive: The New Way to Create Public S3 Buckets

My final, successful main.tf file demonstrates the modern, four-part process required by recent AWS provider versions.

  1. aws_s3_bucket: First, you create the bucket itself, with no permission settings. This is just the container.
  2. aws_s3_bucket_public_access_block: This is the master switch. You must create this resource and set all four arguments (block_public_acls, etc.) to false. This tells AWS, "I know what I'm doing; I intend to make this bucket public."
  3. aws_s3_bucket_ownership_controls: This is another new requirement. It controls how ownership of uploaded objects is handled. BucketOwnerPreferred is the standard setting required to allow ACLs to function.
  4. aws_s3_bucket_acl: Only after the first three resources are in place can you apply the public-read ACL. The depends_on block is a crucial addition that explicitly tells Terraform the correct order of operations, preventing race conditions during creation.

Common Pitfalls

- Using Only the acl Argument: As I discovered, on modern provider versions, this is no longer sufficient and will result in a private bucket, even if Terraform doesn't return an error. - Forgetting s3_use_path_style: In a lab environment, forgetting this provider setting can lead to confusing DNS errors that have nothing to do with the resource configuration itself. - Bucket Name Already Exists: S3 bucket names must be globally unique across all AWS accounts. If I had tried to use a common name, my apply command would have failed with a BucketAlreadyExists error.

Exploring the Commands I Used

- terraform init: Prepared my working directory by downloading the aws provider plugin. - terraform plan: Showed me a "dry run" plan. My final, successful plan correctly showed 4 resources to be created. - terraform apply: Executed the plan and created the public S3 bucket after I confirmed with yes.