Using the AWS Terraform Provider with Impossible Cloud Storage

Impossible Cloud Storage exposes an S3-compatible API, so the standard hashicorp/aws Terraform provider can manage it once you point the provider at the Impossible Cloud endpoints and disable the AWS-specific validation steps that do not apply.   This lets you manage your Impossible Cloud resources - buckets, objects, and IAM users - as infrastructure as code: defined in version-controlled configuration, reviewed like application code, and applied repeatably instead of clicked together by hand in the console. The same approach lets you store Terraform's own state file on Impossible Cloud. 

The configuration in this article was verified against Terraform 1.15.8, AWS provider version 6.x, and the eu-west-3 region of Impossible Cloud. Endpoints for other regions follow the same pattern.

Prerequisites

• Terraform 1.15.8 or later 
• An Impossible Cloud Storage access key and secret key. Create these in the Impossible Cloud Storage Console.
• The regional storage endpoint of the IC region you are planning to use. This article uses https://eu-west-3.storage.impossibleapi.net.

Configure the AWS provider for Impossible Cloud

The AWS provider assumes it is talking to Amazon Web Services, so it performs several checks and lookups that do not apply to Impossible Cloud. Set the three Impossible Cloud endpoints (s3, iam, and sts) and the skip_* flags below. Impossible Cloud supports both path-style and virtual-hosted-style addressing; the examples in this article use path-style by setting s3_use_path_style to true. Including the iam and sts endpoints from the start means the same provider block also works for managing IAM users, covered later in this article.

terraform {
   required_version = ">= 1.10.0"
 
   required_providers {
     aws = {
       source  = "hashicorp/aws"
       version = "~> 6.0"
     }
   }
 }
 
 provider "aws" {
   region     = "eu-west-3"
   access_key = var.ic_access_key
   secret_key = var.ic_secret_key
 
   s3_use_path_style           = true
   skip_credentials_validation = true
   skip_region_validation      = true
   skip_requesting_account_id  = true
   skip_metadata_api_check     = true
 
   endpoints {
     s3  = "https://eu-west-3.storage.impossibleapi.net"
     iam = "https://iam.impossibleapi.net"
     sts = "https://sts.impossibleapi.net"
   }
 }

Keep your access key and secret key out of the configuration files. Supply them through a terraform.tfvars file that you do not commit, or through the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

Run terraform init to download the provider, then terraform plan to confirm Terraform can reach the endpoint and authenticate before you create anything.

Create a bucket

Define an aws_s3_bucket resource. The bucket name must be globally unique.

resource "aws_s3_bucket" "example" {
   bucket = "my-terraform-bucket"
 
   tags = {
     managed_by = "terraform"
   }
 }

Apply the configuration with terraform apply. Bucket create, read, tag update, and destroy all work through Terraform against Impossible Cloud Storage.

Upload an object

Use the aws_s3_object resource to upload content into a bucket. The content_type you set is preserved.

resource "aws_s3_object" "hello" {
   bucket       = aws_s3_bucket.example.id
   key          = "hello.txt"
   content      = "Hello from Terraform"
   content_type = "text/plain"
 }

Manage IAM users and access keys

Because the provider block above already includes the iam and sts endpoints, you can manage IAM users with Terraform directly. Without those endpoints, IAM and STS calls are sent to real AWS and fail with InvalidClientTokenId.

One difference from AWS applies: an Impossible Cloud IAM username must be a valid email address. Using any other format returns ValidationError: Username must be a valid email address.

resource "aws_iam_user" "example" {
   name = "terraform-user@example.com"
 }

Create, read, and delete of IAM users all work through Terraform once the endpoints are set.

Use Impossible Cloud as a Terraform remote state backend

You can store the Terraform state file on Impossible Cloud Storage using the s3 backend. The backend block cannot read variables, so the endpoint and flags are set inline, and credentials are supplied through the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. The state bucket must already exist.

terraform {
   backend "s3" {
     bucket = "my-terraform-state"
     key    = "project/terraform.tfstate"
     region = "eu-west-3"
 
     endpoints = {
       s3 = "https://eu-west-3.storage.impossibleapi.net"
     }
 
     use_path_style              = true
     skip_credentials_validation = true
     skip_region_validation      = true
     skip_requesting_account_id  = true
     skip_metadata_api_check     = true
 
     use_lockfile = true
   }
 }

The use_lockfile option enables native S3 state locking. This is the method to use for state locking; the older DynamoDB-based locking is not available on Impossible Cloud. Run terraform init to configure the backend, then terraform apply. Subsequent runs read the state back from the bucket.

Known differences and limitations

• Supported services. The AWS provider is a general-purpose provider covering hundreds of AWS services, but Impossible Cloud implements only the S3 (storage) and IAM APIs - and only the operations Impossible Cloud has implemented within them. 
• Required provider flags. The four skip_* flags shown above are required. Without them, the provider attempts AWS-specific validation and lookups that fail against Impossible Cloud.
• IAM usernames must be email addresses. This is an Impossible Cloud requirement and differs from AWS, where usernames are free-form.
• Cosmetic AWS-style attributes. After creating a bucket, read-only attributes such as arn, bucket_domain_name, hosted_zone_id, and bucket_regional_domain_name display AWS-style values (for example arn:aws:s3::: and .s3.amazonaws.com). The provider synthesizes these locally; they do not point to AWS and do not affect how the bucket works on Impossible Cloud.