Terraform Remote State Backend using S3 and DynamoDB

Terraform can store state remotely in S3 and lock that state with DynamoDB. In this Terraform tutorial video, I am going to explain how you can configure remote state on S3 backend and enable state locking with Dynamo DB table.
#terraform
#s3bucket
#remotestate #devops

Timestamps -
1. Intro 0:00 - 0:15
2. Benefits of Remote State 0:16 - 0:43
3. Create S3 bucket for storing State file 01:07 - 06:00
4. Create a DynamoDB table for state locking 06:00 - 08:00
5. Apply changes to create s3 and dynamo DB 08:10 - 09:05
6. Add the backend in terraform block 09:05 - 10:45
7. Run the commands and migrate the state 10:46 - 12:40
8. Test the s3 state file 12:41- 14:30
9. Remote state to local state 14:35

The best way to do this is by running Terraform in a remote environment with shared access to state. Remote state solves those challenges. Remote state is simply storing that state file remotely, rather than on your local filesystem. With a single state file stored remotely, teams can ensure they always have the most up to date state file.

Terraform Remote State Storage Options
Terraform supports storing state in

Amazon S3
Azure Blob Storage
Google Cloud Storage
Terraform Cloud
HashiCorp Consul

With Remote state configuration all the state is stored in backends like s3 which is considered as a good practice.
Benefits of using Terraform Remote State
Shared Storage: Remote state (Backend) allow each of your team members to access same Terraform state files to manage infrastructure.
Locking: With fully-featured remote backends, Terraform can lock the state file while changes are being made. This ensures all changes are captured, even if concurrent changes are being attempted against the same state.
Without locking, if two team members are running Terraform at the same time, you may run into race conditions as multiple Terraform processes make concurrent updates to the state files, leading to conflicts, data loss, and state file corruption.
Versioning: Some backends support versioning. This maintains versions of your Terraform state files allowing you to download an old version if needed.
Encryption: Many backends support encryption of the state file both in transit and at rest.
Security: A local state file save the content in plain text. It is very common to have secrets or sensitive data in the state, so local state files are insecure

Remote operations: Some backends allow to manage operations remotely (Terraform plan and apply execution). You don’t need to use terraform on your system to apply the changes. You could either trigger it from a Web UI, API call or CLI tool.
Less Manual Errors: Using a local state file as a shared storage, manually sync the changes could cause someone forget to sync the state file. Remote state will always sync the state automatically whenever it change.

Also Checkout my other videos

@Cloud Champ is a youtube channel I started to record my learning and now is helping a lot of cloud and devops engineer. If you need any help you can always drop a comment or reach out to me.

Tags:
terraform remote state
terraform backend s3
terraform state backend s3 bucket
terraform remote state s3 example
terraform state in s3 bucket
terraform remote state
terraform remote state s3 example
terraform state in s3 bucket
terraform remote state azure
terraform state locking
terraform state file s3
terraform remote state file
terraform backend s3 example
terraform state backend s3
terraform migrate state to s3
terraform state s3 bucket
terraform state s3 dynamodb
terraform tfstate s3
terraform tutorial
techworld with nana terraform
terraform aws
terraform state management s3
state remote s3
terraform state backend s3
terraform migrate state to s3
terraform state s3 bucket
terraform state s3 dynamodb
terraform tfstate s3

[kyronne]

You are a life saver, really well explained was completing a lab could set up the EC2 instances and use variables fine. Configuring the S3 backend with an updated version of Terraform was too hard to do alone. Thank you 🙏

[charleslassiter]

best video on sf management many thanks for sharing

[kwabenasteve]

Thank you for this. saved me hours of tutorials

[practicaldeen]

Thanks CC...much appreciated. Great explanation.

[adedayoademefun]

Hi. Thanks for your wonderful videos. I currently have multiple projects and I’m using one S3 bucket for my state backend. These multiple project state files are separated by paths in the S3 bucket. Now, I want to implement DynamoDB state file lock. Since I have multiple projects, can I use only one DynamoDB table, like I used one S3 bucket for my state file backend? If the answer is yes, how does DynamoDB separate and isolate the locks for each project in this table? Or do I have to create multiple DynamoDB table for all my projects? Your guidance and thoughts on this would be appreciated.

[viral_shortsviralshorts]

You are lifesaver!!
Thanks for this project 😊

[kavyas]

hi can you explaination about given path. any condition in giving the path

[rakshirakshitha]

this video is very helpful. I have one doubt "AES256" is it a name or what? if it is a name can we give whatever as name right

[narasimhakudva]

Hi

I Have aready created a Database and other infra without backend.tf . If I create now will there be a problem

[RolandTutorials]

Thank you cery much, this video is very helpful

[ksubrahmanyam]

Tell me if the state file is deleted, what we need to do to get back state file.

[PriyaDharshini-sb]

Hi, thanks for the tutorial. after moving state file to remote backend, i keep getting the following error from the next init or plan. Any idea how to resolve this? Error: Error acquiring the state lock
│
│ Error message: operation error DynamoDB: PutItem, https response error
│ StatusCode: 400, RequestID:
│ │ The conditional request failed

[opeyemiojo]

Thanks for the tutorial, I was able to follow the steps and the terraform.tfstate file was on s3 bucket, and it the same time i have the terraform.tfstate file locallly on my computer inside the .terraform file, and if I tried deleting locally, it will download it again after running terraform init, is this normal? Can you explain if I am doing anything wrong, thanks

[VijethKalavarkar]

I bro I can't be able to push the code to GitHub coz my file exceed 100 mb .. so what to do I have tried installing lfs also .. in my centos AWS instance

[sribalaje]

Hi, Thanks for the video. I followed the same steps and ran but it doesn't show "Acquiring state lock" when i run "terraform plan" or terraform apply" command. Please help on this.

[martinthong]

Hey Champ, how do I store multiple state files in s3?

[Emil_Benny]

Hello,

How to make use of this state file while working in a team,
do you have to pull the remote s3 state file to local ?
ie, if any other user in the team wanted to know what resources have already been created, how do you make use of this remote s3.

[maheshpansare]

I have try to done that but in my side o am getting so many errors like missing lock items, operation error DynamoDB: PutItein and not able to exesting s3 bucket but i have to init the script

[nelsondario]

how to lock the tf.state in AWS S3 so that other cant write to it?

[pawansolanki]

Hi I am getting error for backend

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Error refreshing state: AccessDenied: Access Denied
status code: 403, request id: 13YNJNZ963P0G2PC, host id:

Any clue