Terraform with Multiple State Files

About The Cloud Coach:
Michael Crilly, aka "The Cloud Coach", is a leading expert in DevOps and automation. The Coach has automated everything from Cisco hardware to AWS resources using various tools like Python, Ruby, Go, Terraform, Packer, CloudFormation, CI/CD, JenkinsCI, GitOps, and much more. The Coach also loves to teach everyone and anyone about the joy of automating complex Cloud resources to deliver a fast, better infrastructure that's cheaper to operate and more stable to manage.

Connect with The Cloud Coach:

Checkout my FREE courses!

Checkout my PREMIUM courses!

[FredericKouidri]

Thank you Michael, I have been searching for a long time the way to pass values from one project to another ! (data + terraform_remote_state)

[giftfelix]

Great stuff man, I love the separation of concern. Thanks

[azy]

Great video! Definitely a big fan of Terragrunt when it comes to the multi statefile setup, definitely cleaner!

[ScrolltheNature]

Loved it !! You clear my doubt as i am implementing terraform for an organization. Thanks for help.

[WebDevCody]

On our team we were looking into ways we could speed up our deploys. This might be useful. I’d need to see the speed benefits gained vs development time to refactor to split up some resources into separate states. We also do a similar thing you mentioned and put all of our policies and roles into a different state that can only be ran manually by a select few people (instead of allowing ci cd to modify them). Good info!

[thecloudera]

Brilliant one!! thanks for the video Michael ...my neurons are making new connections ... how does a module fit-in in this scenario ..whew!! thanks for sharing.

[leriksenbendigo]

Whilst this does have some use cases, I've typically found this to be the much more work for the devops eng to deal with. Terraform is very good at discovering how a change affects multiple resources in the state file. Separating out to separate states doesnt let tf discover/resolve those. terraform_remote_state can help a little. I think splitting out resources to separate state files can be limited to resources common across several environments/projects is ok - e.g. centralised log solution. One other possible case is per-developer environments. One state file per dev allows for developers to change their own env without affecting other devs, allowing new development to be prototyped, and a simple "terraform plan" tells them how to get back to the standard setup. Thanx for the content

[sharonsahadevan]

Great! really helpful and easy to understand. Keep up the awesome work man!

[hamidapremani]

Well-explained and concise, as always. Thanks!

[kristof]

Great video! It helped me to prepare for the Terraform certification.

[umka]

Awesome video with explanation! Thanks a lot!

[Organicnz]

As always clear. Thanks for sharing bro :)

[silopolis-yt]

Excellent video Michael! Thanks for sharing it.
One question: what if the remote state isn't up-to-date with the actual system? Using provider supplied data seems more reliable from this point of view. What are the tradeoffs of both solutions? Does a scenario using remote state for "static"/config data coming from TF, and provider for data about the live system, makes sense, and would it be reliable and maintainable?
On to next one of yours...
Cheers from France :)

[krishna]

Can you please do a video on how Terraform project should be created for multiple environments.

[PENFRIEN]

Great video Michael. :)
Does this separation of state files helps in locking state file with another admins? Maybe I missed it in the video. Thank you and you got a good flow in videos and great explenation. :)

[ankansikdar]

That's a great piece of info you just presented. Great job!

[barrymanilow]

The problem with this approach is you can’t terraform plan everything together before doing any terraform applys. Say you added a new resource to ‘networking’, outputted it and referenced it in ‘application’ - A terraform plan on ‘networking’ would complete but a terraform plan on ‘application’ would fail due to the outputs not being in the networking remote state file yet. This makes CI/CD harder as let’s say these changes were all part of the same commit, and you wanted to plan them all as part of a PR. It would fail due to the dependencies.

[t-xdrp]

If you wanted seperate env's, say, Test, Pre-Prod, Prod etc. would you use the same code with all the same resources in each of the env's and then just plan and apply those changes to the env of choice hence not breaking prod when trying to add a new resource?

[rezaebrahimi]

very good explanation. i get involved into your courses

[subzeroleaf]

Thank you for the great content! That was really clarifying