Infrastructure As Code Explained

Infrastructure as code allows us to version-control the infrastructure that our software systems depend upon. This is a key idea for cloud-based systems, but it is equally applicable to other types of system too. This approach allows us to better achieve the repeatability and reliability that Continuous Delivery demands. There are key ideas here, like configuration-synchronisation and immutable infrastructure that allow us to organise our work and to achieve that repeatability in software development, software testing and release. Infrastructure as code is a key concept in Continuous Delivery and DevOps.

There are many effective infrastructure as code tools that allow us to take better control of these aspects of our systems like Chef, Puppet, Ansible & Terraform, but this topic is less about the tools themselves and more about the concepts.

In this episode, Dave Farley describes the three common patterns for managing change to our infrastructure, and then describes a series of important characteristics of Infrastructure as code and offers a series of tips to help you to implement it.

Continuous Delivery requires us to "control the variables” in our approach to software engineering and software craftsmanship, Infrastructure as code is a key part of the exercise of that control.

If you have ever said “it works on my machine” to anyone, you need Infrastructure as code! This episode is really “Infrastructure as code explained” and so should give you a good start on your journey.

-------------------------------------------------------------------------

📚 BOOKS:

📖 Dave’s NEW BOOK "Modern Software Engineering" is now available on
In this book, Dave brings together his ideas and proven techniques to describe a durable, coherent and foundational approach to effective software development, for programmers, managers and technical leads, at all levels of experience.

📖 "Continuous Delivery Pipelines" by Dave Farley

📖 The original award-winning “Continuous Delivery" book by Dave Farley and Jez Humble

-------------------------------------------------------------------------

-------------------------------------------------------------------------

[biplab]

Great video to understand the concept, thank you!

[JukkaPena]

I really like the concept of writing tests for the IaC. Recently I have been introducing TDD approach to the way I develop infrastructure or deployment pipelines. I have been amazed how it makes the code design a lot better and speeds up the development work when I don’t have to deploy the infrastructure that often. I also feel a lot more confident when I perform the deployment after developing the IaC.

I have been using AWS CDK as IaC solution. While watching this video I realized that, at least with default configuration, it’s more about configuration synchronization than immutable infrastructure approach because it allows configuration drift since it doesn’t override manually done changes.

Thanks for these videos and your books - they have had a big positive impact on my daily work.

[xxxxxxxxxxa]

I found that starting with automation from scratch is somehow hard to figure out, and it's good to start with the opposite approach, the anti-pattern of manual configuration. Then, after having ten or more manually provided cases, it's much easier to see repeatable parts, and some patterns, and thus have a roadmap on how to automate things. It gives you a good insight into what is highly repeatable and common, and what is very customized. Knowing that you can build better automation.
Also postponing decisions can be a good approach in software architecture. As an example, I can give a pre-built server template that was prepared by us to be used by a 3rd-party company when providing a new server suited for our needs. After some time 3rd party company refused to use that template, they forgot to keep that in a procedure. That caused us to have to add and use some shell toolkits providing needed customization in different ways. That would never happen with a more lazy approach, less trust in 3rd-parties and greater observability before starting automation.

[nerrierr]

Very interesting and I agree with everything you said, but a small question though: isn't it that at one point/level in the "pipeline" things need to be configured and administered manually? (ex: your CI/CD tool, network stuff, the actual physical machine, your VM/Container orchestrator and runtime, etc...)
How can you ensure that "everything" is versioned and automatically provisioned (at least without significant effort in development and maintenance)? You mentioned Docker but even the Docker environment needs to be preconfigured, so you end up with a CI/CD pipeline to provision your CI/CD pipeline, and so on and so forth.

[ContinuousDelivery]

What Infrastructure as Code tools are you using? I'm interested to see what is most popular amongst devs 👇

[andreaszetterstrom]

Regarding the immutable infrastructure; how does it work with data the system stores if you blow out the entire system every time you make a modification? Like what happens to databases, caches and so on, stuff that isn't stored in our version system?

[YvesHanoulle]

When I review this video, I had the idea for another video: a video about the test you wrote for testing your CD pipelines.

[gronkymug]

Immutable infrastructure seems to work better for "stateless" infrastructure and not "statefull" like databases with data in them. It may also not be so easy to work with for statefull services. I'm sure it is all doable though. I would really like to see more "magic" solutions provided by cloud providers in the future. Firebase like ones. Possibly some new AI tools can decide the best data structure for our data and store it in the way it can be retrieved the fastest and our services could be deployed to various places (lambda, container, akka, etc.) depending on they type of service we write. I guess (more like hope) that this will happen in the next 10 to 15 years.

[DodaGarcia]

That all makes a lot of sense but it also sometimes feels counterproductive how much IaaS became just as complex as managing infrastructure from scratch used to be

Obviously it comes with the huge benefit (and danger) of being ad hoc and infinitely scalable, but it's interesting to me how cloud solutions got so intricate in their own right that not only do you end up needing a specialist anyway but also they then gave birth to simplified versions of themselves such as Firebase and Amplify

[gonzalowaszczuk]

For the ad hoc approach, the pro is pretty clear: It's easy, fast, and cheap, at least insidiously so.

Like you mention, doing all of these things correctly is hard, so the ad hoc approaches are just the easy alternatives that work good enough now.

Things like immutable infrastructure depend on everything downstream being immutable. If a single piece of dependency can't be made immutable, your whole pipeline is no longer immutable. In legacy systems this can be hard to manage, you basically get the benefit of infrastructure as code only when everything is immutable/versioned/etc, and not even a single moment before.

Strategies to successfully migrate from a non-reproducible infrastructure to a reproducible infrastructure may be more important than creating IaC from scratch, just like the skill of refactoring is arguably more important than the skill of creating good code from scratch.