How to Speed Up Ansible Playbooks Over 600%

Running Ansible playbooks against many hosts can be painfully slow, especially if they are in a region far away from you. I’m always looking for ways to speed up my playbooks and I’ve been using SSH pipelining in Ansible for quite some time. I feel like pipelining has improved the speed of my playbooks, but I’ve never actually tested it.

Recently a tool called Mitogen crossed my path and it claims to give a 1.25x - 7x speedup in Ansible – a bold claim I was immediately skeptical of. Could Mitogen really give a 7x speed increase, and is it any faster than good ol’ pipelining? This was a perfect opportunity to put Mitogen’s claims to the test and finally see how big an improvement pipelining actually gives.

- 0:51 -- Enabling Pipelining & Mitogen
- 2:49 -- Benchmarking All Configurations

MORE INFO

SETUP/EDITOR/IDE

Thanks for watching and hope it's entertaining/useful for you! Hit me up in the comments if you have any questions or feedback 💪.

Related & Channel

Our Site & Socials

[binaryfire]

Great video, thanks! Is there a benefit to using both "pipelining = true" and "strategy = mitogen_linear" at the same time?

[amizednix]

It's 2021 and this is still so useful and helpful! Great videos.

[dhir_AND]

Exactly what I was looking for. Well demonstrated !!

[lvlupdev]

Hi Percy, thank you very much, you solved a 10-hours problem in 7 minutes.

[babbab]

Awesome video! Voice, presentation and comparison are perfect, thank you very much!

[jayeshdhale]

Percy This is one of awesome video series of ansible I ever come across, Awesome

[toptechskills]

Thanks for watching everyone, I hope you found the video useful!




Please comment if you have any questions or feedback.

[tjwatts]

Amazing. I'm.going to test mitogen at work

[alexevinson]

Thanks for sharing really good stuff, Percy, your tutorials are extremely useful and very well done! I wish you to grow your channel up to 200k subscribers at least, just because you deserve :)
By the way, Ansible playbooks can run even faster with asynchronous mode in tasks (especially multiple asynchronous tasks running concurrently), highest number of forks set, and with facts and connection caching enabled, but the last option is for the huge number of hosts to be served at once. Give it a try!

[compumatterllc]

Ecstatic I stumbled upon this post. I have a very wide variety of tasks being run including downloads, software installs, apache and nginx server setups and much more. So it's a lot more complex than just a few text-oriented tasks. My results were not where near as impressive: Only a 40 second improvement.

With pipelining turned on in both cases, it took my sizeable playbook down from:
* Playbook run took 0 days, 0 hours, 10 minutes, 0 seconds TO
* Playbook run took 0 days, 0 hours, 9 minutes, 20 seconds

Do you know of a way to debug / troubleshoot this? Wondering if Mitogen is actually working. Followed your instructions to a 'T'

[enneydee]

Hi Percy, have you tried using "strategy = mitogen_free"? Is there a great difference with the default config "strategy = mitogen_linear"? Please provide insights. Thank you!

[veejeyz]

Hi @Percy.. The contents you teach are extremely helpful and I'd love if you could make one to better understand the deploy_helper module in ansible? Thanks again!!!!

[ramodhyarathnayake]

Awesome Teaching. Great Video, Please do more videos about Ansible. Thanks

[printjunkee]

Hi Percy, thanks for sharing this. Could you please tell me if can use both pipelining and mitogen plugin together, i.e. in the same configuration file. Also, would it better to just use mitogen plugin or both pipelining and mitogen together to speed up the playbook?

[dialaraBilbob]

hi, does it matter whichpath you extract the tar file? I extracted and move it to /etc/ansible directory, when I change ansible.cfg strategy=mitogen_linear, and run playbook it gives error.

[sidweb.]

Great video Percy! May I know which tool are you using to parallelly writing + executing the yml in single window ??

[sandeeputb]

Thanks for your video. Can u tell me the ide name u are using for yml.

[mikecmw]

Hi, do you have any ansible videos showing how to use the cron module? Need help trying to get this to work.

[infantusgodfrey]

My controller is centos7 and when connecting with ubuntu16 it shows the following issue as ubuntu16 has

[narasimhangopinath]

Thanks a lot bro.. Mitogen works cool 👍