Scaling Docker #5 - Routing DNS to Docker images running on a CoreOS Cluster

Our last piece of the Docker puzzle - we need to dynamically direct DNS to our Docker load balancers! Once this is all up and running, it's so stinking fun to deploy and scale entirely new applications onto our infrastructure with just a few commands!

- Cluster computing - CoreOS is our solution here. CoreOS allows us to make an expandable cluster of computers so we can add more computers should our application need more power to scale. CoreOS also allows us to run services on any machine in the cluster and they can all communicate with each other.

- Scheduling: we shouldn't manually choose which app runs on which machine - this idea doesn't work at scale. Fleet will be our scheduler that decides where our apps/services run. If a CoreOS machine goes down, fleet will reschedule any services that it was running onto a new machine.

- Service Registration
Since we never know what ip address or port a service will be running on, we need to register that service so other services can find it and communicate with it. We'll use Etcd and Flannel for this

- Service Discovery:
Now that services have registered, we can discover them and load balance between them with a generic docker nginx container

- Public DNS: Once the app is working, we need a fixed way for the outside world to access it. We'll use Nginx and Confd for this. Confd will watch Etcd for service registration and when services come/go, it will build a new configuration file for Nginx and reload Nginx.

-~-~~-~~~-~~-~-
Also watch: "Responsive Design Tutorial - Tips for making web sites look great on any device"
-~-~~-~~~-~~-~-

[shrikantraut]

This was one great series. I don't really know when I would do the load balancing part but, I really liked how docker-nginx-coreOS co-ordinated with each other.

Thanks a lot.

[michalc]

This is the best video on the internet talking about microservices!

[Omnifarious]

I liked this series, but I'm someone who has to know how it all works from the ground up. So this is just a launching point for me to fully understand Docker.

All of this service creation and discovery stuff is also pretty interesting. And is also something I'm going to have to learn a lot about the exact details of. But it seems there are a whole bunch of different solutions to those problems and no one standout to learn.

[jepkofficial]

Doesn't the single-node nginx + etcd completely negate the whole "high-availability" architecture that we just set up over the past several videos? We just passed lack of fail-over up the chain. Turtles all the way down?

[Tdubs]

Such a great series. I watched all 8 videos in the play list twice now :). Thanks for being a great teacher!

[shawnchong]

Very nice intro stuff dude... this might be 3 yrs old, but I'm learning docker for my job. Will buy your stuff online.

[foxlance]

@LearnCode.academy it's about time you release the Kubernetes-cluster equivalent to this series! Pretty please!

[imtorrellasa]

Thank you very much for this series... I'll try to start testing this in AWS. Great series as always. Thanks!

[AndryanSTGKR]

Thanks for sharing this amazing video.

[PhillippOhlandt]

How do you shared files between the nodes? Let's say user uploads or other application files?

[marchenkoalexandr]

by the way, seems that you did not cover database and storage neither in this series nor o'reilly course, just wondering have you any plans to complete whole circle on that

[djufk]

amazing! thanks a lot! i learned a lot thanks to you. and i'm finally able to start playing with docker and CoreOS.

[abhiksd]

Thank you very much for creating such a great videos series, it has given very good understanding of Docker + CoreOS cluster integration. If you dont mind can you please share configuration files and Dockerfiles which you used in this video for my future reference.

Thanks Again for this greate video series.

[marchenkoalexandr]

Just wondering, after some time what is your opinion, have you done anything that way in production, will you recommend that approach for future projects or there is other more preferred ways? e.g. docker swarm or may be kubernetis or something else.

[MadGoat]

Another note... It seems that you're missing instruction on the "public-nginx-etcd" setup ...
- Does it have its own etcd discovery key or does it use the same key when you're setting it up ?
- How are you launching etcd on that server ?

[Czarmzy]

At what scale do you find the need to employ such techniques? cause obviously thats a lot of ceremony for a simple website :D

[boomtzi]

awesome video series, thanks a lot for sharing this knowledge :)
im gonna start testing stuff right now! :D

[kartikphilo]

hi, i set my container with hostname, i need to point that hostname on the browser. can u please tell me how can i do that?

[BenediktCWolters]

Why not DNS load balance between the nginx load balancers?

[SurendranSelvaraj]

I'm a beginner, I'm trying create a web page where if i click an image it would pop up another image in the BG with text / any content over it. Something similar to e-commerce websites' quick view option of a product. Learn Code, got any tutorials for it?
Please someone share it.