Scaling Docker #3 - Docker Service Registration with Etcd & Flannel

2 of the biggest problems to solve with Docker infrastructure are service registration & service discovery. In this video, we'll cover the more complicated piece, which is service registration.

Every Docker container needs to register itself as a web service that can be consumed or load balanced by other containers.

- Cluster computing - CoreOS is our solution here. CoreOS allows us to make an expandable cluster of computers so we can add more computers should our application need more power to scale. CoreOS also allows us to run services on any machine in the cluster and they can all communicate with each other.

- Scheduling: we shouldn't manually choose which app runs on which machine - this idea doesn't work at scale. Fleet will be our scheduler that decides where our apps/services run. If a CoreOS machine goes down, fleet will reschedule any services that it was running onto a new machine.

- Service Registration
Since we never know what ip address or port a service will be running on, we need to register that service so other services can find it and communicate with it. We'll use Etcd and Flannel for this

- Service Discovery:
Now that services have registered, we can discover them and load balance between them with a generic docker nginx container

- Public DNS: Once the app is working, we need a fixed way for the outside world to access it. We'll use Nginx and Confd for this. Confd will watch Etcd for service registration and when services come/go, it will build a new configuration file for Nginx and reload Nginx.

-~-~~-~~~-~~-~-
Also watch: "Responsive Design Tutorial - Tips for making web sites look great on any device"
-~-~~-~~~-~~-~-

[BogdanKulbida]

To avoid hard code in container port, which is 3000 in this example, you can use the flowing docker inspect command to get the Host port:
For this particular example:

`docker inspect -f '{{(index (index .NetworkSettings.Ports "80/tcp") 0).HostPort}}' someapp%i`

[aravindreddy]

awesome videos. Thanks for taking time and sharing these videos

[BradleyWeston]

How would the client get this data They would have to be aware of someapp{1..4}?
Also how do you handle dynamic ports? If I have 10 containers serving node applications (node sucks unless clustered) on 1 host how would you go about that?

[leodavid]

Very good tutorial, thank you. I would have one question. At some point, you said that "it is not a good practice for containers to register themselfs". Why is not considered a good practise ?
Why not update keys in etcd by running a curl script, instead of creating a sidekick discovery service ?

[noeskasoftware]

now i cannot seem the discovery service to run.if i copy out the execstart command and run it from the commandline the %i does not seem to get replace by the number and i get json invalid url parse error

[noeskasoftware]

Great tutorial, only i dont seem to be able to get it running. I can only ping docker hosts that are on the same machine. I think i opened the correct udp port in the firewall but not sure.