Docker bridge network, HTTP calls between containers VERY slow (after docker upgrade)

Question

Server Features:

os: Ubuntu 14.04 docker: 1.10.2 docker-compose: 1.6.0 

More recently, I upgraded from 1.9 to 1.10 and added docker-compose (but don't use composition yet). The problem with slowness did not occur before the update.

Docker is also configured with my DNS-IP and proxy, for example, in '/ etc / default / docker'

 DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 --dns 138.XX.XX.X" export http_proxy="http://proxy.myproxy.com:8888/" 

(my ip is fully registered there, just using X for the question)

I have two containers (container_a, container_b) that run HTTP servers (Node.js), both containers work on the bridge network (--net = mynetwork), which I created through:

 docker network create mynetwork 

Two containers make HTTP calls to each other, using the name container_name as the "host" for HTTP calls, for example:

 container_b:3000/someurl 

These calls between two containers over the docker bridge network take a very long time (~ 5 seconds). These calls usually take up to 100 ms.

When I change the network from -net = mynetwork on these containers and instead run them as -net = host, and also changing my HTTP calls to use "localhost" as the host instead of the container name and displaying their ports via the -p flag ... Calls are made at the expected time <100ms.

It seems like the docker bridge network has been making my calls between containers for a very long time.

Any ideas on where I can look to diagnose / fix this problem?

Answer 1

This problem arose as a result of a change in the internal DNS released as part of docker 1.10.

Further information can be found here: https://github.com/docker/docker/issues/20661

I turned on debug mode on the daemon and looked at the log when I made requests. At first I could try "8.8.8.8" before moving on to "8.8.4.4", and then finally proceeding to the DNS-IP, which I added for my host and resolved. I assume that my corporate proxy causes these first two requests (8.8 ..) to hang and eventually timeout, as a result of which slowness is allowed with the correct IP address, which was the third on the list.

My solution was to reorder the DNS in the / etc / default / docker file to have my internal IP address first.

 DOCKER_OPTS="--dns 138.XX.XX.X --dns 8.8.8.8 --dns 8.8.4.4 " 

This seems to fix our problem as it allows HTTP requests based on the name of the container between the containers from the first IP address of this host.