Docker: Swarm work nodes do not detect locally built image

Question

Maybe I missed something, but I made a local docker image. I have 3 node. Two workers and one manager. I use labels as a limitation. When I start a service for one of the workers through a restriction, it works fine if this image is publicly available.

That is, if I do this:

docker service create --name redis --network my-network --constraint node.labels.myconstraint==true redis:3.0.7-alpine 

Then the redis service is sent to one of the work nodes and is fully functional. Similarly, if I run my locally built image WITHOUT restriction, since my manager is also a worker, he receives a manager assignment and works fine. However, when I add a constraint that it fails on the working node, from docker service ps 2l30ib72y65h I see:

 ... Shutdown Rejected 14 seconds ago "No such image: my-customized-image" 

Is there a way to get workers to access local images in the swarm node manager? Does it use a specific port that might not be open? If not, what should I do - start the local repository?

Answer 1

The node manager does not distribute local images from itself. You need to deploy the registry server (or the user hub.docker.com). The efforts required for this are not very important:

 # first create a user, updating $user for your environment: if [ ! -d "auth" ]; then mkdir -p auth fi chmod 666 auth/htpasswd docker run --rm -it \ -v `pwd`/auth:/auth \ --entrypoint htpasswd registry:2 -B /auth/htpasswd $user chmod 444 auth/htpasswd # then spin up the registry service listening on port 5000 docker run -d -p 5000:5000 --restart=always --name registry \ -v `pwd`/auth/htpasswd:/auth/htpasswd:ro \ -v `pwd`/registry:/var/lib/registry \ -e "REGISTRY_AUTH=htpasswd" \ -e "REGISTRY_AUTH_HTPASSWD_REALM=Local Registry" \ -e "REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd" \ -e "REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry" \ registry:2 # then push your image docker login localhost:5000 docker tag my-customized-image localhost:5000/my-customized-image docker push localhost:5000/my-customized-image # then spin up the service with the new image name # replace registryhost with ip/hostname of your registry Docker host docker service create --name custom --network my-network \ --constraint node.labels.myconstraint==true --with-registry-auth \ registryhost:5000/my-customized-image 

Answer 2

For me it worked in stages, which is unsafe:

 # Start your registry $ docker run -d -p 5000:5000 --name registry registry:2 # Tag the image so that it points to your registry $ docker tag my_existing_image localhost:5000/myfirstimage # Push it to local registry/repo $ docker push localhost:5000/myfirstimage # For verification you can use this command: $ curl -X GET http://localhost:5000/v2/_catalog # It will print out all images on repo. # On private registry machine add additional parameters to enable insecure repo: ExecStart=/usr/bin/dockerd --insecure-registry IP_OF_CURRENT_MACHINE:5000 # Flush changes and restart Docker: $ systemctl daemon-reload $ systemctl restart docker.service # On client machine we should say docker that this private repo is insecure, so create or modifile the file '/etc/docker/daemon.json': { "insecure-registries":["hostname:5000"] } # Restart docker: $ systemctl restart docker.service # On swarm mode, you need to point to that registry, so use host name instead, for example: hostname:5000/myfirstimage 

Answer 3

Images must be uploaded to the local cache on each node . The reason is that if you save all your images on only one node and that the node goes down, the swarm will not be able to create new tasks (containers) on other nodes.

I personally just pull out a copy of all the images on each node before starting the services. This can be done in a bash script or Makefile (e.g. below)

 pull: @for node in $$NODE_LIST; do OPTS=$$(docker-machine config $$node) set -x docker $$OPTS pull postgres:9.5.2 docker $$OPTS pull elasticsearch:2.3.3 docker $$OPTS pull schickling/beanstalkd docker $$OPTS pull gliderlabs/logspout etc ... set +x done