How to transfer data volumes from only one host to another?

Question

As described in the Docker documentation on Working with Volumes , there is a concept of so-called data-only containers that provide a volume that can be installed in several in other containers, regardless of whether the container really works for data-only.

Basically, that sounds amazing. But there is one thing that I do not understand.

These volumes (which do not explicitly display the folder on the host for mobility reasons, as indicated in the documentation) are created and managed by Docker in some internal folder on the host ( /var/docker/volumes/… ).

Suppose I use such a volume, and then I need to transfer it from one host to another - how do I get the port of the volume? AFAICS has a unique identifier - can I just go and copy the volume and its data container only to the new host? How do I know which files to copy? Or is there some support built into Docker that I haven't found yet?

Answer 1

The official answer is now available here:

Directory Sharing Using Volumes

In the "Backing up, restoring, or migrating volumes" section, you have:

BACKUP:

 sudo docker run --rm --volumes-from DATA -v $(pwd):/backup busybox tar cvf /backup/backup.tar /data 

• --rm : remove the container when it exits
• --volumes-from DATA : attach to volumes shared by the DATA container
• -v $(pwd):/backup : bind the current directory to the container; to write the tar file to
• busybox : small simplified image - useful for quick maintenance
• tar cvf /backup/backup.tar /data : creates an uncompressed tar file of all files in the / data directory li>

RESTORE:

 # create a new data container $ sudo docker create -v /data --name DATA2 busybox true # untar the backup files into the new container᾿s data volume $ sudo docker run --rm --volumes-from DATA2 -v $(pwd):/backup busybox tar xvf /backup/backup.tar data/ data/sven.txt # compare to the original container $ sudo docker run --rm --volumes-from DATA -v `pwd`:/backup busybox ls /data sven.txt 

Answer 2

You can export the volume to tar and transfer it to another computer. And import the data from tar on the second machine. It does not depend on the details of the implementation of volumes.

 # you can list shared directories of the data container docker inspect <data container> | grep "/vfs/dir/" # you can export data container directory to tgz docker run --cidfile=id.tmp --volumes-from <data container> ubuntu tar -cO <volume path> | gzip -c > volume.tgz # clean up: remove exited container used for export and temporary file docker rm `cat id.tmp` && rm -f id.tmp 

Answer 3

Extending the official answer from Docker docs and the top answer here , you can have the following aliases in your .bashrc or .zshrc

 # backup files from a docker volume into /tmp/backup.tar.gz function docker-volume-backup-compressed() { docker run --rm -v /tmp:/backup --volumes-from "$1" debian:jessie tar -czvf /backup/backup.tar.gz "${@:2}" } # restore files from /tmp/backup.tar.gz into a docker volume function docker-volume-restore-compressed() { docker run --rm -v /tmp:/backup --volumes-from "$1" debian:jessie tar -xzvf /backup/backup.tar.gz "${@:2}" echo "Double checking files..." docker run --rm -v /tmp:/backup --volumes-from "$1" debian:jessie ls -lh "${@:2}" } # backup files from a docker volume into /tmp/backup.tar function docker-volume-backup() { docker run --rm -v /tmp:/backup --volumes-from "$1" busybox tar -cvf /backup/backup.tar "${@:2}" } # restore files from /tmp/backup.tar into a docker volume function docker-volume-restore() { docker run --rm -v /tmp:/backup --volumes-from "$1" busybox tar -xvf /backup/backup.tar "${@:2}" echo "Double checking files..." docker run --rm -v /tmp:/backup --volumes-from "$1" busybox ls -lh "${@:2}" } 

Note that the backup is saved in /tmp , so you can move the backup file saved there between the docker hosts.

There are also two pairs of backup / restore aliases. One uses compression and debian: jessie and others without compression, but with busybox. Thank you for using compression if the backup files are large.

Answer 4

I will add another recent tool from IBM, which is actually made to transfer volumes from one container to another. This is an ongoing project. Thus, you may find another version with additional features in the future.

Cargo was designed to transfer containers from one host to another host along with their data with minimal downtime. Cargo uses the ability to combine data from a federated file system to create a single view of the data (mainly the root file system) on the source and target nodes. This allows Cargo to start the container almost immediately (in milliseconds) on the target host, since data from the source root file system is copied to the target hosts on demand (using copy-on-write (COW) ) or lazily in the background (using rsync) .

The important points are: - the centralized server handles the migration process

Link to the project is given here:

 https://github.com/nadgowdas/cargo