Docker code is not as updated as moby in the development repository on github. People have had problems for several years with this. I tried to manually use several patches that are not yet in the upstream, and none of them worked decently.
The gitHub repository for moby (a repository for developing dockers) contains the download-frozen-image-v2.sh script. This script uses bash, curl, and other things, such as JSON interpreters through the command line. It will receive a Docker token, and then load all the layers into a local directory. You can then use the “Docker download” to insert into your local Docker installation.
This is not good with a resume, though. There is a comment in the script that "curl -C" is not working. I tracked and fixed this problem. I made a modification that uses the ".headers" file for initial extraction, which always returned 302 while I watched, and then extracted the final use of curl (+ resumption support) to the layer tar file. It must also execute a loop for the calling function, which retrieves a valid token, which, unfortunately, only lasts about 30 minutes.
He will loop this process until he receives 416 that renewal is impossible, since his ranges have been met. It also checks the size by extracting the twist header. I managed to get all the images needed with this modified script. Docker has many more search-related levels and has remote control processes (the Docker client) that complicate management, and they believe that this problem only affects some people with poor connections.
I hope this script can help you just as it helped me:
Changes: fetch_blob uses a temporary file for the first connection. He then gets a 30x HTTP redirect from this. It tries to get the header from the final URL and checks to see if the local copy has the full file. Otherwise, the resume rolling operation begins. The calling function, which passes the valid token to it, has a loop surrounding the receipt of the token, and fetch_blob, which ensures that the complete file is received.
The only other option is the bandwidth limit variable, which can be set at the top or through the "BW: 10" command line parameter. I needed this so that my connection was viable for other operations.
Click here to modify the script.
In the future, it would be nice if the internal Docker client performed the resume properly. Increasing the amount of time to verify the token would help a lot.
Summary of change code:
#loop until FULL_FILE is set in fetch_blob.. this is for bad/slow connections while [ "$FULL_FILE" != "1" ];do local token="$(curl -fsSL "$authBase/token?service=$authService&scope=repository:$image:pull" | jq --raw-output '.token')" fetch_blob "$token" "$image" "$layerDigest" "$dir/$layerTar" --progress sleep 1 done
Another section from fetch_blob:
while :; do #if the file already exists.. we will be resuming.. if [ -f "$targetFile" ];then #getting current size of file we are resuming CUR='stat --printf="%s" $targetFile' #use curl to get headers to find content-length of the full file LEN='curl -I -fL "${curlArgs[@]}" "$blobRedirect"|grep content-length|cut -d" " -f2' #if we already have the entire file... lets stop curl from erroring with 416 if [ "$CUR" == "${LEN//[!0-9]/}" ]; then FULL_FILE=1 break fi fi HTTP_CODE='curl -w %{http_code} -C - --tr-encoding --compressed --progress-bar -fL "${curlArgs[@]}" "$blobRedirect" -o "$targetFile"' if [ "$HTTP_CODE" == "403" ]; then #token expired so the server stopped allowing us to resume, lets return without setting FULL_FILE and itll restart this func w new token FULL_FILE=0 break fi if [ "$HTTP_CODE" == "416" ]; then FULL_FILE=1 break fi sleep 1 done