Get the latest download link programmatically

Question

I want to be able to download the latest version of software automatically using my bash script. Unfortunately, no site has the latest release link like github. In my case, I need to download the latest stable version of Nginx

I am currently using this http://nginx.org/download/nginx-1.4.7.tar.gz and then I compile from source

The problem is that I will need to check for updates from time to time and update the link.

Is there a way I can use to make my script automatically captured by the latest stable version of Nginx

Installing PS via yum is not an option

Yours faithfully

Answer 1

Update :
A solution that identifies and downloads the latest stable version through http://nginx.org/en/download.html ( http://nginx.org/download/ used in the original solution below does not distinguish between stable and major versions) - works with both Linux and OSX:

 # Determine the latest stable version download URL, assumed to be # the first `/download/nginx-*.tar.gz`-like link following the header # "Stable version". latestVer=$(curl -s 'http://nginx.org/en/download.html' | sed 's/</\'$'\n''</g' | sed -n '/>Stable version$/,$ p' | egrep -m1 -o '/download/nginx-.+\.tar\.gz') # Download. curl "http://nginx.org${latestVer}" > nginx-latest.tar.gz 

Note. It depends on the structure of the HTML page http://nginx.org/en/download.html , which is not the most reliable solution.

---

The original solution , which defines the latest version through http://nginx.org/download/ , where no distinction is made between stable and major versions:

On Linux, try:

  # Determine latest version: latestVer=$(curl 'http://nginx.org/download/' | grep -oP 'href="nginx-\K[0-9]+\.[0-9]+\.[0-9]+' | sort -t. -rn -k1,1 -k2,2 -k3,3 | head -1) # Download latest version: curl "http://nginx.org/download/nginx-${latestVer}.tar.gz" > nginx-latest.tar.gz 

It does not depend on the specific listing order http://nginx.org/download/ ; instead, version numbers are retrieved and sorted accordingly.

---

On OSX - where grep does not support -P and \K not available to remove the left side of the match, try:

 # Determine latest version: latestVer=$(curl 'http://nginx.org/download/' | egrep -o 'href="nginx-[0-9]+\.[0-9]+\.[0-9]+' | sed 's/^href="nginx-//' | sort -t. -rn -k1,1 -k2,2 -k3,3 | head -1) # Download latest version: curl "http://nginx.org/download/nginx-${latestVer}.tar.gz" > nginx-latest.tar.gz 

Answer 2

This will determine the latest stable version:

 $ lynx -dump http://nginx.org/en/download.html \ | awk '/Stable/{t=1}t&&/nginx-/{$0=$2;sub(/.+\]/,"");print;exit}' nginx-1.4.7 

Use this result to compile the correct download URL and use wget / curl to download.

This will determine the download URL of the latest version:

 $ lynx -dump http://nginx.org/download/ | awk '/nginx-.*\.zip$/{url=$2}END{print url}' http://nginx.org/download/nginx-1.5.9.zip 

It depends on how their web server sorts the directory list by date (which it is currently doing).

Answer 3

If using third-party tools is acceptable , here is another option using the multi-platform CLI xidel

 curl "http://nginx.org$( \ xidel 'http://nginx.org/en/download.html' \ -e '(//a[matches(@href, "/download/nginx-.+\.tar\.gz$")])[2]/@href' \ )" -O 

The command uses an XPath expression to retrieve the URL of interest; here the assumption is that the link 2 /download/nginx-*.tar.gz on the page is a stable version (the first representing the mainline version).

This is not the most reliable approach, but it is an inherent problem with crashes of web pages that were not intended for use in programs.

Answer 4

found it at https://drupal.stackexchange.com/a/23701

 wget http://lftp.yar.ru/ftp/$(wget -O- http://lftp.yar.ru/ftp/ | egrep -o 'lftp-[0-9\.]+.tar.gz' | sort -V | tail -1) 

might be a simpler solution

Answer 5

Getting the latest versions of nginx and naxsi does not require sorting, but does require jq .

 latestNginx=$(curl -s http://hg.nginx.org/nginx/tags | grep "^ *release-" | head -1 | cut -c 9-) latestNaxsi=$(curl -s https://api.github.com/repos/nbs-system/naxsi/releases | jq -r .[].tag_name | grep -v rc | head -1)