Geek Answers Handbook

Download binary from Github using Java

I am trying to download this file ( http://github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar ) with the following method and it does not seem to work. I get an empty / damaged file.

String link = "http://github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar"; String fileName = "ChampionHelper-4.jar"; URL url = new URL(link); URLConnection c = url.openConnection(); c.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 1.2.30703)"); InputStream input; input = c.getInputStream(); byte[] buffer = new byte[4096]; int n = -1; OutputStream output = new FileOutputStream(new File(fileName)); while ((n = input.read(buffer)) != -1) { if (n > 0) { output.write(buffer, 0, n); } } output.close();

But I can successfully download the following file from my Dropbox ( http://dl.dropbox.com/u/13226123/ChampionHelper-4.jar ) with the same method.

So, somehow Gitub knows that I'm not an ordinary user trying to download a file. I already tried to change the user agent, but that didn't help either.

So, how do I upload a file that is hosted on my Github account using Java?

EDIT: I tried using apache commons-io for this, but I get the same effect as an empty / damaged file.

+6
source share
5 answers

This does the job:

 public class Download { private static boolean isRedirected( Map<String, List<String>> header ) { for( String hv : header.get( null )) { if( hv.contains( " 301 " ) || hv.contains( " 302 " )) return true; } return false; } public static void main( String[] args ) throws Throwable { String link = "http://github.com/downloads/TheHolyWaffle/ChampionHelper/" + "ChampionHelper-4.jar"; String fileName = "ChampionHelper-4.jar"; URL url = new URL( link ); HttpURLConnection http = (HttpURLConnection)url.openConnection(); Map< String, List< String >> header = http.getHeaderFields(); while( isRedirected( header )) { link = header.get( "Location" ).get( 0 ); url = new URL( link ); http = (HttpURLConnection)url.openConnection(); header = http.getHeaderFields(); } InputStream input = http.getInputStream(); byte[] buffer = new byte[4096]; int n = -1; OutputStream output = new FileOutputStream( new File( fileName )); while ((n = input.read(buffer)) != -1) { output.write( buffer, 0, n ); } output.close(); } }
+2
source

It seems that GitHub gives you several redirection levels when you request this file and fooobar.com/questions/82263 / ... says that URLConnection will not automatically follow redirects that change the protocol. Here is what I see with curl:

First request:

 curl -v http://github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar * About to connect() to github.com port 80 (#0) * Trying 207.97.227.239... connected * Connected to github.com (207.97.227.239) port 80 (#0) > GET /downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar HTTP/1.1 > User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5 > Host: github.com > Accept: */* > < HTTP/1.1 301 Moved Permanently < Server: nginx < Date: Sun, 18 Nov 2012 15:56:36 GMT < Content-Type: text/html < Content-Length: 178 < Connection: close < Location: https://github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar < <html> <head><title>301 Moved Permanently</title></head> <body bgcolor="white"> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx</center> </body> </html> * Closing connection #0

Correction for this location header:

 curl -v https://github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar * About to connect() to github.com port 443 (#0) * Trying 207.97.227.239... connected * Connected to github.com (207.97.227.239) port 443 (#0) * SSLv3, TLS handshake, Client hello (1): * SSLv3, TLS handshake, Server hello (2): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSL connection using RC4-SHA * Server certificate: * subject: businessCategory=Private Organization; 1.3.6.1.4.1.311.60.2.1.3=US; 1.3.6.1.4.1.311.60.2.1.2=California; serialNumber=C3268102; C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=github.com * start date: 2011-05-27 00:00:00 GMT * expire date: 2013-07-29 12:00:00 GMT * subjectAltName: github.com matched * issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert High Assurance EV CA-1 * SSL certificate verify ok. > GET /downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar HTTP/1.1 > User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5 > Host: github.com > Accept: */* > < HTTP/1.1 302 Found < Server: nginx < Date: Sun, 18 Nov 2012 15:58:56 GMT < Content-Type: text/html; charset=utf-8 < Connection: keep-alive < Status: 302 Found < Strict-Transport-Security: max-age=2592000 < Cache-Control: no-cache < X-Runtime: 48 < Location: http://cloud.github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar < X-Frame-Options: deny < Content-Length: 149 < * Connection #0 to host github.com left intact * Closing connection #0 * SSLv3, TLS alert, Client hello (1): <html><body>You are being <a href="http://cloud.github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar">redirected</a>.</body></html>

The header header in this answer returns the actual file. You can use the Apache HTTP client to download it. You can configure it to follow these redirects 301 and 302 during GET.

+3
source

I have found a solution.

Explicitly http://github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar does not link directly to my file.

When viewing the resulting banner with a text editor, I found this:

 <html><body>You are being <a href="http://cloud.github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar">redirected</a>.</body></html>

So this means that the direct link is as follows: http://cloud.github.com/downloads/TheHolyWaffle/ChampionHelper/ChampionHelper-4.jar

And with this link, I can download the file using my method without any problems.

+1
source

Get a direct download link to the source binary, for example. https://github.com/xerial/sqlite-jdbc/blob/master/src/main/resources/org/sqlite/native/Windows/x86_64/sqlitejdbc.dll?raw=true by copying the View Raw link:

Finally, use the following code snippet to download the file:

 public static void download(String downloadURL) throws IOException { URL website = new URL(downloadURL); String fileName = getFileName(downloadURL); try (InputStream inputStream = website.openStream()) { Files.copy(inputStream, Paths.get(fileName), StandardCopyOption.REPLACE_EXISTING); } } public static String getFileName(String downloadURL) { String baseName = FilenameUtils.getBaseName(downloadURL); String extension = FilenameUtils.getExtension(downloadURL); String fileName = baseName + "." + extension; int questionMarkIndex = fileName.indexOf("?"); if (questionMarkIndex != -1) { fileName = fileName.substring(0, questionMarkIndex); } fileName = fileName.replaceAll("-", ""); return URLDecoder.decode(fileName, "UTF-8"); }

You will also need the Apache Commons IO maven dependency for the FilenameUtils class:

 <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>LATEST</version> </dependency>
0
source

More articles:


All Articles