Geek Answers Handbook

Java: how to read content from redirected urls?

I use the following Java code in a Bean to read the contents of a URL:

String url; String inputLine; StringBuilder srcCode=new StringBuilder(); public void setUrl (String value) { url = value; } private void scanWebPage() throws IOException { try { URL dest = new URL(url); URLConnection yc = dest.openConnection(); yc.setUseCaches(false); BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream())); while ((inputLine = in.readLine()) != null) srcCode = srcCode.append (inputLine); in.close(); } catch (FileNotFoundException fne) { srcCode.append("File Not Found") ; } }

The code works fine for most URLs, but does not work for redirected URLs. How can I update the above code to read content from redirected URLs? For redirected URLs, I get "File Not Found" .

+6
source share
2 answers

Give the following:

  HttpURLConnection yc = (HttpURLConnection) dest.openConnection(); yc.setInstanceFollowRedirects( true );

In the context of your code above:

  `String url = "http://java.sun.com"; String inputLine; StringBuilder srcCode=new StringBuilder(); URL dest = new URL(url); HttpURLConnection yc = (HttpURLConnection) dest.openConnection(); yc.setInstanceFollowRedirects( true ); yc.setUseCaches(false); BufferedReader in = new BufferedReader( new InputStreamReader( yc.getInputStream())); while ((inputLine = in.readLine()) != null) { srcCode = srcCode.append (inputLine); } in.close();`

Modified further to help you determine what is happening. This code disables automatic redirection, and then manually prints out the location headers when it goes.

 @Test public void f() throws IOException { String url = "http://java.sun.com"; fetchURL(url); } private HttpURLConnection fetchURL( String url ) throws IOException { URL dest = new URL(url); HttpURLConnection yc = (HttpURLConnection) dest.openConnection(); yc.setInstanceFollowRedirects( false ); yc.setUseCaches(false); System.out.println( "url = " + url ); int responseCode = yc.getResponseCode(); if ( responseCode >= 300 && responseCode < 400 ) { // brute force check, far too wide return fetchURL( yc.getHeaderField( "Location") ); } System.out.println( "yc.getResponseCode() = " + yc.getResponseCode() ); return yc; }
+4
source

this is not a debugger of your program, but you can consider this

 public class GetURLData { public static void main(String args[]) { String url = "the url you want the response from"; HttpClient httpClient = new DefaultHttpClient(); HttpPost httpPost = new HttpPost(url); HttpResponse response; StringBuilder builder= new StringBuilder(); try { response = httpClient.execute(httpPost); BufferedReader in = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8")); char[] buf = new char[8000]; int l = 0; while (l >= 0) { builder.append(buf, 0, l); l = in.read(buf); } System.out.println(builder.toString); } catch (Exception e) { System.out.println("Exception is :"+e); e.printStackTrace(); } } }
0
source

More articles:


All Articles