How to Log in to a Website Using Java

Question

I want to access some pages of the https://myoffice.bt.com website which requires user authentication with java. We must first log in to access the pages. I read the following code.

package root; import java.io.IOException; import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.UsernamePasswordCredentials; import org.apache.commons.httpclient.auth.AuthScope; import org.apache.commons.httpclient.methods.PostMethod; import org.apache.commons.httpclient.params.HttpMethodParams; public class Url { public static void main(String[] args) throws IOException { HttpClient client = new HttpClient(); client.getParams().setParameter( HttpMethodParams.USER_AGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2" ); client.getState().setCredentials( new AuthScope("https://myoffice.bt.com", 443, AuthScope.ANY_REALM), new UsernamePasswordCredentials("username", "password") ); PostMethod get = new PostMethod("https://myoffice.bt.com/youraccount/default.aspx"); get.setDoAuthentication( true ); System.out.println(get.getFollowRedirects()); //get.setFollowRedirects(true); try { // execute the GET int status = client.executeMethod( get ); // print the status and response System.out.println(status + "\n" + get.getResponseBodyAsString()); } finally { // release any connection resources used by the method get.releaseConnection(); } } } 

But he gives the following errors.

 > Jun 22, 2010 12:14:40 PM org.apache.commons.httpclient.HttpMethodDirector isRedirectNeeded INFO: Redirect requested but followRedirects is disabled 302 

If I uncomment the line get.setFollowingRedirects, it gives another error.

 Exception in thread "main" java.lang.IllegalArgumentException: Entity enclosing requests cannot be redirected without user intervention at org.apache.commons.httpclient.methods.EntityEnclosingMethod.setFollowRedirects(Unknown Source) at root.Url.main(Url.java:30) 

Can someone help me? Can we do form-based authentication with HttpClient?

Thanks.

Answer 1

First, please do not name your PostMethod get variable.

Secondly, try the following:

 PostMethod post = new PostMethod("yourUrl") { @Override public boolean getFollowRedirects() { return true; } }; 

If you ever found yourself on the “other side” and want your users not to be hurt, use the 303 (See Other) response code when redirecting the POST request to get instead of the usual 302 and 301 (for RFC ). Regular browsers are generally good, break the rules and DO NOT ask us to confirm these redirects, but many mobile browsers still do.

As for your question about forms-based authentication, you just need to find out the names of the parameters to use (for example, by looking at the source of the website on which you usually “log in” to the system) and then fill them with the appropriate values:

 post.addParameter("username", username); post.addParameter("password", password); 

---

I played with the login form at myoffice.bt.com, there are a few things in JavaScript.

The form is submitted https://myoffice.bt.com/siteminderagent/forms/login.fcc

The form elements that were submitted were as follows ( name=value , some values ​​were empty):

 Segment=btb.hub SubSegment= searchType=0 searchPlatform=BEA lob=btb.hub queryText= searchText= ctl00$ masterWebpartManager$gwpCustomLogin1$CustomLogin1$UserName=your@ email.com ctl00$masterWebpartManager$gwpCustomLogin1$CustomLogin1$PWD=yourpwd ctl00$masterWebpartManager$gwpCustomLogin1$CustomLogin1$RememberMe=on USER=[e-mail] PASSWORD=yourpwd SMENC=ISO-8859-1 SMLOCALE=US-EN userFirstLoginUrl=https://myoffice.bt.com/ManageBusinessApplications/SecretQA.aspx PrivateLoginSuccessUrl=https://myoffice.bt.com/sm/privatecreatesession.aspx?siteArea=btb.mya PublicLoginSuccessUrl=https://myoffice.bt.com/sm/createsession.aspx?siteArea=btb.mya target=https://myoffice.bt.com/sm/privatecreatesession.aspx?siteArea=btb.mya&TARGET=https%3a%2f%2fmyoffice.bt.com%2fdefault.aspx (hidden) submitStatus= smauthreason= smagentname= postpreservationdata= AnonUserName=[e-mail] authMode=SITEMINDER smUrl=https://myoffice.bt.com/siteminderagent/forms/login.fcc notSMUrl=https://myoffice.bt.com/default.aspx smIdentifier=1 

Try adding some or all of them (at least USER and PASSWORD ) to your PostMethod and make sure that you submit the correct URL.

Answer 2

If this site uses Siteminder authentication, you won’t be able to log in that way. Siteminder uses cookies to identify authenticated sessions. These cookies are valid only as long as your session is alive. If you are not logged in, the server redirects you to the Siteminder login page (hence, redirect). So, you will need to redirect, send your credentials (username / password), and then redirect again by sending the received cookies.

I recorded sessions for regression tests using The Grinder ( http://grinder.sourceforge.net/ ) and he was able to log into the Siteminder secure site automatically! So it is definitely possible, but you will have to do a little more than just send an HTTP request ...

The best solution would be some other authentication, for example, certificate-based authentication (but, of course, this should be configured on the server side too, so in this case it may not be an option). Why not ask BT if they provide other authentication methods?

EDIT: I just found this: http://www.codeproject.com/KB/IP/SiteminderHttpWebRequest.aspx The source code is in VB but the article is excellent and there should be no problem translating VB code into Java ...; -)

Answer 3

Java version: works well with site-protected resources verified with commons httpClient 4.3.3

 import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; import java.security.KeyManagementException; import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.security.SecureRandom; import java.util.List; import javax.net.ssl.SSLContext; import org.apache.http.Header; import org.apache.http.HttpEntity; import org.apache.http.client.ClientProtocolException; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.client.methods.HttpUriRequest; import org.apache.http.client.methods.RequestBuilder; import org.apache.http.conn.ssl.SSLConnectionSocketFactory; import org.apache.http.conn.ssl.SSLContexts; import org.apache.http.conn.ssl.TrustSelfSignedStrategy; import org.apache.http.cookie.Cookie; import org.apache.http.impl.client.BasicCookieStore; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.impl.client.LaxRedirectStrategy; import org.apache.http.util.EntityUtils; public class AccessSiteminderProtectedResource { private static final String PASSWORD = "pwd"; private static final String USER_NAME = "userId"; private static final String SITEMINDER_PROTECTED_RESOURCE = "protectedResource"; private static final String SITEMINDER_LOGIN_URL = "siteMinderLoginUrl?TARGET=-SM-" + SITEMINDER_PROTECTED_RESOURCE; public static void main(String[] args) throws Exception { BasicCookieStore cookieStore = new BasicCookieStore(); SSLContext sslcontext = buildSSLContext(); SSLConnectionSocketFactory sslsf = buildSSLConnectionSocketFactory(sslcontext); CloseableHttpClient httpclient = buildHttpClient(cookieStore, sslsf); try { String nextLocation = executeLogin(cookieStore, httpclient); accessApp(httpclient, nextLocation); } finally { httpclient.close(); } } private static SSLContext buildSSLContext() throws NoSuchAlgorithmException, KeyManagementException, KeyStoreException { SSLContext sslcontext = SSLContexts.custom() .setSecureRandom(new SecureRandom()) .loadTrustMaterial(null, new TrustSelfSignedStrategy()).build(); return sslcontext; } private static SSLConnectionSocketFactory buildSSLConnectionSocketFactory( SSLContext sslcontext) { SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory( sslcontext, SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER); return sslsf; } private static CloseableHttpClient buildHttpClient( BasicCookieStore cookieStore, SSLConnectionSocketFactory sslsf) { CloseableHttpClient httpclient = HttpClients.custom() .setSSLSocketFactory(sslsf).setDefaultCookieStore(cookieStore) .setRedirectStrategy(new LaxRedirectStrategy()) .build(); return httpclient; } private static String executeLogin(BasicCookieStore cookieStore, CloseableHttpClient httpclient) throws URISyntaxException, IOException, ClientProtocolException { HttpUriRequest loginPost = RequestBuilder .post() .setUri(new URI(SITEMINDER_LOGIN_URL)) .addParameter("USER", USER_NAME) .addParameter("PASSWORD", PASSWORD).build(); System.out.println("executing request" + loginPost.getRequestLine() + "\n"); CloseableHttpResponse loginResponse = httpclient.execute(loginPost); String nexLocation; try { HttpEntity loginResponseEntity = loginResponse.getEntity(); System.out.println("Login form post Status: " + loginResponse.getStatusLine()); EntityUtils.consume(loginResponseEntity); System.out.println(); System.out.println("Post logon cookies:"); System.out.println(); displayCookies(cookieStore); System.out.println(); System.out.println(); System.out.println("Login Post Headers----------------------------------------"); displayHeaders(loginResponse); System.out.println(); System.out.println(); nexLocation = SITEMINDER_PROTECTED_RESOURCE; } finally { loginResponse.close(); } return nexLocation; } private static void accessApp(CloseableHttpClient httpclient, String nextLocation) throws IOException, ClientProtocolException { HttpGet appGet = new HttpGet(nextLocation); System.out.println("executing request" + appGet.getRequestLine()); CloseableHttpResponse response = httpclient.execute(appGet); try { HttpEntity entity = response.getEntity(); System.out.println("\n\n\n\n---------------------------------------- \n"); System.out.println("App Get Status: " + response.getStatusLine()); System.out.println(EntityUtils.toString(entity)); EntityUtils.consume(entity); } finally { response.close(); } } private static void displayHeaders(CloseableHttpResponse loginResponse) { for (Header header : loginResponse.getAllHeaders()) { System.out.println(header); } } private static void displayCookies(BasicCookieStore cookieStore) { List<Cookie> cookies = cookieStore.getCookies(); if (cookies.isEmpty()) { System.out.println("None"); } else { for (int i = 0; i < cookies.size(); i++) { System.out.println("- " + cookies.get(i).toString()); } } } }