CookieManager for multiple streams

Question

I am trying to make multiple connections through streams.

But each connection seems to override other cookies, resulting in connections using the wrong cookies.

inside the constructor with the stream class:

manager = new CookieManager(); manager.setCookiePolicy(CookiePolicy.ACCEPT_ALL); CookieHandler.setDefault(manager); 

Any way to manage cookies in a stream or in a class?

New failed attempt:

Now each thread uses its own index, but still it seems they override each other in a cookie-wise. Any ideas?

 public class threadedCookieStore implements CookieStore, Runnable { CookieStore[] store = new CookieStore[1000]; int index; public threadedCookieStore(int new_index) { index = new_index; // get the default in memory cookie store store[index] = new CookieManager().getCookieStore(); // todo: read in cookies from persistant storage // and add them store // add a shutdown hook to write out the in memory cookies Runtime.getRuntime().addShutdownHook(new Thread(this)); } public void run() { // todo: write cookies in store to persistent storage } public void add(URI uri, HttpCookie cookie) { store[index].add(uri, cookie); } public List<HttpCookie> get(URI uri) { return store[index].get(uri); } public List<HttpCookie> getCookies() { return store[index].getCookies(); } public List<URI> getURIs() { return store[index].getURIs(); } public boolean remove(URI uri, HttpCookie cookie) { return store[index].remove(uri, cookie); } public boolean removeAll() { return store[index].removeAll(); } } 

Inside the class:

 threadedCookieStore cookiestore = new threadedCookieStore(index); manager = new CookieManager(cookiestore,CookiePolicy.ACCEPT_ALL); manager.setCookiePolicy(CookiePolicy.ACCEPT_ALL); CookieHandler.setDefault(manager); 

Answer 1

Thanks to everyone.

I reviewed all the answers, but no one had a complete solution.

Since google'ing this problem leads to this page here, I will lay out a complete solution and accept my answer:

HowTo:

1 Extend CookieHandler to SessionCookieManager

this is based on How to use different cookies for each connection using HttpURLConnection and CookieManager in Java , nivs describes it correctly, does not provide a complete tho solution, So most / all credits go to it, I just do a full HowTo. SessionCookieManager is based on the Java source code http://docs.oracle.com/javase/7/docs/api/java/net/CookieManager.html

 import java.io.IOException; import java.net.CookieHandler; import java.net.CookiePolicy; import java.net.CookieStore; import java.net.HttpCookie; import java.net.URI; import java.util.Collections; import java.util.Comparator; import java.util.List; import java.util.Map; public class SessionCookieManager extends CookieHandler { private CookiePolicy policyCallback; public SessionCookieManager() { this(null, null); } private final static SessionCookieManager ms_instance = new SessionCookieManager(); public static SessionCookieManager getInstance() { return ms_instance; } private final static ThreadLocal<CookieStore> ms_cookieJars = new ThreadLocal<CookieStore>() { @Override protected synchronized CookieStore initialValue() { return new InMemoryCookieStore(); } }; public void clear() { getCookieStore().removeAll(); } public SessionCookieManager(CookieStore store, CookiePolicy cookiePolicy) { // use default cookie policy if not specify one policyCallback = (cookiePolicy == null) ? CookiePolicy.ACCEPT_ALL //note that I changed it to ACCEPT_ALL : cookiePolicy; // if not specify CookieStore to use, use default one } public void setCookiePolicy(CookiePolicy cookiePolicy) { if (cookiePolicy != null) policyCallback = cookiePolicy; } public CookieStore getCookieStore() { return ms_cookieJars.get(); } public Map<String, List<String>> get(URI uri, Map<String, List<String>> requestHeaders) throws IOException { // pre-condition check if (uri == null || requestHeaders == null) { throw new IllegalArgumentException("Argument is null"); } Map<String, List<String>> cookieMap = new java.util.HashMap<String, List<String>>(); // if there no default CookieStore, no way for us to get any cookie if (getCookieStore() == null) return Collections.unmodifiableMap(cookieMap); List<HttpCookie> cookies = new java.util.ArrayList<HttpCookie>(); for (HttpCookie cookie : getCookieStore().get(uri)) { // apply path-matches rule (RFC 2965 sec. 3.3.4) if (pathMatches(uri.getPath(), cookie.getPath())) { cookies.add(cookie); } } // apply sort rule (RFC 2965 sec. 3.3.4) List<String> cookieHeader = sortByPath(cookies); cookieMap.put("Cookie", cookieHeader); return Collections.unmodifiableMap(cookieMap); } public void put(URI uri, Map<String, List<String>> responseHeaders) throws IOException { // pre-condition check if (uri == null || responseHeaders == null) { throw new IllegalArgumentException("Argument is null"); } // if there no default CookieStore, no need to remember any cookie if (getCookieStore() == null) return; for (String headerKey : responseHeaders.keySet()) { // RFC 2965 3.2.2, key must be 'Set-Cookie2' // we also accept 'Set-Cookie' here for backward compatibility if (headerKey == null || !(headerKey.equalsIgnoreCase("Set-Cookie2") || headerKey.equalsIgnoreCase("Set-Cookie") ) ) { continue; } for (String headerValue : responseHeaders.get(headerKey)) { try { List<HttpCookie> cookies = HttpCookie.parse(headerValue); for (HttpCookie cookie : cookies) { if (shouldAcceptInternal(uri, cookie)) { getCookieStore().add(uri, cookie); } } } catch (IllegalArgumentException e) { // invalid set-cookie header string // no-op } } } } /* ---------------- Private operations -------------- */ // to determine whether or not accept this cookie private boolean shouldAcceptInternal(URI uri, HttpCookie cookie) { try { return policyCallback.shouldAccept(uri, cookie); } catch (Exception ignored) { // pretect against malicious callback return false; } } /* * path-matches algorithm, as defined by RFC 2965 */ private boolean pathMatches(String path, String pathToMatchWith) { if (path == pathToMatchWith) return true; if (path == null || pathToMatchWith == null) return false; if (path.startsWith(pathToMatchWith)) return true; return false; } /* * sort cookies with respect to their path: those with more specific Path attributes * precede those with less specific, as defined in RFC 2965 sec. 3.3.4 */ private List<String> sortByPath(List<HttpCookie> cookies) { Collections.sort(cookies, new CookiePathComparator()); List<String> cookieHeader = new java.util.ArrayList<String>(); for (HttpCookie cookie : cookies) { // Netscape cookie spec and RFC 2965 have different format of Cookie // header; RFC 2965 requires a leading $Version="1" string while Netscape // does not. // The workaround here is to add a $Version="1" string in advance if (cookies.indexOf(cookie) == 0 && cookie.getVersion() > 0) { cookieHeader.add("$Version=\"1\""); } cookieHeader.add(cookie.toString()); } return cookieHeader; } static class CookiePathComparator implements Comparator<HttpCookie> { public int compare(HttpCookie c1, HttpCookie c2) { if (c1 == c2) return 0; if (c1 == null) return -1; if (c2 == null) return 1; // path rule only applies to the cookies with same name if (!c1.getName().equals(c2.getName())) return 0; // those with more specific Path attributes precede those with less specific if (c1.getPath().startsWith(c2.getPath())) return -1; else if (c2.getPath().startsWith(c1.getPath())) return 1; else return 0; } } } 

Please note that in my case I changed the default value of CookiePolicy to ACCEPT_ALL

2 In the global scope, before starting any threads, call:

 CookieHandler.setDefault(SessionCookieManager.getInstance()); 

3 When your thread is finished, call it inside:

 SessionCookieManager.getInstance().clear(); 

again: not my idea, just putting it together. All loans are Java and https://stackoverflow.com/users/1442259/nivs

Answer 2

Thank you, I tried to use your answer, but it was based on the old version of CookieManager (maybe why you should have used ACCEPT_ALL) and referred to the package-closed InMemoryCookieStore, so it inspired me to make the final decision. It should have been obvious to all of us before: ThreadLocal CookieStore proxy class.

 CookieHandler.setDefault(new CookieManager(new ThreadLocalCookieStore(), null)); 

from

 import java.net.CookieManager; import java.net.CookieStore; import java.net.HttpCookie; import java.net.URI; import java.util.List; public class ThreadLocalCookieStore implements CookieStore { private final static ThreadLocal<CookieStore> ms_cookieJars = new ThreadLocal<CookieStore>() { @Override protected synchronized CookieStore initialValue() { return (new CookieManager()).getCookieStore(); /*InMemoryCookieStore*/ } }; @Override public void add(URI uri, HttpCookie cookie) { ms_cookieJars.get().add(uri, cookie); } @Override public List<HttpCookie> get(URI uri) { return ms_cookieJars.get().get(uri); } @Override public List<HttpCookie> getCookies() { return ms_cookieJars.get().getCookies(); } @Override public List<URI> getURIs() { return ms_cookieJars.get().getURIs(); } @Override public boolean remove(URI uri, HttpCookie cookie) { return ms_cookieJars.get().remove(uri, cookie); } @Override public boolean removeAll() { return ms_cookieJars.get().removeAll(); } } 

It seems to me like a charm

Answer 3

You can set a CookieHandler that manages the ThreadLocal CookieManager instances.

Answer 4

CookieStore ThreadLocal CookieStore is the best way to go. For the sake of memory efficiency, I provide here a simple implementation of a regular CookieStore , so you do not need to instantiate an entire CookieManager for each stream (if you have more than a few).

 /** * @author lidor * A simple implementation of CookieStore */ public class CookieJar implements CookieStore { private Map<URI, List<HttpCookie>> jar; private List<HttpCookie> freeCookies; public CookieJar() { jar = new HashMap<URI, List<HttpCookie>>(); freeCookies = new ArrayList<HttpCookie>(); } @Override public void add(URI uri, HttpCookie cookie) { if (uri != null) { if (!jar.containsKey(uri)) jar.put(uri, new ArrayList<HttpCookie>()); List<HttpCookie> cookies = jar.get(uri); cookies.add(cookie); } else { freeCookies.add(cookie); } } @Override public List<HttpCookie> get(URI uri) { Log.trace("CookieJar.get (" + this + ") called with URI " + uri + " (host=" + uri.getHost() + ")"); List<HttpCookie> liveCookies = new ArrayList<HttpCookie>(); if (jar.containsKey(uri)) { for (HttpCookie cookie : jar.get(uri)) { if (!cookie.hasExpired()) liveCookies.add(cookie); } } for (HttpCookie cookie : getCookies()) { if (cookie.getDomain().equals(uri.getHost())) if (!liveCookies.contains(cookie)) liveCookies.add(cookie); } return Collections.unmodifiableList(liveCookies); } @Override public List<HttpCookie> getCookies() { List<HttpCookie> liveCookies = new ArrayList<HttpCookie>(); for (URI uri : jar.keySet()) for (HttpCookie cookie : jar.get(uri)) { if (!cookie.hasExpired()) liveCookies.add(cookie); } for (HttpCookie cookie : freeCookies) { if (!cookie.hasExpired()) liveCookies.add(cookie); } return Collections.unmodifiableList(liveCookies); } @Override public List<URI> getURIs() { return Collections.unmodifiableList(new ArrayList<URI>(jar.keySet())); } @Override public boolean remove(URI uri, HttpCookie cookie) { if (jar.containsKey(uri)) { return jar.get(uri).remove(cookie); } else { return freeCookies.remove(cookie); } } @Override public boolean removeAll() { boolean ret = (jar.size() > 0) || (freeCookies.size() > 0); jar.clear(); freeCookies.clear(); return ret; } } 

So, if you have this CookieJar, you can change the ms_cookieJars to this:

 private final static ThreadLocal<CookieStore> ms_cookieJars = new ThreadLocal<CookieStore>() { @Override protected synchronized CookieStore initialValue() { return new CookieJar(); } }; 

Answer 5

Based on the answers in this thread, I created another very simple ThreadLocalCookieStore implementation and clicked on GitHub (also providing it as a Maven dependency):

 public class ThreadLocalCookieStore implements CookieStore { private final static ThreadLocal<CookieStore> stores = new ThreadLocal<CookieStore>() { @Override protected synchronized CookieStore initialValue() { return (new CookieManager()).getCookieStore(); //InMemoryCookieStore } }; @Override public void add(URI uri, HttpCookie cookie) { getStore().add(uri,cookie); } @Override public List<HttpCookie> get(URI uri) { return getStore().get(uri); } @Override public List<HttpCookie> getCookies() { return getStore().getCookies(); } @Override public List<URI> getURIs() { return getStore().getURIs(); } @Override public boolean remove(URI uri, HttpCookie cookie) { return getStore().remove(uri,cookie); } @Override public boolean removeAll() { return getStore().removeAll(); } @Override public int hashCode() { return getStore().hashCode(); } protected CookieStore getStore() { return stores.get(); } public void purgeStore() { stores.remove(); } } 

Without a lot of code, it’s very easy to set up cookie storage with any policy value, for example:

 CookieHandler.setDefault(new java.net.CookieManager( new ThreadLocalCookieStore(), CookiePolicy.ACCEPT_ALL)); 

In addition, the dependency function has a small sevlet @WebFilter to separate cookie stores on multiple serlvet requests, if necessary.

Answer 6

You can set a different path for cookies. This way it will not be overwritten.

http://docs.oracle.com/javase/6/docs/api/java/net/HttpCookie.html#setPath%28java.lang.String%29

Answer 7

What about ThreadLocal CookieManager? Same idea as some other answers, but apparently requires less code:

 public class ThreadLocalCookies extends CookieManager { private static CookiePolicy s_policy = null; private static ThreadLocal<CookieManager> s_impl = new ThreadLocal<CookieManager>() { @Override protected CookieManager initialValue() { if (null == s_policy) { throw new IllegalStateException("Call install() first"); } return new CookieManager(null, s_policy); } }; public static void install() { install(CookiePolicy.ACCEPT_ALL); } public static void install(CookiePolicy policy) { s_policy = policy; CookieHandler.setDefault(new ThreadLocalCookies()); } public static void clear() { s_impl.set(new CookieManager(null, s_policy)); } @Override public Map<String, List<String>> get(URI uri, Map<String, List<String>> requestHeaders) throws IOException { return s_impl.get().get(uri, requestHeaders); } @Override public void put(URI uri, Map<String,List<String>> responseHeaders) throws IOException { s_impl.get().put(uri, responseHeaders); } }