Just for fun, I'm testing how good JSOUP is in getting page content, and I'm very disappointed with its performance.
JSOUP seems to be having trouble extracting the source code from ASK.com pages, although I can see all the source code in Chrome without any problems.
I think JSOUP is either going short here, or I'm doing something wrong. I want to clarify what’s the matter ...
Could you tell me why the following does not work in order to get the whole document from the page:
String url = "http://www.ask.com/answers/428870481/what-are-some-remedies-to-put-a-toddler-to-sleep"; String ua = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"; Document doc = Jsoup.connect(url).userAgent(ua).ignoreHttpErrors(true).timeout(30 * 1000).get();
If you check the contents of the document, it is truncated with a "..." at the end. Everything that was loaded only with partial content, and not with all the code. Why?
source share