Geek Answers Handbook

How to extract text without HTML tags from a webpage using HtmlUnit?

I'm just starting out with HTMLUnit, and what I'm looking for is to take a webpage and extract raw text from it minus all the html markup.

Can htmlunit accomplish this? If so, how? Or is there another library I should look at?

for example, if the page contains

<body><p>para1 test info</p><div><p>more stuff here</p></div>

I would like him to output

para1 test info more stuff here

thank

+5
source share
1 answer

http://htmlunit.sourceforge.net/gettingStarted.html indicates that this is indeed possible.

@Test
public void homePage() throws Exception {
    final WebClient webClient = new WebClient();
    final HtmlPage page = webClient.getPage("http://htmlunit.sourceforge.net");
    assertEquals("HtmlUnit - Welcome to HtmlUnit", page.getTitleText());

    final String pageAsXml = page.asXml();
    assertTrue(pageAsXml.contains("<body class=\"composite\">"));

    final String pageAsText = page.asText();
    assertTrue(pageAsText.contains("Support for the HTTP and HTTPS protocols"));
}

NB: the page.asText () command seems to offer exactly what you need.

Javadoc for asText (Inherited from DomNode to HtmlPage)

+5

More articles:


All Articles