Explicit xpath memory error?

Question

I have a situation that has tormented me for several months: I continue to receive OOM (Heap Space) exceptions and check heap heaps. I found millions of instances of objects that I never allocated, but which were probably distributed in the base libraries. After a lot of blood, sweat and tears, I managed to localize the code that generated the memory leak, and I made a minimal, complete and verifiable code sample to illustrate this:

import java.util.logging.Level; import java.util.logging.Logger; import javafx.application.Application; import javafx.beans.value.ChangeListener; import javafx.beans.value.ObservableValue; import javafx.concurrent.Worker; import javafx.scene.web.WebEngine; import javafx.stage.Stage; import javax.xml.xpath.XPath; import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpressionException; import javax.xml.xpath.XPathFactory; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class MVC extends Application implements ChangeListener<Worker.State>{ private final WebEngine engine = new WebEngine(); private final String url = "https://biblio.ugent.be/publication?sort=publicationstatus.desc&sort=year.desc&limit=250&start=197000"; private final XPath x = XPathFactory.newInstance().newXPath(); @Override public void start(Stage primaryStage) throws Exception { System.setProperty("jsse.enableSNIExtension", "false"); engine.getLoadWorker().stateProperty().addListener(this); engine.load(url); } public static void main(String[] args) { launch(args); } private NodeList eval(Node context, String xpath) throws XPathExpressionException{ return (NodeList)x.evaluate(xpath, context, XPathConstants.NODESET); } @Override public void changed(ObservableValue<? extends Worker.State> observable, Worker.State oldValue, Worker.State newValue) { if (newValue==Worker.State.SUCCEEDED) { try { while(true){ NodeList eval = eval(engine.getDocument(), "//span[@class='title']"); int s = eval.getLength(); } } catch (XPathExpressionException ex) { Logger.getLogger(MVC.class.getName()).log(Level.SEVERE, null, ex); } } } } 

The code performs the following actions:

• Download the document using JavaFX WebEngine .
• execute the xpath request in the document endlessly using javax.xml packages , without storing the result or pointers in it .

To start, create a JavaFX application, add a file named MVC.java to the default package, enter the code and press mileage. Any profiling tool (I use VisualVM) should quickly show you that in a matter of minutes the heap is growing uncontrollably. The following objects appear to be selected but not released:

• java.util.HashMap$Node
• com.sun.webkit.Disposer$WeakDisposerRecord
• com.sun.webkit.dom.NamedNodeMapImpl$SelfDisposer
• java.util.concurrent.LinkedBlockingQueue$Node

This behavior happens every time I run the code, regardless of the load url or xpath that I execute in the document.

The setup I tested with:

• MBP under OS X Yosemite (updated)
• JDK 1.8.0_60

Can anyone reproduce this problem? Is this an actual memory leak? Is there anything I can do?

change

One of my colleagues reproduced the problem on a w7 machine with JDK 1.8.0_45, and this also happens on the Ubuntu server.

change 2

I tested jaxen as an alternative to the javax.xml package, but the results are the same, which makes me believe that the error lies deep inside webkit sun

Answer 1

I reproduced the leak using jdk1.8.60 on Ubuntu. I have done quite a lot of profiling and debugging, and the main reason is simple, and it can be easily fixed. No memory leak in XPath materials.

There is a class com.sun.webkit.Disposer that continuously cleans up some of the internal structures that are created during XPath evaluation. Elimination within the user causes a cleanup through Invoker.getInvoker (). InvokeOnEventThread (this) ;. You can see this if you decompile the code. There are different invoker implementations using different streams. If you are running JavaFX, Invoker periodically flushes the JavaFX stream.

However, your modified listening method is also called in the JavaFX thread, and it never returns, so cleanup never occurs.

I changed your code so that the changed method only spawns a new thread and returns it, and processing is asynchronous. And guess what - memory no longer grows:

 @Override public void changed(ObservableValue<? extends Worker.State> observable, Worker.State oldValue, Worker.State newValue) { if (newValue==Worker.State.SUCCEEDED) { new Thread(() ->{ try { while(true){ NodeList eval = eval(engine.getDocument(), "//span[@class='title']"); int s = eval.getLength(); } } catch (XPathExpressionException ex) { Logger.getLogger(MVC.class.getName()).log(Level.SEVERE, null, ex); } }).start(); } }