Memory consumed by the stream

Question

I need to track the amount of memory consumed by threads created by my application. The idea is to take corrective action if the greedy thread consumes too much memory. I mentioned How much memory does my java thread take? . One suggestion for this link is to use getThreadAllocatedBytes in ThreadMXBean. , I experimented with getThreadAllocatedBytes with the following task.

 List<Long> primes = new ArrayList<Long>(); long i = 0; while (true) { primes.add(++i); if ((i % 10) == 0) { primes.clear(); System.runFinalization(); System.gc(); } } 

I have been doing this task on four threads for a considerable amount of time. Although the operation does not accumulate memory continuously, the values ​​returned by getThreadAllocatedBytes continue to increase and do not drop even once. This means that getThreadAllocatedBytes does not return the actual amount of memory in the heap used by the thread. It returns the total amount of memory allocated on the heap for the stream since it started. My platform information is as follows:

Linux PG85213.egi.ericsson.com 3.5.0-030500-generi # 201207211835 SMP Sat Jul 21 22:35:55 ​​UTC 2012 x86_64 x86_64 x86_64 GNU / Linux java version "1.7.0_45"
Java (TM) SE Runtime Environment (build 1.7.0_45-b18) Java HotSpot (TM) 64-bit server VM (build 24.45-b08, mixed mode)

Is the above behavior the desired behavior of getThreadAllocatedBytes ? If so, there is no way to find the effective memory in the heap used by the thread.

Listing the complete help system:

 package workbench; import java.lang.management.ManagementFactory; import com.sun.management.ThreadMXBean; import java.util.ArrayList; import java.util.Collections; import java.util.List; import java.util.concurrent.CountDownLatch; import java.util.concurrent.Executors; import java.util.logging.Level; import java.util.logging.Logger; public class AnotherWorkBench { private static final CountDownLatch latch = new CountDownLatch(4); static final List<Long> threadIds = Collections.synchronizedList(new ArrayList<Long>()); private void dummyJob() { List<Long> primes = new ArrayList<Long>(); long i = 0; while (true) { primes.add(++i); if ((i % 10) == 0) { primes.clear(); //introduce sleep to prevent process hogging try { Thread.currentThread().sleep(2000); } catch (InterruptedException ex) { Logger.getLogger(AnotherWorkBench.class.getName()).log(Level.SEVERE, null, ex); } System.runFinalization(); System.gc(); } } } private void runDummyJobs() { Runnable dummyJob = new Runnable() { @Override public void run() { threadIds.add(Thread.currentThread().getId()); latch.countDown(); dummyJob(); } }; Runnable memoryMonitorJob = new Runnable() { @Override public void run() { System.out.println(Thread.currentThread().getName() + " : Monitor thread started"); ThreadMXBean threadMxBean = (ThreadMXBean) ManagementFactory.getThreadMXBean(); threadMxBean.setThreadAllocatedMemoryEnabled(true); while (true) { for (Long threadId : threadIds) { System.out.println(Thread.currentThread().getName() + " : Thread ID : " + threadId + " : memory = " + threadMxBean.getThreadAllocatedBytes(threadId) + " bytes"); } //wait between subsequent scans try { System.out.println(Thread.currentThread().getName() + " : secondary sleep"); Thread.currentThread().sleep(5000); System.out.println(Thread.currentThread().getName() + " : out of secondary sleep"); } catch (InterruptedException ex) { Logger.getLogger(WorkBench.class.getName()).log(Level.SEVERE, null, ex); } } } }; Executors.newSingleThreadExecutor().submit(dummyJob); Executors.newSingleThreadExecutor().submit(dummyJob); Executors.newSingleThreadExecutor().submit(dummyJob); Executors.newSingleThreadExecutor().submit(dummyJob); try { latch.await(); } catch (InterruptedException ex) { Logger.getLogger(AnotherWorkBench.class.getName()).log(Level.SEVERE, null, ex); } Executors.newSingleThreadExecutor().submit(memoryMonitorJob); } /** * @param args the command line arguments */ public static void main(String[] args) { new AnotherWorkBench().runDummyJobs(); } } 

Answer 1

As far as I know, there is no reliable way to do this at runtime. And as indicated in the original question , the heap is a shared resource, and therefore the heap size of one thread does not make sense, since it will overlap with links to objects from other threads.

However, when I want to know the 'saved' size of one stream, and yes the saved size is another, but similar metric to the one you requested, then I do this by taking a bunch of dump and then using MAT ( http: // www .eclipse.org / mat / ).

I knew that people use Java agents to control the distribution of objects , and then use a weak link to monitor when receiving GC "d. However, the performance impact on this is very large. Very high.

It is best to use runtime heuristics and unit testing to ensure that memory stays within . For example, you can use JMX to control heap sizes, and when you see the old generation grow, you can raise a warning. Using getThreadAllocatedBytes to calculate the distribution speed can also be useful.

Good runtime monitoring tools: appdynamics , newrelic , visualvm and yourkit

For offline memory analysis, mat and jclarity are very good.

A very useful tool helping one place, whether it be a leak or at least different from expectations, is to print a count of the number of instances of each class currently on the heap: jcmd <pid> GC.class_histogram .

Answer 2

Java VisualVM can be used to "monitor a local application and view real-time data at a high level in the heap of memory, activity flow, and classes loaded into the Java Virtual Machine (JVM). Monitoring the application incurs a small overhead and can be used for long periods of time .

See also How to control Java memory usage? for other features.