How do threads in Java affect memory consumption?

Question

I use threads many times, but I have never read much about how they actually work. I also don’t know much about them, except that the flow is just a metaphor. A stream represents only a sequence of bytes. I don’t know much about how they actually work, I assume that opening a file stream in Java interacts with OSs that have functionality to give a “pointer” to the stream.

Basically, my question is how threads affect memory consumption. When you have, for example, an input stream and you start reading from it, do you just start to increase memory consumption with the number of bytes read? When you open a stream in Java, are you actually not loading the complete file before reading? If you read from one stream and write directly to another stream, do you increase the amount of memory with only the amount of bytes read (and possibly in the buffer)? If you read bytes into a byte array in java, then do you increase memory consumption with file size?

It may seem like a strange question, but I may need some kind of guidance / correction as I understand it. Thanks.

Answer 1

After you start reading from InputStream almost no significant memory overhead. To open a file, there is very little OS overhead and tiny overhead in the JVM for new placement of objects. There might also be a little overhead if you use a BufferedInputStream , which is 8K by default.

The overhead for writing depends very much on where you write. If it is a FileOutputStream , then it will be the same as described above. If this is a ByteArrayOutputStream , then it is (2 * stream length) bytes in the best case and (3 * stream length) in the worst case. That is, to copy 10k bytes from an InputStream to a byte array, 30k bytes will be allocated in the worst case.

The reason for this is that the size increase of ByteArrayOutputStream increases 2 times after reaching the limit, and also allocates a new buffer when toByteArray() called.

Answer 2

All of the answers above are great answers, but I don’t think they answer your original question about memory consumption.

In Java, you can view streams in several ways. First you have Raw streams, which are the lowest level stream and interact with the base OS (File, Network, etc.) with minimal memory overhead. Secondly, these are buffered streams that can be used to wrap a raw stream and add some buffering and significantly improve performance. Stream buffering adds a fixed amount of memory overhead for buffering and can be set by your application. Not sure what the default value is, but it is probably something minimal like 32K.

The third type of stream is a memory stream (i.e. ByteArrayInput / Ouput), they use as much memory as you write to them, and will grow as needed, and not get rid of their memory until the reference counter reaches zero (they are no more a long). These threads are very useful, but obviously can consume a lot of memory.

The final type is not really a stream, but an I / O class called Readers, which provide assistance in converting data to and from a stream, as mentioned above. These flows work either on raw materials. buffered or memory stream and will consume the same amount of memory as the main stream that is used.