Why write to Stream in pieces?

Question

I am wondering why so many examples read byte arrays into streams in cartridges, and not all at once ... I know this is a soft question, but I'm interested.

I understand a little that the equipment and fill buffers can be very size dependent, and you won’t want to write to the buffer again until it is flushed to where it needs to go, etc .... but with .Net platform (and other modern languages) I see examples of both. So when do you use when and when, or the second absolute, no?

Here is the thing (code) that I mean:

var buffer = new byte[4096]; while (true) { var read = this.InputStream.Read(buffer, 0, buffer.Length); if (read == 0) break; OutputStream.Write(buffer, 0, read); } 

but not:

 var buffer = new byte[InputStream.Length]; var read = this.InputStream.Read(buffer, 0, buffer.Length); OutputStream.Write(buffer, 0, read); 

I believe both are legal? So, why go through the whole bustle of the while loop (in which would you decide to structure it)?

I play devil's advocates here because I want to study the way I can :)

Answer 1

In the first case, you need only 4 KB of memory. In the second case, you will need as much memory as the input stream data. If the input stream is 4 GB, you need 4 GB.

Do you think it would be nice if the file copy operation required 4 GB of RAM? What if you have to prepare a disk image, 20GB?

There is this thing with tubes. You do not often use them on Windows, but a similar case is often found on other operating systems. The second case expects that all data will be read, and only then writes it to the output. However, it is sometimes recommended to write data as soon as possible - the first case will start writing to the output stream as soon as the first 4 KB of input is read. Think about serving web pages: it is recommended that the web server send data as soon as possible, so that the client web browser starts displaying the headers and the first part of the content without waiting for the whole body.

However, if you know that the input stream will not exceed 4 KB, then both cases are equivalent.

Answer 2

Sometimes InputStream.Length is invalid for some source, for example, from a network transport, or the buffer can be huge, for example, read from a huge file. IMO

Answer 3

It protects you from a situation when your input stream is several gigabytes.

Answer 4

You have no idea how much Read data can return. This can lead to serious performance issues if you are reading a very large file.

If you have control over the input and are sure that the size is reasonable, then you can certainly read the entire array right away. But be especially careful if the user can provide arbitrary input.