Good design: how to pass InputStreams as an argument?

Question

I have a large file on which I open FileInputStream. This file contains some files, each of which has an offset from the beginning and size. In addition, I have a parser that should evaluate such a contained file.

File file = ...; // the big file long offset = 1734; // a contained file offset long size = 256; // a contained file size FileInputStream fis = new FileInputStream(file ); fis.skip(offset); parse(fis, size); public void parse(InputStream is, long size) { // parse stream data and insure we don't read more than size bytes is.close(); } 

I feel this is not a good practice. Is there a better way to do this, possibly using buffering?

In addition, I feel that the skip () method slows down the reading process a lot.

Answer 1

It sounds like what you really want is a kind of "partial" input stream - the same as ZipInputStream, where you have the stream in the stream.

You can write this yourself by proxying all the InputStream methods to the original input stream, creating the appropriate settings for the offset, and checking reading beyond the end of the subfile.

Is that what you are talking about?

Answer 2

Firstly, FileInputStream.skip () has an error that can make the file skip above the EOF file marker, so be careful of this.

I personally found that working with Input / OutputStreams is a pain compared to using FileReader and FileWriter, and you show the main problem that I am facing: the need to close streams after use. One of the problems is that you can never be sure that you have closed all resources correctly unless you make the code too careful, like this:

 public void parse(File in, long size) { try { FileInputStream fis = new FileInputStream(in); // do file content handling here } finally { fis.close(); } // do parsing here } 

This, of course, is bad in the sense that it will lead to the creation of new objects all the time, which can eventually eat a lot of resources. The good side of this is, of course, that the stream will be closed, even if the file processing code throws an exception.

Answer 3

It sounds like a typical file attachment, for example, "zip".

The usual way to handle this is to actually have a separate instance of InputStream for each nested logical stream. They will perform the necessary operations in the base phsycial stream, and buffering can be both on the base stream and in the logical stream, depending on what suits best. This means that the logical stream encapsulates all the placement information in the base stream.

You could have some kind of factory method that would have a signature like this:

 List<InputStream> getStreams(File inputFile) 

You can do the same with OutputStreams.

There are some details of this, but may this be enough for you?

Answer 4

In the general case, the code that opens the file should close the file - the parse () function should not close the input stream, since it has the highest arrogance to assume that the rest of the program will not want to continue reading other files contained in a large one.

You have to decide whether the interface for parse () should be only a stream and a length (with a function capable of assuming the file is correctly positioned) or whether the interface should include an offset (so the function first positions and then reads). Both options are possible. I would be inclined to let parse () do the positioning, but this is not a clear solution.

Answer 5

You can use a wrapper class in RandomAccessFile - try this

You can also try wrapping this in a BufferedInputStream and see if performance improves.