How can I implement an OutputStream to rewind?

Question

After writing some processed content to the output stream, I need to go back to the beginning of the stream and write out some metadata for the content. The data that I write is very large, up to 4 GB, and can be written either directly to a file or to a memory buffer, depending on various environmental factors.

How can I implement an OutputStream that allows me to record headers after writing content?

Answer 1

Here is the output stream of a random access file.

Note that if you use it for a lot of streaming output, you can temporarily wrap it in a BufferedOutputStream to avoid a lot of small recordings (just make sure you dump it before you drop the shell or use the underlying stream directly).

import java.io.*; /** * A positionable file output stream. * <p> * Threading Design : [x] Single Threaded [ ] Threadsafe [ ] Immutable [ ] Isolated */ public class RandomFileOutputStream extends OutputStream { // ***************************************************************************** // INSTANCE PROPERTIES // ***************************************************************************** protected RandomAccessFile randomFile; // the random file to write to protected boolean sync; // whether to synchronize every write // ***************************************************************************** // INSTANCE CONSTRUCTION/INITIALIZATON/FINALIZATION, OPEN/CLOSE // ***************************************************************************** public RandomFileOutputStream(String fnm) throws IOException { this(fnm,false); } public RandomFileOutputStream(String fnm, boolean syn) throws IOException { this(new File(fnm),syn); } public RandomFileOutputStream(File fil) throws IOException { this(fil,false); } public RandomFileOutputStream(File fil, boolean syn) throws IOException { super(); File par; // parent file fil=fil.getAbsoluteFile(); if((par=fil.getParentFile())!=null) { IoUtil.createDir(par); } randomFile=new RandomAccessFile(fil,"rw"); sync=syn; } // ***************************************************************************** // INSTANCE METHODS - OUTPUT STREAM IMPLEMENTATION // ***************************************************************************** public void write(int val) throws IOException { randomFile.write(val); if(sync) { randomFile.getFD().sync(); } } public void write(byte[] val) throws IOException { randomFile.write(val); if(sync) { randomFile.getFD().sync(); } } public void write(byte[] val, int off, int len) throws IOException { randomFile.write(val,off,len); if(sync) { randomFile.getFD().sync(); } } public void flush() throws IOException { if(sync) { randomFile.getFD().sync(); } } public void close() throws IOException { randomFile.close(); } // ***************************************************************************** // INSTANCE METHODS - RANDOM ACCESS EXTENSIONS // ***************************************************************************** public long getFilePointer() throws IOException { return randomFile.getFilePointer(); } public void setFilePointer(long pos) throws IOException { randomFile.seek(pos); } public long getFileSize() throws IOException { return randomFile.length(); } public void setFileSize(long len) throws IOException { randomFile.setLength(len); } public FileDescriptor getFD() throws IOException { return randomFile.getFD(); } } // END PUBLIC CLASS 

Answer 2

If you know the size of the header, you can write an empty header first and then come back to fix it with the RandomAccessFile at the end. If you do not know the size of the header, then you have a fundamental situation when file systems usually do not allow data to be inserted. Therefore, you need to write to a temporary file, and then write the real file.

Answer 3

Lucene seems to have an implementation; and api look ok.

 getFilePointer() void seek(long pos) 

http://lucene.apache.org/java/1_4_3/api/org/apache/lucene/store/OutputStream.html

I think they complete RandomAccessFile

Answer 4

One way is to write the initial content to the memory buffer first, and then the headers to the "real" output stream, followed by the stream of buffered content, and from there just write to the unbuffered stream. It seems that the initial segment will not be so long as to make buffering reasonable. Regarding its implementation, you can use ByteArrayOutputStream to buffer, and then your class, OutputStream, takes a "real" output stream as an argument; and just switch between them as needed. You may need to extend the OutputStream API to determine what metadata to write as these triggers switch from buffered mode.

As mentioned in another answer, RandomAccessFile will also work, although it will not implement OutputStream.