Reading binary in Java vs C ++

Question

I have a binary file (about 100 MB) that I need to quickly read. In C ++, I could just load the file into a char pointer and go through it, incrementing the pointer. This, of course, will be very fast.

Is there a relatively quick way to do this in Java?

Answer 1

If you use a memory-mapped file or a regular buffer, you can read data as fast as your equipment allows.

File tmp = File.createTempFile("deleteme", "bin"); tmp.deleteOnExit(); int size = 1024 * 1024 * 1024; long start0 = System.nanoTime(); FileChannel fc0 = new FileOutputStream(tmp).getChannel(); ByteBuffer bb = ByteBuffer.allocateDirect(32 * 1024).order(ByteOrder.nativeOrder()); for (int i = 0; i < size; i += bb.capacity()) { fc0.write(bb); bb.clear(); } long time0 = System.nanoTime() - start0; System.out.printf("Took %.3f ms to write %,d MB using ByteBuffer%n", time0 / 1e6, size / 1024 / 1024); long start = System.nanoTime(); FileChannel fc = new FileInputStream(tmp).getChannel(); MappedByteBuffer buffer = fc.map(FileChannel.MapMode.READ_ONLY, 0, size); LongBuffer longBuffer = buffer.order(ByteOrder.nativeOrder()).asLongBuffer(); long total = 0; // used to prevent a micro-optimisation. while (longBuffer.remaining() > 0) total += longBuffer.get(); fc.close(); long time = System.nanoTime() - start; System.out.printf("Took %.3f ms to read %,d MB MemoryMappedFile%n", time / 1e6, size / 1024 / 1024); long start2 = System.nanoTime(); FileChannel fc2 = new FileInputStream(tmp).getChannel(); bb.clear(); while (fc2.read(bb) > 0) { while (bb.remaining() > 0) total += bb.get(); bb.clear(); } fc2.close(); long time2 = System.nanoTime() - start2; System.out.printf("Took %.3f ms to read %,d MB File via NIO%n", time2 / 1e6, size / 1024 / 1024); 

prints

 Took 305.243 ms to write 1,024 MB using ByteBuffer Took 286.404 ms to read 1,024 MB MemoryMappedFile Took 155.598 ms to read 1,024 MB File via NIO 

This is 10 times larger for the file than you want. This is so fast because the data is cached in memory (and I have an SSD drive). If you have fast hardware, data can be read quickly.

Answer 2

Of course, you can use a memory mapped file.

Here are two good links with sample code:

• Thinking in Java: memory mapped files
• Java tips: how to create a memory mapped file

---

If you do not want to go along this route, just use a regular InputStream (for example, DataInputStream after transferring it to BufferedInputStream .

Answer 3

Most files do not need memory matching, but can simply be read with standard Java I / O, especially since your file is so small. A smart way to read these files is to use BufferedInputStream.

 InputStream in = new BufferedInputStream(new FileInputStream("somefile.ext")); 

Buffering is already optimized in Java for most computers. If you had a larger file, say, 100 MB, then you would look at optimizing it.

Answer 4

Take a look here at this blog post on how to read a binary into a byte array in Java:

http://www.spartanjava.com/2008/read-a-file-into-a-byte-array/

Copied from link:

 File file = new File("/somepath/myfile.ext"); FileInputStream is = new FileInputStream(file); // Get the size of the file long length = file.length(); if (length > Integer.MAX_VALUE) { throw new IOException("The file is too big"); } // Create the byte array to hold the data byte[] bytes = new byte[(int)length]; // Read in the bytes int offset = 0; int numRead = 0; while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) { offset += numRead; } // Ensure all the bytes have been read in if (offset < bytes.length) { throw new IOException("The file was not completely read: "+file.getName()); } // Close the input stream, all file contents are in the bytes variable is.close() 

Answer 5

Reading a file from disk will be the slowest part for kilometers, so it probably doesn't matter. Of course, from this individual operation, the JVM still takes ten years, so add this time.

Answer 6

Using the DataInputStream Java SDK may be useful here. DataInputStream provides functions such as readByte () or readChar (), if necessary. A simple example would be:

 DataInputStream dis = new DataInputStream(new FileInputStream("file.dat")); try { while(true) { byte b = dis.readByte(); //Do something with the byte } } catch (EOFException eofe) { //Stream Ended } catch (IOException ioe) { //Input exception } 

Hope this helps. Of course, you can read the entire stream into an array of bytes and scroll it as well ...