BufferedInputStream in bytes [] to send via Socket to the database

Question

I was looking for the answer to this question, but actually could not find anything. Earlier today, I asked how to create a file in a string through an array of bytes, and then go back to search later.

What I was told was that I just had to store an array of bytes to avoid annoying encoding problems. So now I started working on it, but now I hit the wall.

Basically, I used unbuffered streams before to turn a file into an array of bytes. This works well theoretically, but it takes up a lot of memory, which will ultimately lead to a heap size exception. Instead, I have to use buffered streams (or, as I was told), and the problem I am having right now comes from BufferedInputStream to byte []. I tried to copy and use the methods found in this documentation

http://docs.guava-libraries.googlecode.com/git/javadoc/index.html?com/google/common/io/package-summary.html

Where I exchange unbuffered streams for buffered streams. The only problem is that I cannot directly convert the stream of buffered output to an array of bytes, as I can, with an unbuffered stream.

Help?:)

import java.io.BufferedInputStream; import java.io.BufferedOutputStream; import java.io.ByteArrayOutputStream; import java.io.IOException; public final class BufferedByteStream { private static final int BUF_SIZE = 1024000; public static long copy(BufferedInputStream from, BufferedOutputStream to) throws IOException { byte[] buf = new byte[BUF_SIZE]; long total = 0; while(true) { int r = from.read(buf); if(r == -1) { break; } to.write(buf, 0, r); total += r; } return total; } public static byte[] toByteArray(BufferedInputStream in) throws IOException { BufferedOutputStream out = new BufferedOutputStream(new ByteArrayOutputStream()); copy(in, out); return out. // <--- Problem is here } } 

EDIT:

I am still getting heap errors. So I will now post all the code:

main.java

 import java.io.*; import java.util.logging.Level; import java.util.logging.Logger; import jserver.io.BufferedByteStream; /** * * @author Vipar */ public class main { public static void main(String[] args) { File f = new File("<doesn't matter>"); try { byte[] buf; try (BufferedInputStream bis = new BufferedInputStream(new FileInputStream(f))) { buf = BufferedByteStream.toByteArray(bis); bis.close(); } File f2 = new File("<doesn't matter>"); try (FileOutputStream fos = new FileOutputStream(f2)) { fos.write(buf); fos.close(); } } catch (FileNotFoundException ex) { Logger.getLogger(main.class.getName()).log(Level.SEVERE, null, ex); } catch (IOException ex) { Logger.getLogger(main.class.getName()).log(Level.SEVERE, null, ex); } } } 

BufferedByteStream.java

 import java.io.BufferedInputStream; import java.io.BufferedOutputStream; import java.io.ByteArrayOutputStream; import java.io.IOException; public final class BufferedByteStream { private static final int BUF_SIZE = 1024000; public static long copy(BufferedInputStream from, BufferedOutputStream to) throws IOException { byte[] buf = new byte[BUF_SIZE]; long total = 0; while(true) { int r = from.read(buf); if(r == -1) { break; } to.write(buf, 0, r); total += r; } return total; } public static byte[] toByteArray(BufferedInputStream in) throws IOException { ByteArrayOutputStream bytesOut = new ByteArrayOutputStream(); BufferedOutputStream out = new BufferedOutputStream(bytesOut); copy(in, out); return bytesOut.toByteArray(); } } 

Answer 1

Take a look at ByteArrayOutputStream: Java 7 API java.io.ByteArrayOutputStream

 bytesOut = new ByteArrayOutputStream(); byte[] bytes = bytesOut.toByteArray(); 

Update: If you insist on what you are doing, you can simply assign the ByteArrayOutputStream variable to a variable and get an array like this:

 ByteArrayOutputStream bytesOut = new ByteArrayOutputStream() BufferedOutputStream out = new BufferedOutputStream(bytesOut); copy(in, out); return bytesOut.toByteArray(); 

Update 2: It seems that the real question is how to copy the file without first reading it in memory:

1) Manually:

  byte[] buff = new byte[64*1024]; //or some size, can try out different sizes for performance BufferedInputStream in = new BufferedInputStream(new FileInputStream("fromFile")); BufferedOutputStream out = new BufferedOutputStream(new FileoutputStream("toFile")); int n = 0; while ((n = in.read(buff)) >= 0) { out.write(buff, 0, n); } in.close(); out.close(); 

2) Effectively using the OS and without a loop, etc .:

 FileChannel from = new FileInputStream(sourceFile).getChannel(); FileChanngel to = new FileOutputStream(destFile).getChannel(); to.transferFrom(from, 0, from.size()); //or from.transferTo(0, from.size(), to); from.close(); to.close(); 

3) If you have Java 7, you can simplify closing exceptions and threads or just copy the file with the new APIs in java 7:

 java.nio.file.Files.copy(...); 

see java.nio.file.Files

Answer 2

DataFetcher is perfect for this:

http://tus.svn.sourceforge.net/viewvc/tus/tjacobs/io/DataFetcher.java?revision=34&view=markup

You can use clearBuffer to clear the buffer between readings if you run out of memory

You will also need the Timeout class - it is in the same project in the same package.