How to set buffer size on BufferedWriter over FileWriter

Question

I ran into a problem with BufferedWriter when writing data to a single file with some streams.

I set the size of the BufferedWriter buffer, but no matter what number I set, it flushes the data to disk when the buffer is 8192 (the default buffer size), and not the size I set (here is 16384), is there a problem with my code?

This is how I create a BufferedWriter :

 new BufferedWriter(new FileWriter(fileName, true), 16384); 

This is the full code:

 import java.io.BufferedWriter; import java.io.File; import java.io.FileWriter; import java.io.IOException; public class Test1 { public static void main(String[] args) throws IOException { for(int i =0;i<10;i++){ MyThread r = new MyThread(); Thread t = new Thread(r); t.start(); } } } class MyThread implements Runnable { public void run() { String s = "{addffffffkkkljlkj2015dd}\n"; BufferedWriter bw = null; try { bw = new BufferedWriter(new FileWriter( "/Users/liaoliuqing/Downloads/1.txt", true),16384); } catch (IOException e) { e.printStackTrace(); } for(int i =0 ; i<1000; i++){ try { bw.write(String.format("%03d", i)+s); //bw.flush(); } catch (IOException e) { e.printStackTrace(); } } } } 

Answer 1

I solve the problem using OutputStream and not writer, here is the code:

 bw = new BufferedOutputStream( new FileOutputStream(new File("/Users/liaoliuqing/Downloads/1.txt"),true),165537); 

Answer 2

FileWriter actually uses its own 1024 byte fixed-size buffer. BufferedWriter , on the other hand, shows that it uses a buffer size of 8192 bytes (the default), which can be configured by the user to any other desired size.

And for further troubled waters, the implementation of Java 6 OutputStreamWriter actually delegates to StreamEncoder , which uses its own buffer with a default size of 8192 bytes. The StreamEncoder buffer is StreamEncoder configurable, although there is no way to access it directly through the included OutputStreamWriter.

Answer 3

Is there a problem with my code?

Some. Basically: possible I / O and concurrency errors. The file buffer size can be less dangerous (and one that you cannot handle efficiently).

• Trying to open an already open file . All your streams are trying to write to a single file ( 1.txt ). This can be a problem. FileWriter documentation says:

  Some platforms, in particular, allow you to open a file for writing with only one FileWriter (or another file). In such situations, the constructors of this class will not work if the file is already open.

• Lines can be cut and blended . If you have multiple streams with corresponding buffers at one point, on the same output, you may not even need strange race conditions or streams stopped to the right of the middle or a write operation to see that your result is damaged.

  As a solution to me (if your threads should have the same output), you can use a shared object with synchronized access to take care of the actual recording. I applied SafeAppender in my example, but there are probably better alternatives there.

• Without cleaning and closing buffers mean (tail) of your data will be lost (for example, tears in the rain). Finally, a block is usually good to take care of this.

• Also, as indicated by other users, BufferedWriter buffer size does not affect the buffer size in FileOutputStream (and therefore FileWriter ). And it looks like the java.io and java.nio APIs do not offer any way to handle this. If you look at the sources of the Java library, you may notice that the size of the BufferedWriter buffer simply means the number of characters that you store before writing to the delegate output. The default size (8192) is optimal for most cases, and increasing it may mean more problems (potentially losing more data) than benefits.

This is my code if it serves you:

 // http://stackoverflow.com/questions/32451526/how-to-set-the-buffer-size-on-a-bufferedwriter-over-a-filewriter public class TestWriter { public static class SafeAppender { private BufferedWriter bw; private int users = 0; public SafeAppender(File f) throws IOException { bw = new BufferedWriter(new FileWriter(f)); } public synchronized void append(String s) throws IOException { bw.write(s); } public synchronized void incrUsers() { users ++; } public synchronized void decrUsers() { if (--users <= 0) { try { bw.flush(); System.err.println("INFO-appender-flush()"); } catch (Throwable whatever) { /* log-if-you-care*/} } } // Might be called by GC, or not @Override protected void finalize() throws Throwable { try { bw.close(); System.err.println("INFO-appender-close()"); } catch (Throwable whatever) { /* log-if-you-care */} super.finalize(); } } private static class MyRunnable implements Runnable { final static String S = "{addffffffkkkljlkj2015dd}"; SafeAppender appender; String threadId; public MyRunnable (SafeAppender a, String tid) { appender = a; threadId = tid; } public void run() { appender.incrUsers(); try { for(int i =0 ; i<1000; i++){ // NOTE: Not a good idea to printStackTrace if each line fails. Let thread fail String line = String.format("%s-%03d-%s\n", threadId, i, S); appender.append(line); } } catch (IOException e) { System.err.printf("ERROR-%s-%s\n", threadId, e.toString()); } finally { appender.decrUsers(); } } } public static void main(String[] args) { try { File f = File.createTempFile("TestWriter", ".txt"); System.err.printf("INFO-main-Writing into %s\n", f.getCanonicalPath()); SafeAppender appender = new SafeAppender (f); for(int i =0;i<10;i++){ MyRunnable r = new MyRunnable(appender, ""+i); Thread t = new Thread(r); t.start(); } } catch (Throwable e) { e.printStackTrace(System.err); } } } 

Answer 4

What you see is not the size of the BufferedWriter buffer, but the size of the buffer used inside FileWriter. Quoting from the Java documentation ( http://docs.oracle.com/javase/7/docs/api/java/io/FileWriter.html )

The constructors of this class assume that the default character encoding and byte byte size are acceptable by default. To determine these values ​​yourself, create an OutputStreamWriter in FileOutputStream.

So, if you want to have fine grain control when the data is actually written to disk, you must create an instance of BufferedWriter as

 bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(new File('my_file.txt),true)));