Writing to FileOutputStream from multiple threads in Java

Question

Is it safe to call write on a Java FileOutputStream object form with multiple threads? Will the output be serialized correct?

clarification:

In my case, the class log contains a FileOutputStream link, and several threads can trigger a log entry that formats the output and calls a FileOutputStream record.

Should I synchronize my logging method to ensure that messages from multiple threads are not mixed?

Answer 1

A file cannot be opened more than once in write-mode , so there will be no answer.

After viewing your edit, yes, you should enter the synchronization in your log to make sure that the stream is available only one stream at a time. Just a suggestion, why don't you go for Log4J ? It already handles your use case.

Answer 2

Here is a simple implementation of a synchronized logger using java nio FileChannel. In this example, log messages are limited to 1024 bytes. You can adjust the length of the log message by changing the value of BUFFER_SIZE.

 import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.nio.ByteBuffer; import java.nio.channels.FileChannel; import java.util.HashMap; /** * The MyLogger class abstracts the writing of log messages to a file. * This is a synchronized implementation due to the usage of java.nio.channels.FileChannel * which is used to write log messages to the log file. * * The MyLogger class maintains a HashMap of MyLogger instances per log file. * The Key is the MD5 hash of the log file path and the Value is the MyLogger instance for that log file. * */ public final class MyLogger { private static final int BUFFER_SIZE = 1024; private static final int DIGEST_BASE_RADIX = 16; private static final String LINE_SEPARATOR = System.getProperty("line.separator"); private static HashMap<String, MyLogger> sLoggerMap; private FileChannel mLogOutputChannel; private ByteBuffer mByteBuffer; private String mLogDir; private String mLogFileName; /** * Private constructor which creates our log dir and log file if they do not already already exist. * If the log file exists, then it is opened in append mode. * * @param logDir * The dir where the log file resides * @param logFileName * The file name of the log file * @throws IOException * Thrown if the file could not be created or opened for writing. */ private MyLogger(String logDir, String logFileName) throws IOException { mLogDir = logDir; mLogFileName = logFileName; // create the log dir and log file if they do not exist FileOutputStream logFile; new File(mLogDir).mkdirs(); final String logFilePath = mLogDir + File.separatorChar + mLogFileName; final File f = new File(logFilePath); if(!f.exists()) { f.createNewFile(); } logFile = new FileOutputStream(logFilePath, true); // set up our output channel and byte buffer mLogOutputChannel = logFile.getChannel(); mByteBuffer = ByteBuffer.allocate(BUFFER_SIZE); } /** * Writes the given log message to the log file that is represented by this MyLogger instance. * If the log message could not be written to the log file an error is logged in the System log. * * @param logMessage * The log message to write to the log file. */ public void log(String logMessage) { // write the log message to the log file if (mLogOutputChannel != null) { mByteBuffer.put(logMessage.getBytes()); mByteBuffer.put(LINE_SEPARATOR.getBytes()); mByteBuffer.flip(); try { mLogOutputChannel.write(mByteBuffer); // ensure that the data we just wrote to the log file is pushed to the disk right away mLogOutputChannel.force(true); } catch (IOException e) { // Could not write to log file output channel e.printStackTrace(); return; } } if(mByteBuffer != null) { mByteBuffer.clear(); } } /** * Get an instance of the MyLogger for the given log file. Passing in the same logDir and logFileName will result in the same MyLogger instance being returned. * * @param logDir * The directory path where the log file resides. Cannot be empty or null. * @param logFileName * The name of the log file Cannot be empty or null. * @return The instance of the MyLogger representing the given log file. Null is returned if either logDir or logFilename is null or empty string. * @throws IOException * Thrown if the file could not be created or opened for writing. */ public static MyLogger getLog(String logDir, String logFileName) throws IOException { if(logDir == null || logFileName == null || logDir.isEmpty() || logFileName.isEmpty()) { return null; } if(sLoggerMap == null) { sLoggerMap = new HashMap<String, MyLogger>(); } final String logFilePathHash = getHash(logDir + File.separatorChar + logFileName); if(!sLoggerMap.containsKey(logFilePathHash)) { sLoggerMap.put(logFilePathHash, new MyLogger(logDir, logFileName)); } return sLoggerMap.get(logFilePathHash); } /** * Utility method for generating an MD5 hash from the given string. * * @param path * The file path to our log file * @return An MD5 hash of the log file path. If an MD5 hash could not be generated, the path string is returned. */ private static String getHash(String path) { try { final MessageDigest digest = MessageDigest.getInstance("MD5"); digest.update(path.getBytes()); return new BigInteger(digest.digest()).toString(DIGEST_BASE_RADIX); } catch (NoSuchAlgorithmException ex) { // this should never happen, but just to make sure return the path string return path; } } } 

Here's how you use it:

 MyLogger aLogger = MyLogger.getLog("/path/to/log/dir", "logFilename"); aLogger.log("my log message"); 

Answer 3

Not. Java does not support streaming to a single thread from multiple threads.

If you want to use streaming streams, check out this site: http://lifeinide.com/post/2011-05-25-threaded-iostreams-in-java/

It explains the situation well and has sample code for ThreadedOutputStream that will do what you want.

Answer 4

If you want to keep order (i.e. message 1 in the output stream got to message 2), you must block the stream. This in turn reduces concurrency. (All threads will be put on the lock / semaphore queue and wait for the thread to become available to them)

If you are only interested in writing a stream at the same time and do not care about the order, you can have buffers for each stream. Each stream is written to its own buffer. When the buffer is full, it receives a lock (which may include waiting for a lock) in the stream and empties its contents into the stream.

Edit: I just realized that if you care about the order and still want multithreading, if you also write time in the output stream in unix format (as long). After the stream is dumped onto some other container, the contents can be sorted by time, and you should have an ordered file.