Thread safe StreamWriter C # how to do this? 2

Question

So this is a continuation of my last question. So the question was “What is the best way to create a program that is thread safe in the sense that it needs to write double values ​​to a file. If a function that stores values ​​through a streamwriter is called by multiple threads? What is the best way to do this ??”

And I changed the code found on MSDN, how about the following? This correctly writes everything to a file.

namespace SafeThread { class Program { static void Main() { Threading threader = new Threading(); AutoResetEvent autoEvent = new AutoResetEvent(false); Thread regularThread = new Thread(new ThreadStart(threader.ThreadMethod)); regularThread.Start(); ThreadPool.QueueUserWorkItem(new WaitCallback(threader.WorkMethod), autoEvent); // Wait for foreground thread to end. regularThread.Join(); // Wait for background thread to end. autoEvent.WaitOne(); } } class Threading { List<double> Values = new List<double>(); static readonly Object locker = new Object(); StreamWriter writer = new StreamWriter("file"); static int bulkCount = 0; static int bulkSize = 100000; public void ThreadMethod() { lock (locker) { while (bulkCount < bulkSize) Values.Add(bulkCount++); } bulkCount = 0; } public void WorkMethod(object stateInfo) { lock (locker) { foreach (double V in Values) { writer.WriteLine(V); writer.Flush(); } } // Signal that this thread is finished. ((AutoResetEvent)stateInfo).Set(); } } } 

Answer 1

Thread and QueueUserWorkItem are the lowest available APIs for threads. I would not use them if I finally had no other choice. Try the Task class for a much higher level of abstraction. For more information, see my recent blog post on this topic .

You can also use BlockingCollection<double> as the correct producer / consumer queue instead of trying to manually create it with the smallest API available for synchronization.

Reinventing these wheels correctly is surprisingly difficult. I highly recommend using classes designed for this type of need (e.g. Task and BlockingCollection ). They are built into the .NET 4.0 platform and are available as add-ons for .NET 3.5 .

Answer 2

• the code has the author as a var instance, but uses a static locker. If you had multiple instances writing different files, there is no reason why they would need to use the same lock.
• in the corresponding note, since you already have a writer (as a private instance of var), you can use this to lock instead of using a separate lock object in this case - this makes things a little easier.

The “correct answer” really depends on what you are looking for in terms of blocking / blocking. For example, it’s easiest to skip the intermediate data structure, just have the WriteValues ​​method so that each stream reports its results and writes them to a file. Something like:

 StreamWriter writer = new StreamWriter("file"); public void WriteValues(IEnumerable<double> values) { lock (writer) { foreach (var d in values) { writer.WriteLine(d); } writer.Flush(); } } 

Of course, this means that workflows are serialized during the “report results” phases depending on the performance characteristics, which can be just good (5 minutes to create, for example, 500 ms for recording).

At the other end of the spectrum, you must write workflows to the data structure. If you're in .NET 4, I would recommend using ConcurrentQueue instead of doing it yourself.

In addition, you can make input / output files in larger batches than those reported by worker threads, so you can just write in the background thread at some frequency. This end of the spectrum looks something like this: you have to remove the Console.WriteLine calls in real code, they are just there so you can see how it works in action)

 public class ThreadSafeFileBuffer<T> : IDisposable { private readonly StreamWriter m_writer; private readonly ConcurrentQueue<T> m_buffer = new ConcurrentQueue<T>(); private readonly Timer m_timer; public ThreadSafeFileBuffer(string filePath, int flushPeriodInSeconds = 5) { m_writer = new StreamWriter(filePath); var flushPeriod = TimeSpan.FromSeconds(flushPeriodInSeconds); m_timer = new Timer(FlushBuffer, null, flushPeriod, flushPeriod); } public void AddResult(T result) { m_buffer.Enqueue(result); Console.WriteLine("Buffer is up to {0} elements", m_buffer.Count); } public void Dispose() { Console.WriteLine("Turning off timer"); m_timer.Dispose(); Console.WriteLine("Flushing final buffer output"); FlushBuffer(); // flush anything left over in the buffer Console.WriteLine("Closing file"); m_writer.Dispose(); } /// <summary> /// Since this is only done by one thread at a time (almost always the background flush thread, but one time via Dispose), no need to lock /// </summary> /// <param name="unused"></param> private void FlushBuffer(object unused = null) { T current; while (m_buffer.TryDequeue(out current)) { Console.WriteLine("Buffer is down to {0} elements", m_buffer.Count); m_writer.WriteLine(current); } m_writer.Flush(); } } class Program { static void Main(string[] args) { var tempFile = Path.GetTempFileName(); using (var resultsBuffer = new ThreadSafeFileBuffer<double>(tempFile)) { Parallel.For(0, 100, i => { // simulate some 'real work' by waiting for awhile var sleepTime = new Random().Next(10000); Console.WriteLine("Thread {0} doing work for {1} ms", Thread.CurrentThread.ManagedThreadId, sleepTime); Thread.Sleep(sleepTime); resultsBuffer.AddResult(Math.PI*i); }); } foreach (var resultLine in File.ReadAllLines(tempFile)) { Console.WriteLine("Line from result: {0}", resultLine); } } } 

Answer 3

So, you say you want the stream to write data to a single file using StreamWriter? Easy. Just lock the StreamWriter object.

5 threads will be created here. Each thread will perform 5 "actions", and at the end of each action it will write 5 lines to a file called "file".

 using System; using System.Collections.Generic; using System.IO; using System.Threading; namespace ConsoleApplication1 { class Program { static void Main() { StreamWriter Writer = new StreamWriter("file"); Action<int> ThreadProcedure = (i) => { // A thread may perform many actions and write out the result after each action // The outer loop here represents the multiple actions this thread will take for (int x = 0; x < 5; x++) { // Here is where the thread would generate the data for this action // Well simulate work time using a call to Sleep Thread.Sleep(1000); // After generating the data the thread needs to lock the Writer before using it. lock (Writer) { // Here we'll write a few lines to the Writer for (int y = 0; y < 5; y++) { Writer.WriteLine("Thread id = {0}; Action id = {1}; Line id = {2}", i, x, y); } } } }; //Now that we have a delegate for the thread code lets make a few instances List<IAsyncResult> AsyncResultList = new List<IAsyncResult>(); for (int w = 0; w < 5; w++) { AsyncResultList.Add(ThreadProcedure.BeginInvoke(w, null, null)); } // Wait for all threads to complete foreach (IAsyncResult r in AsyncResultList) { r.AsyncWaitHandle.WaitOne(); } // Flush/Close the writer so all data goes to disk Writer.Flush(); Writer.Close(); } } } 

The result should be a file "file" with 125 lines in it with all the "actions" performed simultaneously, and the result of each action written synchronously with the file.

Answer 4

The code that you have there is finely broken - in particular, if the first work item of the queue is executed first, it will immediately clear the (empty) list of values ​​before terminating, after which your worker will go over and fill the list (which will eventually be ignored) . The auto-reset event also does nothing, because nothing ever requests or waits for its state.

Also, since each thread uses a different lock, the locks do not matter! When accessing the streaming browser, you must provide a single lock. You do not need a lock between the flush code and the generation code; you just need to make sure that the flash starts after the generation is complete.

You are probably on the right track, although I would use a fixed-size array instead of a list and discard all entries from the array when it is full. This avoids running out of memory if the thread is long-lived.