What is the most efficient logging method for applications

Question

I work in a client server application in which several clients and the server are socket-based for financial transactions, where performance is very important. I am currently using the system.IO namespace streamwriter to write logs to a file. For one transaction, I need to call the streamwriter method 50 times to record a different value and for more than 50,000 transactions, the time spent on this log becomes very important.

How can I reduce my application’s logging time? Do I need to choose some other approach or some other class instead of streamwriter? What would be the best way to log with less time.

Answer 1

If performance is key, I would consider Event Tracking for Windows (AKA ETW).

With .NET 4.5 and the implementation of the EventSource class , this has made ETW much easier to implement than in the past.

Vance Morrison Weblog contains good articles on this topic.

For an overview of the architecture, see Improving Debugging Tuning and Performance Using ETW .

There is also a semantic application unit from the Microsoft Templates and Practices group, which makes it easy to enable EventSource functions and manage log behavior.

Answer 2

I suggest you try Log4Net, you can configure where (file, database, xml) and when (bath, transaction, ...) and easily switch the trace level (debug, info, warning, ...)

Writing a journal system from scratch is not worth it.

Answer 3

Download the values ​​before writing them to disk.

Complete the recording only after the transaction is completed.

Something like that:

StringBuilder builder = new StringBuilder(); // do transaction step 1 builder.Append("Transaction step 1" + environment.NewLine); // or however you add a line to you log // step 2 builder.Append("Transaction step 2" + environment.NewLine); //... // step 50 builder.Append("Transaction step 50" + environment.NewLine); // now write to the file File.WriteAllText(@"C:\log.txt", builder.ToString()); 

You can add some processing if there is an error at any stage for logging.

You can also use some open source tool like log4net: http://logging.apache.org/log4net/ .

Answer 4

I suggest logging into the database (high performance, possibly embedded sqlite / sqlce). Bonus - you can structure and request journal entries.

Answer 5

To reduce the time spent on the magazine, I suggest:

• Make sure that data conversion requires minimal conversion / formatting.
• Create or use a logging library that:
  • When called, it puts registration data into the buffer (along with time, stream identifier, and other tags).
  • Periodically flushes buffered data to disk (i.e., immediately when the buffered data is large enough to fill at least one physical block in the log file or immediately when the system goes into standby mode or periodically every x seconds).
  • It blocks the log file for exclusive write access, so you can view it while the software is running, but other processes cannot block it underfoot.
  • Uses a separate thread for flushing processing, i.e. Does not slow down worker threads.
• If you have a lot of server processes, consider using IPC to send log data to a single point to minimize the number of active files that were written and the number of buffers used (you may need to test to see if it's worth it, and you may need to add tags to show the source of each entry).
• Schedule backups of the logs in standby or standby mode so that they do not become too large.

Answer 6

One way to improve the performance of your application and to be able to record all of these log messages is through the queue in the MSMQ queue, and the Windows application will process the log message queues when the server is not under load. You may have a queue on a completely separate server.

Implementation. You can configure the WCF web service, which uses MSMQ to process your log messages. This makes it easy to configure the Windows service.