Using StreamWriter to implement logging and top removal

Question

My C # winforms 4.0 application uses a thread-safe stream block for internal and debug logging. When my application opens, it deletes the file and recreates it. When the application closes, it saves the file.

What I would like to do is change my application so that it is added instead of replacing. This is a simple fix.

However, here is my question:

I want to save an AROUND log file with a maximum of 10 megabytes. My limitation would be simple. When you go to close the file, if the file exceeds 10 megabytes, adjust the first 10%.

Is there a "better" way, and then do the following:

• Close file
• Check if there is a file <10 meg
• If so, open the file
• Parsing the whole thing
• Drop the first 10%
• Write the file back
• Close

Edit: well, I ended my own (shown below) proposal to move openly to Log4Net - this is good, but the time it takes to learn a new library and move all my journal entries (thousands) is not an effective time for a little improvement, which I tried to do.

private static void PerformFileTrim(string filename) { var FileSize = Convert.ToDecimal((new System.IO.FileInfo(filename)).Length); if (FileSize > 5000000) { var file = File.ReadAllLines(filename).ToList(); var AmountToCull = (int)(file.Count * 0.33); var trimmed = file.Skip(AmountToCull).ToList(); File.WriteAllLines(filename, trimmed); } } 

Answer 1

I researched this once and never came up with anything, but I can offer you plan B here:

I use the selection below to save a maximum of 3 log files. First, log file 1 is created and added. When it exceeds maxsize, log 2 and later logs 3 are created. When log 3 is too large, log 1 is deleted and the rest of the logs are flushed to the stack.

 string[] logFileList = Directory.GetFiles(Path.GetTempPath(), "add_all_*.log", SearchOption.TopDirectoryOnly); if (logFileList.Count() > 1) { Array.Sort(logFileList, 0, logFileList.Count()); } if (logFileList.Any()) { string currFilePath = logFileList.Last(); string[] dotSplit = currFilePath.Split('.'); string lastChars = dotSplit[0].Substring(dotSplit[0].Length - 3); ctr = Int32.Parse(lastChars); FileInfo f = new FileInfo(currFilePath); if (f.Length > MaxLogSize) { if (logFileList.Count() > MaxLogCount) { File.Delete(logFileList[0]); for (int i = 1; i < MaxLogCount + 1; i++) { Debug.WriteLine(string.Format("moving: {0} {1}", logFileList[i], logFileList[i - 1])); File.Move(logFileList[i], logFileList[i - 1]); // push older log files back, in order to pop new log on top } } else { ctr++; } } } 

Answer 2

This is obtained from bigtech's answer:

 private static string RollLogFile() { string path = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments); string appName = Path.GetFileNameWithoutExtension(Environment.GetCommandLineArgs()[0]); string wildLogName = string.Format("{0}*.log",appName); int fileCounter = 0; string[] logFileList = Directory.GetFiles(path, wildLogName, SearchOption.TopDirectoryOnly); if (logFileList.Length > 0) { Array.Sort(logFileList, 0, logFileList.Length); fileCounter = logFileList.Length - 1; //Make sure we apply the MaxLogCount (but only once to reduce the delay) if (logFileList.Length > MaxLogCount) { //Too many files - remove one and rename the others File.Delete(logFileList[0]); for (int i = 1; i < logFileList.Length; i++) { File.Move(logFileList[i], logFileList[i - 1]); } --fileCounter; } string currFilePath = logFileList[fileCounter]; FileInfo f = new FileInfo(currFilePath); if (f.Length < MaxLogSize) { //still room in the current file return currFilePath; } else { //need another filename ++fileCounter; } } return string.Format("{0}{1}{2}{3:00}.log", path, Path.DirectorySeparatorChar, appName, fileCounter); } 

Using:

 string logFileName = RollLogFile(); using (StreamWriter sw = new StreamWriter(logFileName, true)) { sw.AutoFlush = true; sw.WriteLine(string.Format("{0:u} {1}", DateTime.Now, message)); } 

Answer 3

The solutions here did not work for me. I took user3902302's answer, which was again based on Bigtech's answer and wrote the full class. In addition, I DO NOT use StreamWriter, you can change one line (AppendAllText versus peer StreamWrite).

There is little error handling (for example, retrying access on failure, although blocking should catch all internal concurrent access).

This may be enough for some people who previously had to use such a large solution as log4net or nlog. (And the log4net RollingAppender is not even thread safe, it is one. :))

 public class RollingLogger { readonly static string LOG_FILE = @"c:\temp\logfile.log"; readonly static int MaxRolledLogCount = 3; readonly static int MaxLogSize = 1024; // 1 * 1024 * 1024; <- small value for testing that it works, you can try yourself, and then use a reasonable size, like 1M-10M public static void LogMessage(string msg) { lock (LOG_FILE) // lock is optional, but.. should this ever be called by multiple threads, it is safer { RollLogFile(LOG_FILE); File.AppendAllText(LOG_FILE, msg + Environment.NewLine, Encoding.UTF8); } } private static void RollLogFile(string logFilePath) { try { var length = new FileInfo(logFilePath).Length; if (length > MaxLogSize) { var path = Path.GetDirectoryName(logFilePath); var wildLogName = Path.GetFileNameWithoutExtension(logFilePath) + "*" + Path.GetExtension(logFilePath); var bareLogFilePath = Path.Combine(path, Path.GetFileNameWithoutExtension(logFilePath)); string[] logFileList = Directory.GetFiles(path, wildLogName, SearchOption.TopDirectoryOnly); if (logFileList.Length > 0) { // only take files like logfilename.log and logfilename.0.log, so there also can be a maximum of 10 additional rolled files (0..9) var rolledLogFileList = logFileList.Where(fileName => fileName.Length == (logFilePath.Length + 2)).ToArray(); Array.Sort(rolledLogFileList, 0, rolledLogFileList.Length); if (rolledLogFileList.Length >= MaxRolledLogCount) { File.Delete(rolledLogFileList[MaxRolledLogCount - 1]); var list = rolledLogFileList.ToList(); list.RemoveAt(MaxRolledLogCount - 1); rolledLogFileList = list.ToArray(); } // move remaining rolled files for (int i = rolledLogFileList.Length; i > 0; --i) File.Move(rolledLogFileList[i - 1], bareLogFilePath + "." + i + Path.GetExtension(logFilePath)); var targetPath = bareLogFilePath + ".0" + Path.GetExtension(logFilePath); // move original file File.Move(logFilePath, targetPath); } } } catch (Exception ex) { System.Diagnostics.Debug.WriteLine(ex.ToString()); } } } 

change
Since I just noticed that you asked a slightly different question: should your lines vary greatly in size, this will be a variation (which in 90% of cases will not improve compared to yours, although it can be very slightly faster, a new unhandled error was also introduced (\ n is not present)):

  private static void PerformFileTrim(string filename) { var fileSize = (new System.IO.FileInfo(filename)).Length; if (fileSize > 5000000) { var text = File.ReadAllText(filename); var amountToCull = (int)(text.Length * 0.33); amountToCull = text.IndexOf('\n', amountToCull); var trimmedText = text.Substring(amountToCull + 1); File.WriteAllText(filename, trimmedText); } } 

Answer 4

This feature allows you to rotate the journal on weekdays. The first time our application starts on Monday, any existing entry for Monday will be checked, if it has not already been initialized today, it will discard old entries and reinitialize a new file. Throughout the day, the file will continue to add text to the same log file.

Thus, 7 log files will be created. debug-Mon.txt, debog-Tue.txt ...

it will also add the name of the method that actually registered the message along with the date. very useful for general use.

 private void log(string text) { string dd = DateTime.Now.ToString("yyyy-MM-dd"); string mm = DateTime.Now.ToString("ddd"); if (File.Exists("debug-" + mm + ".txt")) { String contents = File.ReadAllText("debug-" + mm + ".txt"); if (!contents.Contains("Date: " + dd)) { File.Delete("debug-" + mm + ".txt"); } } File.AppendAllText("debug-" + mm + ".txt", "\r\nDate: " + DateTime.Now.ToString("yyyy-MM-dd HH:mm:s") + " =>\t" + new System.Diagnostics.StackFrame(1, true).GetMethod().Name + "\t" + text); } 

Answer 5

I was looking through win32 api, and I'm not even sure that this can be done using my own win32 vfs calls, no matter through .Net.

The only solution I would like to use would be to use memory mapped files and manually transfer data that apparently supports .Net 4.0.

Memory mapped files