Configurable confidential data masking through log4net

Question

I am considering using log4net as my logging framework for a new project starting soon. One of the problems that I encountered during prototyping, which I cannot find a definitive answer to, is how you can clear or mask the contents of a message in a custom and neat way.

Presumably, let me say that I want several cleaners to be put in place, but I also want to follow the principle of single responsibility. Some cleaner examples:

• Card Cleaner / PAN Cleaner
• Password cleaner
• Clearing personal data

I know that you should never record this information in plain text, and the code executing the logs will never consciously do this. I want to have the last level of protection, however, if the data is distorted and the confidential data somehow glides to where it should not; magazines are the worst case scenario.

Option 1:

I found this StackOverflow article that details a possible solution, but it involves using reflection. This is undesirable for performance, but also seems to be hacked for manipulating internal storage mechanisms. Editing-log4net-messages-before-they-reach-the-appenders

Option 2:

The proposed answer to the same question involves the use of PatternLayoutConverter. This is normal for a single cleanup operation, but you cannot use multiple operations such as:

public class CardNumberCleanerLayoutConverter : PatternLayoutConverter { protected override void Convert(TextWriter writer, LoggingEvent loggingEvent) { string message = loggingEvent.RenderedMessage; // TODO: Replace with real card number detection and masking. writer.Write(message.Replace("9", "*")); } } 

 <layout type="log4net.Layout.PatternLayout"> <converter> <name value="cleanedMessage" /> <type value="Log4NetPrototype.CardNumberCleanerLayoutConverter, Log4NetPrototype" /> </converter> <converter> <name value="cleanedMessage" /> <type value="Log4NetPrototype.PasswordCleanerLayoutConverter, Log4NetPrototype" /> </converter> <conversionPattern value="%cleanedMessage" /> </layout> 

In the event of a name clash, as shown above, the converter loaded last will be the one that will act. Using the above example, this means that passwords will be cleared, but not card numbers.

Option 3:

The third option I've tried is to use chained ForwarderAppender instances, but this quickly complicates the configuration, and I would not consider this an ideal solution. Since the LoggingEvent class has an immutable RenderedMessage property, we cannot change it without creating a new instance of the LoggingEvent class and passing it as shown below:

 public class CardNumberCleanerForwarder : ForwardingAppender { protected override void Append(LoggingEvent loggingEvent) { // TODO: Replace this with real card number detection and masking. string newMessage = loggingEvent.RenderedMessage.Replace("9", "*"); // What context data are we losing by doing this? LoggingEventData eventData = new LoggingEventData() { Domain = loggingEvent.Domain, Identity = loggingEvent.Identity, Level = loggingEvent.Level, LocationInfo = loggingEvent.LocationInformation, LoggerName = loggingEvent.LoggerName, ExceptionString = loggingEvent.GetExceptionString(), TimeStamp = loggingEvent.TimeStamp, Message = newMessage, Properties = loggingEvent.Properties, ThreadName = loggingEvent.ThreadName, UserName = loggingEvent.UserName }; base.Append(new LoggingEvent(eventData)); } } public class PasswordCleanerForwarder : ForwardingAppender { protected override void Append(LoggingEvent loggingEvent) { // TODO: Replace this with real password detection and masking. string newMessage = loggingEvent.RenderedMessage.Replace("4", "*"); // What context data are we losing by doing this? LoggingEventData eventData = new LoggingEventData() { Domain = loggingEvent.Domain, Identity = loggingEvent.Identity, Level = loggingEvent.Level, LocationInfo = loggingEvent.LocationInformation, LoggerName = loggingEvent.LoggerName, ExceptionString = loggingEvent.GetExceptionString(), TimeStamp = loggingEvent.TimeStamp, Message = newMessage, Properties = loggingEvent.Properties, ThreadName = loggingEvent.ThreadName, UserName = loggingEvent.UserName }; base.Append(new LoggingEvent(eventData)); } } 

Conformity (very difficult to follow):

 <log4net> <appender name="LocatedAsyncForwardingAppender" type="Log4NetPrototype.LocatedAsyncForwardingAppender, Log4NetPrototype"> <appender-ref ref="CardNumberCleanerForwarder" /> </appender> <appender name="CardNumberCleanerForwarder" type="Log4NetPrototype.CardNumberCleanerForwarder, Log4NetPrototype"> <appender-ref ref="PasswordCleanerForwarder" /> </appender> <appender name="PasswordCleanerForwarder" type="Log4NetPrototype.PasswordCleanerForwarder, Log4NetPrototype"> <appender-ref ref="LogFileAppender" /> </appender> <appender name="LogFileAppender" type="Log4NetPrototype.LogFileAppender, Log4NetPrototype"> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="%m" /> </layout> </appender> <root> <level value="DEBUG" /> <appender-ref ref="LocatedAsyncForwardingAppender" /> </root> </log4net> 

Does anyone have any other suggestion on how this can be implemented when theoretically the number of cleaners can be tuned due to performance?

Answer 1

In your question, you already say that you should turn to the case and not register confidential data. This can be provided with a fourth use case for code reviews and viewing the data that is logged. Your registration applications should not register any confidential data because the reason is a security risk. Trusting any code with filters, confidential data probably will not work if you make changes to your project. Your QA process must be really good to catch such errors (I have never seen a tester going through all the logs). Therefore, I would like to go to option 4, which ensures that you do not register this information in the first place.