We would like to keep track of all the major events in our systems. For example, when the database can save the current status of the user, the event log should record all changes to this status along with when the changes occurred.
The event logging object should be as close as possible to the zero overhead for the event thrower, should contain structured information (as opposed to text log messages), and should support distributed deployment (many mailboxes throw a lot of events).
In a previous life, we had a UDP-based system that worked well because we had excellent control over the system (packet loss was minimized). Even throws will knock down UDP packets that will be caught and logged on other boxes. I'm looking for something similar, hopefully open source, off the shelf and deploying to more general networks. Also, I am open to suggestions on how to build something like this.
This should work in several languages, but it is primarily intended for Java and Python. Applications participating in the reception (events) will vary; some of them will be web applications, others will be package-oriented applications. The results are likely to live in Hadoop / HDFS / HBase.
source
share