How to send structured data through syslog?

Question

I have structured data, pairs of key values ​​that should be logged through syslog. In the end, we want to see statistics about these indicators. How should we encode and then decode it on the receiver side?

One parameter that we analyze part of the syslog log message and based on this parsing, insert it into the relational database table.

The second idea that we got is to send the data in JSON, and on the receiver side we process the table of the relational database as a job queue, the records must be analyzed before being inserted into a separate table.

In addition, key value pairs may vary depending on what we want to record.

Answer 1

RFC 5424 for the syslog protocol defines the STRUCTURED-DATA field:

The syslog message has the following ABNF definition [RFC5234]:

  SYSLOG-MSG = HEADER SP STRUCTURED-DATA [SP MSG]
     (...)
     STRUCTURED-DATA = NILVALUE / 1 * SD-ELEMENT
     SD-ELEMENT = "[" SD-ID * (SP SD-PARAM) "]"
     SD-PARAM = PARAM-NAME "="% d34 PARAM-VALUE% d34
     SD-ID = SD-NAME
     PARAM-NAME = SD-NAME
     PARAM-VALUE = UTF-8-STRING;  characters '"', '\' and
                                      ;  ']' MUST be escaped.
     SD-NAME = 1 * 32PRINTUSASCII
                       ;  except '=', SP, ']',% d34 (")
     (...)

( example here )

However, this RFC does not seem to be widely supported. You may need to develop your own protocol on top of the old RFC 3164 . The JSON encoded message component sounds like a very reasonable option.

If you have full control over the entire chain, you should evaluate the possibility of entering the target database.