Geek Answers Handbook

How to view JSON logs of a managed virtual machine in Log Viewer?

I am trying to get JSON formatted logs on a Compute Engine VM instance to appear in the Google Developer Console Log Viewer. According to this documentation , it should be possible:

Applications using App Engine managed virtual machines must write their own log files to the VM log directory in / var / log / app _engine / custom_logs. These files are automatically collected and made available in the Viewer logs.

Custom log files must have the suffix .log or .log.json. If the suffix is ​​.log.json, the logs should be in JSON format with one JSON object per line. If the suffix is ​​-.log, log entries are treated as plain text.

This does not work for me: logs ending in .logare visible in the log viewer, but appear as plain text. Logs ending in .log.jsonare not displayed at all.

This also contradicts another recent article, which states that file names must end in .log, and its contents are treated as plain text .

As far as I can tell, Google uses fluentd to index log files in the log viewer. In the GitHub repository, I cannot find any evidence that indexes are .log.jsonindexed.

Does anyone know how to do this? Or the documentation is outdated, and for some reason this function has been deleted?

+4
2

JSON Logviewer :

JSON

JSON- , :

{
    "message": "Error occurred!.",
    "severity": "ERROR",
    "timestamp": {
        "seconds": 1437712034000,
        "nanos": 905
    }
}

( Google: https://code.google.com/p/googleappengine/issues/detail?id=11678#c5)

python-json-logger

: https://github.com/madzak/python-json-logger

def get_timestamp_dict(when=None):
    """Converts a datetime.datetime to integer milliseconds since the epoch.

    Requires special handling to preserve microseconds.

    Args:
        when:
            A datetime.datetime instance. If None, the timestamp for 'now'
            will be used.

    Returns:
    Integer time since the epoch in milliseconds. If the supplied 'when' is
    None, the return value will be None.
    """
    if when is None:
        when = datetime.datetime.utcnow()

    ms_since_epoch = float(time.mktime(when.utctimetuple()) * 1000.0)
    return {
        'seconds': int(ms_since_epoch),
        'nanos': int(when.microsecond / 1000.0),
    }

def setup_json_logger(suffix=''):
    try:

        from pythonjsonlogger import jsonlogger

        class GoogleJsonFormatter(jsonlogger.JsonFormatter):

            FORMAT_STRING = "{message}"

            def add_fields(self, log_record, record, message_dict):
                super(GoogleJsonFormatter, self).add_fields(log_record,
                                                            record,
                                                            message_dict)

                log_record['severity'] = record.levelname
                log_record['timestamp'] = get_timestamp_dict()
                log_record['message'] = self.FORMAT_STRING.format(
                                            message=record.message,
                                            filename=record.filename,
                                            )

        formatter = GoogleJsonFormatter()

        log_path = '/var/log/app_engine/custom_logs/worker'+suffix+'.log.json'
        make_sure_path_exists(log_path)
        file_handler = logging.FileHandler(log_path)

        file_handler.setFormatter(formatter)
        logging.getLogger().addHandler(file_handler)

    except OSError:
        logging.warn("Custom log path not found for production logging")

    except ImportError:
        logging.warn("JSON Formatting not available")

, setup_json_logger - worker .

+3

NodeJS, , Google Developer Console. /var/log/app _engine, . , , , , .log.

, ? , " Google" " "? !

0

More articles:


All Articles