OpenShift Aggregated Logging: Parse Apache Access Log

Question

When using OpenShift Aggregated Logging I get logs that are well loaded in elasticsearch. However, the line registered by apache ends in the message field.

I would like to create queries in Kibana where I can access the URL, status code and other fields separately. To do this, you need to perform a special analysis of the Apache access log.

How can i do this?

This is an example record, as shown in kiban:

 { "_index": "42-steinbruchsteiner-staging.3af0bedd-eebc-11e6-af4b-005056a62fa6.2017.03.29", "_type": "fluentd", "_id": "AVsY3aSK190OXhxv4GIF", "_score": null, "_source": { "time": "2017-03-29T07:00:25.595959397Z", "docker_container_id": "9f4fa85a626d2f5197f0028c05e8e42271db7a4c674cc145204b67b6578f3378", "kubernetes_namespace_name": "42-steinbruchsteiner-staging", "kubernetes_pod_id": "56c61b65-0b0e-11e7-82e9-005056a62fa6", "kubernetes_pod_name": "php-app-3-weice", "kubernetes_container_name": "php-app", "kubernetes_labels_deployment": "php-app-3", "kubernetes_labels_deploymentconfig": "php-app", "kubernetes_labels_name": "php-app", "kubernetes_host": "itsrv1564.esrv.local", "kubernetes_namespace_id": "3af0bedd-eebc-11e6-af4b-005056a62fa6", "hostname": "itsrv1564.esrv.local", "message": "10.1.3.1 - - [29/Mar/2017:01:59:21 +0200] "GET /kwf/status/health HTTP/1.1" 200 2 "-" "Go-http-client/1.1"\n", "version": "1.3.0" }, "fields": { "time": [ 1490770825595 ] }, "sort": [ 1490770825595 ] } 

Answer 1

Disclaimer I have not tested this in openshift. I do not know what technical stack you use for your microservice.

Here's how I do it in a spring boot application (with a log) deployed to Kubernetes.

1. Use a logarithmic log for the log (this will record logs in Json format, which is more suitable for ELK)

I have a gradle dependency to enable this

 compile "net.logstash.logback:logstash-logback-encoder:3.5" 

Then configure LogstashEncoder as an encoder in the application, in logback- spring.groovy / logback- spring.xml (or logabck.xml)

2. Have multiple filters or libraries to write access logs

For 2. Either use

A. Use "net.rakugakibox.springbootext: spring-boot-ext-logback-access: 1.6" library

(This is what I use)

It gives in good json format as shown below.

 { "@timestamp":"2017-03-29T09:43:09.536-05:00", "@version":1, "@message":"0:0:0:0:0:0:0:1 - - [2017-03-29T09:43:09.536-05:00] \"GET /orders/v1/items/42 HTTP/1.1\" 200 991", "@fields.method":"GET", "@fields.protocol":"HTTP/1.1", "@fields.status_code":200, "@fields.requested_url":"GET /orders/v1/items/42 HTTP/1.1", "@fields.requested_uri":"/orders/v1/items/42", "@fields.remote_host":"0:0:0:0:0:0:0:1", "@fields.HOSTNAME":"0:0:0:0:0:0:0:1", "@fields.content_length":991, "@fields.elapsed_time":48, "HOSTNAME":"ABCD" } 

OR

B. Use Tee Filter Log

OR

C. spring CommonsRequestLoggingFilter (not really verified)

Add bean definition

  @Bean public CommonsRequestLoggingFilter requestLoggingFilter() { CommonsRequestLoggingFilter crlf = new CommonsRequestLoggingFilter(); crlf.setIncludeClientInfo(true); crlf.setIncludeQueryString(true); crlf.setIncludePayload(true); return crlf; } 

Then set org.springframework.web.filter.CommonsRequestLoggingFilter to DEBUG, this can be done using application.properties by adding:

 logging.level.org.springframework.web.filter.CommonsRequestLoggingFilter=DEBUG