Geek Answers Handbook

Logstash Grok Filter Apache Access Log

I looked through here and there, but could not find a working permit. I am trying to use Grok Filter inside a Logstash configuration file to filter the Apache-Access log file. The log message is as follows: {"message":"00.00.0.000 - - [dd/mm/YYYY:hh:mm:ii +0000] \"GET /index.html HTTP/1.1\" 200 00"}.

At this point, I could only filter the client ip using grok {match => ["message", "% {IP: client_ip}"]}.

I want to filter:

 - The GET method, - requested page (index.html), - HTTP/1.1\, - server response 200 - the last number 00 after 200 inside the message body

Please note: none of them work for me:

  • grok {match => {"message" => "% {COMBINEDAPACHELOG}"}} or
  • grok {match => ["message", "% {COMBINEDAPACHELOG}"]}
+6
source share
3 answers
 grok { match => [ "message", "%{IP:client_ip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:apache_timestamp}\] \"%{WORD:method} /%{NOTSPACE:request_page} HTTP/%{NUMBER:http_version}\" %{NUMBER:server_response} " ] }
+17
source

Use the Grok debugger to get an exact match in your log format. This is the only way.

http://grokdebug.herokuapp.com/

+14
source

Use the following:

 filter { grok { match => { "message" => "%{COMMONAPACHELOG}" } } }

As you can see from your template, COMBINEDAPACHELOG will fail because there are some missing components:

 COMBINEDAPACHELOG %{COMMONAPACHELOG} %{QS:referrer} %{QS:agent}

https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns

0
source

More articles:


All Articles