Attaching a TTL field with every log sent via logstash in Elasticsearch

Question

Summary: I want to attach a TTL field with logs in logstash and send them to Elastic search.

I already looked through the documentation, but could not get it, because it is not very clear.

This is my config file in logstash.

input { stdin { type => "stdin-type" } } output { stdout { debug => true debug_format => "json"} elasticsearch {} } 

Now suppose that for every magazine I read, I want to add a TTL with it, say, 5 days.

I know how to activate the TTL option in the search for elasticity. But what changes I will have to make in the elastic search configuration files is not very clear to me. The documentation asks for the mappings folder, but it is not in the boot folder with the elastic directory.

We are looking for expert help.

Answer 1

Take a look here if you want to put the mapping in the file system. You have to go to the config folder and create a folder here called mappings , and another one with the name of the index inside mappings . Since logstash creates the default index per day, you'd better use the _default name for this folder so that the mapping applies to all indexes. The file created in this folder must have the name of the type to which you want to apply the mapping. I don’t remember exactly what type of logstash uses, so I would use the _default_ mapping definition. Just call the _default_.json file and put the following contents into it:

 { "_default_" : { "_ttl" : { "enabled" : true } } } 

As you can see, the type name should be displayed both in the file name and in its contents.

Otherwise, you may not put files on the file system. You can create an index template containing your custom mapping, for example the following:

 { "template" : "logstash-*", "mappings" : { "_default_" : { "_ttl" : { "enabled" : true } } } } 

The mapping will then be applied to all indices whose name matches the template template. If you use the _default_ mapping definition, the mapping will be the default for all types to be created.