Geek Answers Handbook

Add a timestamp to ElasticSearch using Elasticsearch-py using Bulk-API

I am trying to add a timestamp to my data, add an elasticsearch-py index, and then display the data using kibana.

My data appears in the kiban, but my timestamp is not used. When I go to the "Opening" tab after setting up my index template, I get 0 results (yes, I tried to set the search time).

Here is what my json index looks like:

{'index': 
         {'_timestamp': u'2015-08-11 14:18:26', 
          '_type': 'webapp_fingerprint', 
          '_id': u'webapp_id_redacted_2015_08_13_12_39_34',
          '_index': 'webapp_index'
         }
}

****JSON DATA HERE***

This will be accepted by elasticsearch and will be imported into Kibana, but the _timestamp field will not actually be indexed (it appears in the drop-down list when setting up the index template in the "Time field name" field).

I also tried formatting metaFields as follows:

{'index': {
           '_type': 'webapp_fingerprint', 
           '_id': u'webapp_id_redacted_2015_08_13_12_50_04', 
           '_index': 'webapp_index'
          }, 
           'source': {
                      '_timestamp': {
                                     'path': u'2015-08-11 14:18:26',
                                     'enabled': True, 
                                     'format': 'YYYY-MM-DD HH:mm:ss'
                                    }
                     }
}

This also does not work.

, _timestamp , elasticsearch.

{'index': {
           '_timestamp': {
                          'path': u'2015-08-11 14:18:26', 
                          'enabled': True, 
                          'format': 'YYYY-MM-DD HH:mm:ss'
                         }, 
           '_type': 'webapp_fingerprint', 
           '_id': u'webapp_id_redacted_2015_08_13_12_55_53', 
           '_index': 'webapp_index'
          }
}

:

elasticsearch.exceptions.TransportError: 
TransportError(500,u'IllegalArgumentException[Malformed action/metadata 
line [1], expected a simple value for field [_timestamp] but found [START_OBJECT]]')

, - , . , . , . .

+4
1

​​. , , .

request_body = {
    "settings" : {
        "number_of_shards": 1,
        "number_of_replicas": 0
    },
    "mappings" : {
        "_default_":{
            "_timestamp":{
                 "enabled":"true",
                 "store":"true",
                 "path":"plugins.time_stamp.string",
                 "format":"yyyy-MM-dd HH:m:ss"
             }
         }
    }
}
print("creating '%s' index..." % (index_name))
res = es.indices.create(index = index_name, body = request_body)
print(" response: '%s'" % (res))
+5

More articles:


All Articles