Fluentd for Logstash Output Plugin

Question

I am trying to read from the scribe server using flunetd and display these logs which will be stored in logstash. I know that it’s very foolish to write scribe_central logs to another central registrar, but we need this to be done in our current architecture.

Does anyone know if there is any plugin for this? I searched Google but could not find.

Answer 1

For Scribe ↔ Fluentd there is a fluent-plugin-scribe :

There are several options for Fluentd ↔ Logstash:

• Use Redis in the middle and use fluent-plugin-redis and input_redis on the Logstash side. So it will be Fluentd -> Redis -> Logstash. This is what Logstash recommends with Logstash senders anyway.
• Alternatively, you can use the Fluentd out_forward plugin with Logstash TCP input. Logstash has a Fluentd codec to handle input coming from Fluentd.

Answer 2

You can redirect it directly to your tcp logstash input :)

I wrote a flunetd output plugin to send fluentd events to a common receiver using a secure tcp connection (can be configured for non-fixed ones as well).

fluent-plugin-loomsystems

To add the plugin to your fluentd agent, use the following command:

gem install fluent-plugin-loomsystems 

also see the full original answer , good luck.

Answer 3

I worked with http output from fluentd and http input for logstash, below the configuration fragments:

fluentd:

 <match **> @type http endpoint_url http://logstash-box:8080/ http_method put serializer json raise_on_error false </match> 

Logstash:

 input { http { port => 8080 } } 

Discussion: You may need to install the gem install fluent-plugin-out-http