Geek Answers Handbook

Query data from Apache Flink

I want to upgrade from a home streaming server to Apache Flink. One thing we have is Apache Storm, such as a DRPC interface to run requests against the state stored in the processing topology.

So, for example: I have a set of sensors on which I run a moving average. I want to run a query by topology and return all sensors where this average value exceeds a fixed value.

Is there an equivalent in Flink, or, if not, a better way to achieve equivalent functionality?

+4
source share
2 answers

Flink Flink does not currently have a solution to query the internal status of operations. However, you are lucky because there are two solutions: we made an example of a word-counting example that allows you to query the state. It is available here: https://github.com/dataArtisans/query-window-example

For one of the upcoming versions of Flink, we are also working on a common solution for the use case used by the user. This will query the status of any internal operation.

In addition, it may also be enough to simply periodically output values ​​to something like Elasticsearch using the Window operation. Then the results can simply be requested from Elasticsearch.

+4
source

More articles:


All Articles