Creating an Apache Storm sink that emits tuples every X seconds

Question

I have a topology that receives data from the MQTT broker, and I want the spout to behave like this:

• Retrieve a batch of tuples (or a list of rows in one tuple) every x seconds. How do I achieve this? I read a little about Storm Trident, but it IBatchSpoutdoesn’t seem to allow me to allocate tuples in batch mode with a specific time interval.

• What should the nozzle do if there is no new data? It cannot block the thread as it is the main Storm thread, right?

Answer 1

MQTT-. , MongoSpout.

nextTuple.

, Storm , Spout . , , . nextTuple, ack fail . , , (, ), .

, nextTuple, .

private static final EMISSION_PERIOD = 2000; // 2 seconds
private long lastEmission;

@Override
public void nextTuple() {
    if (lastEmission == null ||
            lastEmission + EMISSION_PERIOD >= System.currentMillis()) {
        List<Object> tuple = pollMQTT();
        if (tuple != null) {
            this.collector.emit(tuple);
            return;
        }
    }
    Utils.sleep(50);
}

, MQTT spout. , .

Answer 2

, Storm MQTT. .