Paho MQTT client behavior when broker / client time is disconnected

Question

I have several QoS2 level messages that cause problems in a scenario when a broker or MQTT client has problems. These problems may include

• the client begins to see server timeouts
• the client has lost contact with the broker (Internet connection down, problem with the broker, ....) for a while and will connect again.

Typically, when an MQTT client begins to receive timeouts or other errors from the broker, the message is stored in a persistent storage (messages in flight) and will eventually be reissued.

However, in the event that the Paho client loses connection with the broker, messages will no longer be considered in flight and will not be stored by Paho. At this point, it seems that the application becomes responsible for maintaining these messages (outside of paho) and re-publishing them.

Am I saying correctly that when the MQTT broker becomes unavailable, the Paho MQTT client cannot help me guarantee that these QoS2 messages will be resent?

So, how can I make a distinction between the following case where client.publish threw a MqttException where Paho did not save the message flow.

Client is currently disconnecting (32102) at org.eclipse.paho.client.mqttv3.internal.ClientComms.shutdownConnection(ClientComms.java:297) at org.eclipse.paho.client.mqttv3.internal.CommsSender.handleRunException(CommsSender.java:154) at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:131) at java.lang.Thread.run(Thread.java:745) 

And the next in which it is preserved continues the light

 Timed out waiting for a response from the server (32000) at org.eclipse.paho.client.mqttv3.internal.Token.waitForCompletion(Token.java:94) at org.eclipse.paho.client.mqttv3.MqttToken.waitForCompletion(MqttToken.java:50) at org.eclipse.paho.client.mqttv3.MqttClient.publish(MqttClient.java:315) at org.eclipse.paho.client.mqttv3.MqttClient.publish(MqttClient.java:307) 

Obviously, I can also start doing bookkeeping and save all the inappropriate messages separately, but then I can get duplicates of the QoS level (messages received by both Paho and myself reissued).

How to program the client?

• Do I need to do my own persistence with Paho based on exception codes?
• Do I need to take into account the connectionLost callback and assume that from now on Paho will not be saved for me until the MQTT client recovers?
• Before publishing, I need to check if the client is connected correctly, and assuming that Paho will continue the message?

Here are some exceptions and behavior of Paho's perseverance

• Connection lost (32109): message saved by paho
• The client is currently disconnecting (32102): the message is lost paho
• Server response timeout (32000): message is saved by paho
• Client not connected (32104): message lost paho

What are some of the best practices here with Paho?

Answer 1

I did not design the Java client, but I see how this happens and what it can be confusing. I assume that we are talking about a synchronous client here? And that all of these exceptions occur when publish is called?

Basic principles:

• if the client API is not connected (including disconnection) at the time of sending the message, then no attempt is made to send the message, and it is not saved.
• If the client API is currently connected, the message is sent, and it is saved before that. The connection may fail before the end of the QoS 2 exchange, in which the message will be retried when the client reconnects.

but both of these situations are reported as exceptions, which is useless.

In client C, I follow the rules:

• if the message is saved, the result is returned, regardless of whether the QoS 2 exchange is completed
• connectLost callback is called if the client disconnects

so that you know that if you get an error message, you must repeat the publication call.

For the Java client, we could improve the situation:

• only throws an exception if the message is not saved

or

2. create another exception class that clearly showed when the message was and wasn’t saved, something like

   • MqttException superclass - message is not saved
   • MqttIncompleteException - The message has been saved.

Please note that in June we are planning "offline buffering" for release 1.2, which means that the message can be saved when the client is not connected.

Answer 2

Of course, offline buffering will benefit, at least when using the android client. I reused sqlite from the paho android service which save messages when the service is left alone to save messages when the client is disconnected with far good results. I don’t know what the future solution will be when considering a wider platform solution, maybe just allow messages to be stored in the permanent file / memory storage in QOS2 when the client is disconnected? Not sure how this will affect when a large message queue will accumulate and should be sent.