Regarding packet loss from publisher to broker

Hello HiveMQ Community,

I am considering migrating from AWS IoT Core to HiveMQ and have a few queries:

  1. Suppose the broker becomes unreachable and a mobile application (acting as a publisher client) constantly receives packets from an IoT device at a rate of 1 message/second. When the publisher client tries to publish a message, it awaits a PUBACK from the broker. However, due to the broker’s unreachability due to internet connection issues, the message fails to get published. Let’s say the connection between the broker and the publisher client isn’t restored for 5 minutes (assuming the client’s session hasn’t expired due to a long keep-alive interval). Once the connection is re-established, how will the publisher handle the backlog of 5x60 messages? Are these messages stored somewhere in the interim? If so, where? Does the application layer need to make this decision? Also, is there something akin to a message queue inside the client library? I believe there might be, as this could explain how messages get published if the PUBACK packet isn’t received from the broker. If such a queue exists, what’s its capacity in terms of the maximum messages it can hold?

  2. Given an unstable internet connection on the application layer side, how can we ensure that all packets are successfully published to the broker?

Given that I’m using QoS level 1, does HiveMQ provide a solution for situations like this?

Thank you in advance for your guidance!

Hi @qwert477

It’s great to hear that you’re considering migrating to HiveMQ! Regarding your question about handling network failures, with MQTT, it’s crucial not to assume that the network is always reliable. Designing your application to handle network failures from the beginning is a best practice.

Automatic reconnection and offline buffering are key concepts in building resilient MQTT applications. However, it’s important to note that these features are solely implemented by the MQTT client based on the underlying MQTT client library, not the MQTT broker itself. HiveMQ does provide an open-source Java library called “hivemq-mqtt-client,” which can be used to build MQTT clients.

Remember to review the documentation and resources provided by HiveMQ for details on implementing these resilience features in your MQTT client application. Here’s a useful article on MQTT and resilience: MQTT and Resilience. Additionally, you can find examples of auto-reconnect and offline buffering in the HiveMQ Community forum: Example of Auto-Reconnect and Offline Buffering.

Building a robust MQTT application involves careful consideration of these resilience features to ensure that your IoT devices’ data is reliably and efficiently transmitted even in challenging network conditions. Best of luck with your migration to HiveMQ!

Best regards,
Dasha from HiveMQ Team

1 Like

Thank you, @Daria_H, for your response. I still have a question about the difference between your offline buffering implementation and Paho’s option of setting the maximum number of outgoing messages with QoS > 0 that can be in the outgoing message queue. Is there a distinction between the two, or are they essentially the same? Can you please confirm this? It’s important for my decision-making to understand these differences.

@AaronTLFranz

Hello @qwert477,

The hivemq-mqtt-client library does not include a max_queued_messages_set() function.

I trust this information is helpful.

Best regards.