Overwriting of topic

1

Hi,

We have requirement where we have to send data for single topic for multiple times.
Details:

  1. Send data/message on a topic
  2. Again Send data/message on same topic, then it should not overwrite the data/message sent in step 1.
  3. Data/message sent in step 1 and 2 should be intact.

In HiveMQ topic message data is overwritten if message sent multiple times, means latest sent message is available for subscriber.
Please assist on this ASAP.

Regards,
Girish

2

Hi Girish,

Welcome to the world of MQTT!

I am not sure where this statement comes from, perhaps you could describe your issue in more detail?

In HiveMQ topic message data is overwritten if message sent multiple times, means latest sent message is available for subscriber.

In the MQTT, to put it simple, you have:

  • a publisher (publishes the message to a topic),
  • a broker (passes the message to the subscribers of the topic) and
  • a subscriber (once it subscribes, it will be receiving all messages for the topic).

Kind regards,
Dasha from HiveMQ team

3

Your statement: " * a subscriber (once it subscribes, it will be receiving all messages for the topic)."
– subscriber will receive all message for the topic but for same topic previous message will be overwritten with new message, is it right?

4

Hi @girishgsv ,

No it is not.

As I have pointed out, in MQTT specification there is no concept of “overwriting” messages. Hence my question to you, where this information comes from.

For your reference MQTT specification 3.1.1: MQTT Version 3.1.1 and MQTT specification 5: MQTT Version 5.0

We wrote this series to bring anybody up to speed with MQTT without requiring to read the whole MQTT specification. It explains the core of MQTT concepts, its features and other essential information.

I hope this helps,
Dasha from HiveMQ team

5

Hi @girishgsv ,

Perhaps you are talking about RETAINED messages?

Please confirm,
Thanks
Dasha from HiveMQ team

6

I am not talking about retained message. I am talking about non-retained messages.
example:

  1. Send an message “Test” on topic “A”
  2. Send another message “Test1” on topic “A”
    In this case subscriber will receive latest message “Test1” sent on topic “A”
7

Hi @girishgsv,

the behaviour you describe is the mechanism of retained messages.

From the spec:

If the RETAIN flag is set to 1 in a PUBLISH packet sent by a Client to a Server, the Server MUST replace any existing retained message for this topic and store the Application Message [MQTT-3.3.1-5], so that it can be delivered to future subscribers whose subscriptions match its Topic Name.

And with non-retained aka “normal” messages what you describe is not possible at least with MQTT.

For normal messages we queue them for existing subscribers (if no qos 0 is used). So taking your example I would queue messages and when client can receive messages he will get “Test” then “Test1”, but only when the client was already subscribed before the messages where sent.

So I would say for your initial issue shouldn’t normal messages suffice as they are queued and not overwritten (like retained messages are)?

Greetings,
Michael

1 Like
8

Ok, Thanks. This answers my query.
One more query regarding Queueing in MQTT

  1. Can I clean all the topics from Queue at a time.
  2. How the messages/topics are deleted form Queue.
9

You can’t delete them manually. But you have some options to keep the queue slim.

  1. message expiry: either set by client (MQTT 5 only) or by HiveMQ (also for MQTT 3), if the message expiry of a messages has passed the message will be deleted from the queue.
  2. queue limit: It’s possible to set a queue limit for a session (by default it’s 1000 for clients), you can set if with config.xml which is applied to all clients or you can set it on client basis via the HiveMQ extension sdk.
  3. delete/overwrite session of the subscriber: as a queue is part of the session of a client if you delete the session you delete the whole queue (but you also lose all other session information, like subscriptions).

A message is deleted from the queue when:

  • the message was sent out to the subscriber
  • the message is expired

NOTE: Each client has it’s own queue.

Greetings,
Michael

1 Like
10

We have specific requirement that,
If multiple subscribers are subscribed to single topic, in this case how to decide that when to delete the topic/message, Because there is possibility that some subscribers have received the message but some are still not. e.g. assume that there are 10 subscribers that have subscribed to single topic, 5 of them received the message, eventually broker went down and there are still 5 subscribers remaining to receive the message. If we use message expiry then it might possible that remaining 5 subscribers never receive the message. How to handle these cases in HiveMQ.
And same is possible with Queue limit.