Can we read data from special topic like dropped or dead in hivemq. If yes then How we can read and will there be any consequences by doing this?

HiveMQ Community Edition
21

Hi @michael_w ,

Apologies for distrubing you again.

Just need one clarification, Suppose one message went into the ‘$dead’ topic after that will this go into the ‘$expired’ once the message got expired ?

Thanks !

22

Hi @Rahul,

yes it is possible. But we prevent circular sending of messages, so $dead → $expired is possible but $dead → $expired → $dead is prevented.

Greetings,
Michael

2 Likes
23

Hi @michael_w ,

Got it.

I have more scenario which is as follow

  • $Dropped/#, $Expired/#, $Dead/# are the special topic. As per my understanding, these can have huge number of messages stored based on the load or due to un-attached subscriber.
  • Once we try to subscribe messages from these topics by creating a subscriber but that has some limitation of queuing ( default is 1000 ).
  • If we publishing message in these special topics due to any reason then how should we subscribe because at last we have some in-flight queue mechanism at broker. So once it reach to queuing limit again messages are getting dropped.

Working like : Message → $dropped/ → Subscriber ( it has default queue = 1000) → after some point again messages are getting dropped → $dropped/

It is becoming cyclic process. How we should handled this one ?

Could you have look on it ?

Thanks !

24

Hi @Rahul,

Message → $dropped/ → Subscriber ( it has default queue = 1000) → after some point again messages are getting dropped → $dropped/

in this case if they are dropped when queued for $dropped/ they are simply gone. This is because we prevent the cyclic behaviour internally.

How we should handled this one ?

  • You could use shared subscriptions (or consumer) to receive the messages from the special $ topics, they have the default queue limit of 500.000 (bonus: you can use multiple clients to process the messages)
  • You can increase the queue limit for your client that receives the messages from the special topics (this is done via an extension that has a Simple/EnhancedAuthenticator, see MQTT Client Authentication Options for Your HiveMQ Extension :: HiveMQ Documentation)

I personally would prefer shared subscriptions or a consumer to be used to process messages for $topics.

Greetings,
Michael

3 Likes
25

Hi @michael_w ,

Thanks a lot for sharing this info.