# Should message fan in be avoided? If so, how?

**URL:** <https://community.hivemq.com/t/should-message-fan-in-be-avoided-if-so-how/1824>\
**Category:** HiveMQ Community Edition\
**Created:** [February 16, 2023, 12:16am UTC](https://community.hivemq.com/t/should-message-fan-in-be-avoided-if-so-how/1824 "2023-02-16T00:16:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![bshurn](https://avatars.discourse-cdn.com/v4/letter/b/57b2e6/32.png) [@bshurn](https://community.hivemq.com/u/bshurn)\
**Post date:** [February 16, 2023, 12:16am UTC](https://community.hivemq.com/t/should-message-fan-in-be-avoided-if-so-how/1824/1 "2023-02-16T00:16:44Z")

</div>

Hi all,

I am designing a solution to manage MQTT devices & have a question about message fan in – where a large number of clients (100K+) publish messages to a single topic tree.

Some background…  
In order for the solution to manage devices, it needs to know some information about each and every device. At startup each client sends a message which includes the information the solution needs for device management.

The question is… which topic should each client publish to?

**Option 1: Single topic tree**  
Clients publish to: clientinformation/[client-id]

Concern: Message fan-in.  
In this approach multiple clients are publishing to one solution side client subscribed to the wildcard topic ‘clientinformation/+’.

Having one client responsible for processing these messages raises concerns around latency and rate limiting. This also seems to give us no ability to scale horizontally.

Are these concerns valid? Are there well known solutions to this type of problem?

**Option 2: Partitioned topics**  
Assuming the concerns from Option #1 are valid, I am considering the alternative where each client is assigned a partition as part of the bootstrapping process (when they discover the broker endpoint).  
This partition information would be included in the topic tree to allow for concurrent processing.

Topic template: clientinformation/[partition]/[client-id]  
For example:  
ClientA publishes to clientinformation/1/clienta  
ClientB publishes to clientinformation/2/clientb  
ClientC publishes to clientinformation/N/clientc

The solution would then deploy N clients, each subscribed to a single partition:  
Solution1 subscribes to clientinformation/1/+  
Solution2 subscribes to clientinformation/2/+  
SolutionN subscribes to clientinformation/N/+

Concerns here are:

1. Is this a problem we need to solve, or is this strictly a theoretical concern
2. Are there well established solutions to this problem?
3. How we’ll reliably deliver this partition information to devices.
4. How or if we’ll need to repartition.

---

<div class="post-metadata">

**Author:** ![Daria\_H](https://avatars.discourse-cdn.com/v4/letter/d/dfb087/32.png) [@Daria\_H](https://community.hivemq.com/u/Daria_H)\
**Post date:** [February 16, 2023, 2:58pm UTC](https://community.hivemq.com/t/should-message-fan-in-be-avoided-if-so-how/1824/2 "2023-02-16T14:58:02Z")

</div>

Hi @bshurn ,

Thank you for your interest in MQTT, welcome to the community!

You can use [Shared Subscription](https://www.hivemq.com/docs/hivemq/4.11/user-guide/shared-subscriptions.html#concepts) to balance the load of a single subscription across multiple MQTT clients.

For Enterprise level solutions it is possible to stream messages from a [HiveMQ broker to Kafka](https://www.hivemq.com/solutions/technology/kafka-solution/).

I hope it helps.  
Kind regards,  
Dasha from HiveMQ team