Check count of connections

HiveMQ Community Edition
1

Hi,

I am using Apache JMeter to test 50000 connections concurrently to connect with HiveMQ Community Edition(2020.6) in server.

While checking connections count in server with below command:
netstat | grep :1883 | wc -l
it shows only 16351.
Can you please help me to know why it’s not accepting rest of the connections.

2

Hi @Amrinder,

my guess is that your machine doesn’t have enough ports available to establish the 50k connections.
You can view the range on your machine with:

sysctl net.ipv4.ip_local_port_range (linux)
sysctl net.inet.ip.portrange.first net.inet.ip.portrange.last (mac)

For example my laptop has this range to use for ephemeral ports:

49152  65535

As a connection from A to B has this tuple:

(local ip, local port, peer ip, peer port)

This would mean I could, with one local ip, make max. 16.383 connections to peerIp:peerPort. As the only variable is local port for which the system only gives me the port between 49152-65535.

If you have also similar numbers then I know two options:

  • adapt the system configuration to allow more ephemeral ports
  • or add multiple listeners for HiveMQ so that you have different peerPorts
    ** in my case I would need four listeners to connect 50k client to HiveMQ
  • (This is a guess) I’m not sure if JMeter can be configured to use more interfaces for establishing connections, so the local ip part could be variable to allow more connections.

Greetings,
Michael

3

Hi @michael_w,

Thanks for the help.
Our server has this range:
net.ipv4.ip_local_port_range = 32768 60999

Can you please provide suggestion for the given scenario.

1- How to establish multiple connections with single port as the data frequency is 10 sec and number of iot devices is 100000 with packet size of 500 bytes.

In the current situation we are using raw tcp connection and able to connect 300 iot devices with the single port. So we are expecting similar configuration with HiveMQ.
Please let us know if there is any flaw in our assumptions.

4

Hi @Amrinder,

so if I understand correctly you want to connect 100k clients to HiveMQ. As you only want to use one peer port (1883) this rules out the multiple listener approach.

Even if you configure ephemeral ports to use the full port range (0-65535) you still can only connect theoretically 65535 clients with one local IP. The solution is therefore to use more “local” IPs.

Now as I don’t use JMeter for performance/load testing I can only give you superficial help:

  • make JMeter use more IP’s that are used for the connections
  • Use the distributed approach, this allows you to use more machines for making the connections

Greetings,
Michael

5

Hi @michael_w

Thanks for the information. I am still not clear, if it is not possible to connect multiple IOT devices with a signal port (say 300 devices per port). We are using similar architecture in our present Raw TCP connection, where we are connecting around 400 IOT devices on a single port. So we are able to maintain connection of 10000 IOT devices in a single machine with 25 ports.

I would like to understand if we can use around 350 ports with 300 connections at each port in a single machine (or around 175 ports with 300 connections at each port in two machines) using HiveMQ. Also, it would be helpful if you can suggest some MQTT tool for load testing after deployment of the system.

6

Hi @Amrinder,

not sure I quite understand your problem. It should be no problem to establish 100k (or even a million) connections to one port. I will try to explain the problem you are currently have in detail (the solution for that I already wrote in the last response).

As I wrote before each TCP connection is made up of this tuple:

(local ip, local port, peer ip, peer port)

Lets say peerIp is 222.222.222.222 and peerPort is 1883 for HiveMQ (or any server). This mean peerIp and peerPort are static as you always have connect via this address:

(local ip, local port, 222.222.222.222, 1883)

Now we have only “local ip” and “local port” that can be varied to make connections to the server. Now lets further say JMeter (or any tool for making connection) uses the ip “111.111.111.111” you get this template:

(111.111.111.111, local port, 222.222.222.222, 1883)

The last variable part is local port and here we are limited by the fact that you can only have max 65536* (2^16) ports that you could put into “local port”, therefore as I wrote before with only one local ip you can at max establish 65536 connections:

(111.111.111.111, 0, 222.222.222.222, 1883) (connection tuple for client1)
(111.111.111.111, 1, 222.222.222.222, 1883) (connection tuple for client2)
                                    ......
(111.111.111.111, 65534, 222.222.222.222, 1883) (connection tuple for client65535)
(111.111.111.111, 65535, 222.222.222.222, 1883) (connection tuple for client65536)

*theoretically, not sure if all port can be used

As peerIp and peerPort are fix and localPort can only have max 65536 values you need to introduce variation into local ip (and that are my two points in the last response) two achieve 100k connections.

I / We are using an internal tool that I’m regrettably not allowed to hand out. But I can tell you this much, we scale the instances where the tool is running on to achieve millions of connections similar to the distributed approach of JMeter.

Greetings,
Michael

7

Hi @michael_w

Thanks for the help. It’s working now.

1 Like