Hivemq high cpu usage

Hi everyone, im running Hivemq on amazon ec2 on container which has 1 cpu, 475 mb ram, 8.65 gb disk space. there i have 2 connections and total inbound publish messages are 319 and total outbound publish messages are 313. problem is that cpu is always on 100%, ram 150-200/475mb and dis space 3.70/8.65. can anyone help me identify why cpu always run on 100%?

im using my authentication extension

public class CustomAuthenticatorService implements SimpleAuthenticator {

private static final Logger logger = LoggerFactory.getLogger(CustomAuthenticatorService.class);
private final HttpClient httpClient = HttpClient.newHttpClient();

@Override
public void onConnect(@NotNull SimpleAuthInput simpleAuthInput, @NotNull SimpleAuthOutput simpleAuthOutput) {
    try {
        ConnectPacket connectPacket = simpleAuthInput.getConnectPacket();
        String clientId = connectPacket.getClientId();
        logger.info("Started authentication of device: " + clientId);

        if (connectPacket.getPassword().isEmpty()) {
            System.out.println("Authentication failed: invalid client credentials " + clientId);
            simpleAuthOutput.failAuthentication("Invalid Client credentials");
            return;
        }
        ByteBuffer readOnlyBuffer = connectPacket.getPassword().get().asReadOnlyBuffer();
        CharBuffer charBuffer = StandardCharsets.US_ASCII.decode(readOnlyBuffer);
        String text = charBuffer.toString();
        WashingDeviceAuthParam washingDeviceAuthParam = new WashingDeviceAuthParam(text, clientId);
        String jsonPayload = washingDeviceAuthParam.toJson();

        HttpRequest httpRequest = HttpRequest
                .newBuilder(URI.create("autentication endpoint"))
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(jsonPayload))
                .build();

        HttpResponse<String> httpResponse = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString());

        if (httpResponse.statusCode() >= 200 && httpResponse.statusCode() <= 299 && httpResponse.body().equals("1")) {
            logger.info("Client successfully authenticated: " + clientId);
            simpleAuthOutput.authenticateSuccessfully();
        } else {
            logger.info("Client failed authentication: " + clientId);
            simpleAuthOutput.failAuthentication("Invalid credentials");
        }

    } catch (Exception exception) {
        logger.info("Client failed authentication: ");
        logger.info("Exception during authentication: " + exception);
        simpleAuthOutput.failAuthentication("Error during authentication");
    }
}

}

Hi @lichvayne

Your high CPU usage is likely caused by the synchronous HTTP request in your authentication extension. Here’s what’s happening:

  1. Blocking HTTP Call in Authentication

    • The httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString()); call is blocking, meaning the authentication thread waits until the HTTP response is received.
    • If the authentication endpoint is slow, under high load, or has network issues, this causes delays and spikes in CPU usage.
    • Multiple clients trying to authenticate at the same time will worsen the issue since HiveMQ will use multiple threads waiting for these responses.
  2. CPU Resource Constraints

    • Your instance only has 1 CPU and 475MB RAM, which is very limited for running HiveMQ + authentication logic.
    • The combination of HiveMQ handling MQTT messages and blocking HTTP requests leads to CPU saturation.

:wrench: Solutions to Fix High CPU Usage

1. Use Asynchronous HTTP Requests

Instead of blocking the authentication thread, use sendAsync():

httpClient.sendAsync(httpRequest, HttpResponse.BodyHandlers.ofString())
    .thenAccept(httpResponse -> {
        if (httpResponse.statusCode() >= 200 && httpResponse.statusCode() <= 299 && httpResponse.body().equals("1")) {
            logger.info("Client successfully authenticated: " + clientId);
            simpleAuthOutput.authenticateSuccessfully();
        } else {
            logger.info("Client failed authentication: " + clientId);
            simpleAuthOutput.failAuthentication("Invalid credentials");
        }
    })
    .exceptionally(exception -> {
        logger.info("Exception during authentication: " + exception);
        simpleAuthOutput.failAuthentication("Error during authentication");
        return null;
    });

:white_check_mark: This prevents HiveMQ authentication threads from being blocked.

2. Use a Local Cache to Reduce HTTP Requests

If the authentication response doesn’t change often, cache successful authentications using a ConcurrentHashMap:

private final Map<String, Boolean> authCache = new ConcurrentHashMap<>();

@Override
public void onConnect(@NotNull SimpleAuthInput simpleAuthInput, @NotNull SimpleAuthOutput simpleAuthOutput) {
    ConnectPacket connectPacket = simpleAuthInput.getConnectPacket();
    String clientId = connectPacket.getClientId();

    if (authCache.containsKey(clientId) && authCache.get(clientId)) {
        logger.info("Client authenticated from cache: " + clientId);
        simpleAuthOutput.authenticateSuccessfully();
        return;
    }

    // Perform async authentication...
    httpClient.sendAsync(httpRequest, HttpResponse.BodyHandlers.ofString())
        .thenAccept(httpResponse -> {
            if (httpResponse.statusCode() >= 200 && httpResponse.body().equals("1")) {
                authCache.put(clientId, true);
                simpleAuthOutput.authenticateSuccessfully();
            } else {
                simpleAuthOutput.failAuthentication("Invalid credentials");
            }
        })
        .exceptionally(exception -> {
            simpleAuthOutput.failAuthentication("Error during authentication");
            return null;
        });
}

:white_check_mark: This avoids repeatedly hitting the authentication endpoint.

3. Upgrade Your EC2 Instance

  • 1 CPU & 475MB RAM is too low for HiveMQ.
    Consider upgrading to at least:
    • 2 vCPUs
    • 2GB RAM
    • 20GB Disk
  • If you have a high number of clients connecting simultaneously, increasing resources will prevent performance bottlenecks.

Final Recommendation

  1. Change send() to sendAsync() :white_check_mark:
  2. Implement a local cache to reduce HTTP calls :white_check_mark:
  3. Upgrade your EC2 instance if possible :white_check_mark:

Let me know if you need further debugging!

Best,
Dasha from The HiveMQ Team