Error starting HiveMQ node after upgrading to 4.5

HiveMQ Commercial Offerings
1

Our cluster is running 4.2.0 and I’m trying to upgrade to 4.5. A direct upgrade on a node resulted in the following errors when starting:

2021-01-15 08:35:18,655 INFO  - Cluster join process is still ongoing. Please do not shut down HiveMQ.
2021-01-15 08:35:19,266 ERROR - Exception on ClientSessionPutAllRequest_4_3
java.lang.StringIndexOutOfBoundsException: String index out of range: -3
        at java.base/java.lang.String.substring(String.java:1841)
        at gf.k.a(Unknown Source)
        at gf.j.a(Unknown Source)
        at gf.h.a(Unknown Source)
        at gf.h.a(Unknown Source)
        at gf.h.a(Unknown Source)
        at fV.d.a(Unknown Source)
        at fV.d.doTask(Unknown Source)
        at v.v.a(Unknown Source)
        at v.x$b.run(Unknown Source)
        at eD.v$b.run(Unknown Source)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

And:

2021-01-15 08:35:19,662 ERROR - Could not join cluster. Shutting down HiveMQ.
java.util.concurrent.ExecutionException: n.g: Request: ReplicationFinishedRequest, Receiver: H6R7k.
        at com.google.common.util.concurrent.AbstractFuture.getDoneValue(Unknown Source)
        at com.google.common.util.concurrent.AbstractFuture.get(Unknown Source)
        at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(Unknown Source)
        at com.hivemq.HiveMQServer.e(Unknown Source)
        at com.hivemq.HiveMQServer.b(Unknown Source)
        at com.hivemq.HiveMQServer.a(Unknown Source)
        at com.hivemq.HiveMQServer.main(Unknown Source)
Caused by: n.g: Request: ReplicationFinishedRequest, Receiver: H6R7k.
        at ad.j.c(Unknown Source)
        at Z.g$a.onFailure(Unknown Source)
        at com.google.common.util.concurrent.Futures$CallbackListener.run(Unknown Source)
        at com.google.common.util.concurrent.DirectExecutor.execute(Unknown Source)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(Unknown Source)
        at com.google.common.util.concurrent.AbstractFuture.complete(Unknown Source)
        at com.google.common.util.concurrent.AbstractFuture.setException(Unknown Source)
        at com.google.common.util.concurrent.SettableFuture.setException(Unknown Source)
        at r.d.a(Unknown Source)
        at r.d.a(Unknown Source)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
        at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
        at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
        at com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(Unknown Source)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-01-15 08:35:19,669 INFO  - Shutting down HiveMQ. Please wait, this could take a while...

Upgrading a node from 4.2.0 to 4.4.3 worked fine. I tried going to 4.4.3 first and then to 4.5 but same error messages as above.

2

Hi @anders,

It is correct that you cannot directly upgrade vom HiveMQ 4.2.x to HiveMQ 4.5.x.
Make sure to always upgrade a maximum of 1 minor version. (i.e. 4.4.x to 4.5.)

I tried going to 4.4.3 first and then to 4.5 but same error messages as above.

How many cluster nodes do you have?
Is it possible that one of the nodes was still running on HiveMQ 4.2.x or 4.3.x when you saw the error?

Regards,
Florian

2 Likes
3

Hi, we have 3 nodes in the cluster, and the two others are running 4.2.0, I can try upgrading the two others to 4.4.3 first and then try one up to 4.5

4

HI @anders ,

After upgrading all nodes to 4.4.x, you can upgrade the cluster to 4.5.0.

Regards,
Florian