When autoscaling group remove my ec2 instance, how to keep data persistence by EBS?

HiveMQ Commercial Offerings
1

Hello,
when you are using AWS ASG (autoscaling group) with launch_template, by this way, the new created EC2 instances will attach to individual new EBS volume. If data persistence is required, how can I keep the EBS data?
For example, when a instance is terminated by ASG, the data will loss if EBS data is not persistent.
(The reason I use EC2 cluster because HiveMQ recommends so)

Thanks !

2

Hi yifan,

Please be welcome in our HiveMQ community!

Could you please give us more detail regarding what you are trying to achieve?

  1. For what are you planning to store the data?
  2. Are you running a HiveMQ CE or a cluster? For example, when you are running a cluster and one node is removed, the remaining nodes will hold the state and redistribute accordingly.

Thanks,
Dasha from HiveMQ Team

3

Hello Daria, thanks for replying.

  1. my goal is to save messages for Subscriber in case EC2 instance is down. For example, when a Publisher sends a message to MQTT broker, before the message is forward to the Subscribers, if HiveMQ instance is down/stop, Autoscaling Group may spawn a new EC2 instance with new EBS storage, then, the message is lost. I don’t know how to keep data persistent, without using a external database.
  2. I think I am running HiveMQ CE, because I use Autoscaling Group, and run the image AMI “ami-0be6cbb57a7373d3e” (HiveMQ 4.7)
    do you think I made a mistake here? should I run ASG in order to create a cluster?
    I followed this page: How to install HiveMQ with Terraform
    and I am using Terrafrom as well.
    if you have some example please share with me, thanks you!!
4

Hi yifan,

Thank you for sharing the details! HiveMQ CE (open source version) does not support clustering. Clustering feature is available with a Professional and Enterprise Editions (paid offerings).

In general, in a cluster the state is distributed between the nodes, so when a broker node is lost, the messages/queues etc. are still available (up until -1 nodes being lost at the same time).

When you are adopting a Professional or Enterprise edition, our Professional Services might be able to help on the topic of deploying HiveMQ on AWS with Autoscaling Groups.

I am sorry I could not help you more and provide a ready solution for you.

Kind regards,
Dasha from HiveMQ Team

5

Hi Daria, just want to confirm, if I want to follow this tutorial HiveMQ Cluster :: HiveMQ Documentation, then I can’t use this image “ami-0be6cbb57a7373d3e” (HiveMQ 4.7) right?
Can I have a free trail option for Professional or Enterprise version to test cluster feature?

6

Hi Yifan,

sorry for the misunderstanding and thank you for clarification! Yes, you can deploy a HiveMQ 4.7.x trial instance on AWS EC2 by using our pre-built AMIs (‘trial’ is limited to max 25 client connections etc.)

AWS installation is covered in detail in this article: Building an elastic high availability MQTT broker cluster on AWS
HiveMQ AMI-s for different regions: HiveMQ Installation using Amazon Webservices. Does you AMI come from this list (unfortunately I cannot see the AMI with your ID).

Please let me know if this helps with your original question?
Thanks,
Dasha from HiveMQ Team