Problems with the Sparkplug InfluxDB extension

HiveMQ Extensions
1

Hello, I have issues using the HiveMQ Sparkplug extension with InfluxDB. I’m following the instructions from here Sparkplug InfluxDB Extension for HiveMQ and i’m running this on a balena.io device with InfluxDB 2.71 and HiveMQ 4.12.0.

I was testing some changes and I got this error on the balenaCloud logs:

2023-10-02T14:43:46+02:00 hivemq4 2023-10-02 12:43:46,719 TRACE - No matching normal/shared subscriber or consumer found for PUBLISH with topic ‘spBv1.0/Sparkplug Devices/NBIRTH/Node-Red’
2023-10-02T14:44:01+02:00 influxdb 2023-10-02T12:44:01.085867Z info Unauthorized {“log_id”: “0kd6U5Zl000”, “error”: “session not found”}

However, after doing some changes testing the buckets and everything. Sent a new message and I don’t get any InfluxDB error or anything.

2023-10-02T15:04:43+02:00 hivemq4 2023-10-02 13:04:43,337 TRACE - Sending retained message with topic [spBv1.0/Sparkplug Devices/NCMD/Node-Red] for client [mqtt_164b1c5f195363a1]
2023-10-02T15:04:54+02:00 hivemq4 2023-10-02 13:04:54,380 TRACE - No matching normal/shared subscriber or consumer found for PUBLISH with topic ‘spBv1.0/My Devices/NCMD/EoN Name/My Device’
2023-10-02T15:04:55+02:00 hivemq4 2023-10-02 13:04:55,915 TRACE - No matching normal/shared subscriber or consumer found for PUBLISH with topic ‘spBv1.0/My Devices/NCMD/EoN Name/My Device’

This is the configuration of my sparkplug.properties file:

root@91f7174fea0c:/opt/hivemq-4.12.0/extensions/hivemq-sparkplug-extension# cat sparkplug.properties
# --
# Copyright 2021-present HiveMQ GmbH
# --

## influxdb properties
influxdb.mode:cloud
influxdb.host:localhost
influxdb.port:8086
influxdb.protocol:http
influxdb.auth:mytoken goes here==

influxdb.prefix:
influxdb.database:balena

influxdb.reportingInterval:1
influxdb.connectTimeout:5000

influxdb.tags:host=localhost

# InfluxDB cloud options
influxdb.bucket:balena
influxdb.organization:balena

## sparkplug properties

What am i missing? why i don’t see any error on the influxDB? on the other hand Influx UI tells me

Captura de pantalla 2023-10-02 a les 15.12.43

Could you please help me here?

2

Hi Marc,

Thank you for getting in touch. It’s fantastic to hear that you’re interested in MQTT and the HiveMQ broker. Welcome to the HiveMQ Community! We’re thrilled to have you join us.

In response to the log message you mentioned, No matching normal/shared subscriber or consumer found for PUBLISH with topic ‘spBv1.0/My Devices/NCMD/EoN Name/My Device,’ please be aware that this message simply indicates that there is currently no MQTT client subscribed to the topic spBv1.0/My Devices/NCMD/EoN Name/My Device. It is not related to InfluxDB issues. To verify this, you can subscribe an MQTT client to either the topic spBv1.0/My Devices/NCMD/EoN Name/My Device or spBv1.0/#. After doing so, you should observe that the message No matching normal/shared subscriber or consumer found no longer appears in the log.

Concerning the Sparkplug InfluxDB Extension for HiveMQ:

Your sparkplug.properties file appears to be valid, as I was able to successfully run the extension with your settings (using my own API token, of course).

To ascertain whether the Sparkplug InfluxDB Extension for HiveMQ has started successfully, please verify that the ‘hivemq.log’ contains the following messages:

INFO  - Starting HiveMQ extension system.
INFO  - Starting extension with id "hivemq-sparkplug-extension" at /Users/ds/hivemq/hivemq-4.12.0/extensions/hivemq-sparkplug-extension
INFO  - Creating InfluxDB Cloud sender for endpoint localhost, bucket something, organization something
INFO  - Extension "HiveMQ Sparkplug Extension" version 1.0.0-beta started successfully.

Presence of these messages in the log indicates that the extension has been successfully initiated.

In the event of any issues while writing data to the InfluxDB bucket, you may encounter warning messages in the log, as outlined below:

  • For a 404 error (Not Found):
WARN  - Unable to report to InfluxDB with error 'Server returned HTTP response code: 404 for URL: http://localhost:8086/api/v2/write?precision=s&org=something2&bucket=something with content :'Not Found''. Discarding data.
  • For a 401 error (Unauthorized):
WARN  - Unable to report to InfluxDB with error 'Server returned HTTP response code: 401 for URL: http://localhost:8086/api/v2/write?precision=s&org=something&bucket=something with content :'Unauthorized''. Discarding data.
  • If InfluxDB is not available:
WARN  - Unable to connect to InfluxDB. Discarding data.

The installation of the extension is straightforward, and to illustrate the process, I have created a brief video tutorial on the installation steps, which you can view here: Installation Video.

Please note that when you update the sparkplug.properties file, it is essential to Stop and then Start the extension again to ensure that it reads the updated settings from the sparkplug.properties file. To accomplish this, you need to DISABLE and then re-enable the extension as follows:

touch $HIVEMQ_HOME/extensions/hivemq-sparkplug-extension/DISABLED
sleep 10
rm $HIVEMQ_HOME/extensions/hivemq-sparkplug-extension/DISABLED

If you have any further questions or require additional assistance, please do not hesitate to ask. We are here to provide support.

Best regards,

Dasha from HiveMQ Team

1 Like
3

Thank you @Daria_H for your detailed answer.

Now what I get is

2023-10-06T13:47:14+02:00  influx  2023-10-06T11:47:14.114586Z  info    Unauthorized    {"log_id": "0kiCq4Ol000", "error": "session not found"}
2023-10-06T13:47:14+02:00  hivemq4  2023-10-06 11:47:14,127 WARN  - Unable to report to InfluxDB with error 'Server returned HTTP response code: 401 for URL: http://influx:8086/write?db=balena&precision=s with content :'Unauthorized''. Discarding data.
2023-10-06T13:47:19+02:00  hivemq4  2023-10-06 11:47:19,265 WARN  - Unable to report to InfluxDB with error 'Server returned HTTP response code: 401 for URL: http://influx:8086/write?db=balena&precision=s with content :'Unauthorized''. Discarding data.
2023-10-06T13:47:20+02:00  hivemq4  2023-10-06 11:47:20,311 WARN  - Unable to report to InfluxDB with error 'Server returned HTTP response code: 401 for URL: http://influx:8086/write?db=balena&precision=s with content :'Unauthorized''. Discarding data.
2023-10-06T13:47:24+02:00  hivemq4  2023-10-06 11:47:24,486 WARN  - Unable to report to InfluxDB with error 'Server returned HTTP response code: 401 for URL: http://influx:8086/write?db=balena&precision=s with content :'Unauthorized''. Discarding data.
4

Hi @marc-balena

It appears that the extension has been initiated and is operational. I encountered a similar issue when I was utilizing the previous API token. May I kindly inquire if you have verified that you are utilizing the accurate API token sourced from InfluxDB?

Best regards,
Dasha from HiveMQ Team

5

The main difference in my process is that i create the influxDB service with environment variables in the docker compose. It looks like it doesn’t like the new token i create afterwards.

6

Hi Marc,

I hope this message finds you well.

If you would be kind enough to provide us with the Docker command along with its associated parameters, as well as the sparkplug.properties file you are currently using, we would greatly appreciate it. By doing so, we can thoroughly review the situation and offer our assistance in identifying any potential issues.

Your cooperation in this matter would be immensely valuable. Thank you in advance.

Best regards,
Dasha

1 Like
7

Hello @Daria_H the code is here GitHub - mpous/hivemq-influxdb

you will see that InfluxDB environment variables are defined on the docker-compose and the HiveMQ is defined on its directory.

Let me know if you can build this!

8

@Daria_H it worked!

I changed the influxdb.mode to cloud instead to the default value http and it works!

Happy to get your feedback from the github repository that i shared here!

9

Once this extension works I see that if I change the topic and publish new data, i can’t access to this data on the Influx DB interface.

Do you have any experience with this @Daria_H ?

I’m using the repo I shared previously and sending data with NodeRED to the local HiveMQ. Thanks!

10

Hi @marc-balena

Thank you for reaching out with your concern. I understand that you’re encountering difficulties accessing data on the Influx DB interface after changing the topic and publishing new data with the setup from the repository you shared.

In order to provide you with the most accurate assistance, could you please provide more details on how you are reproducing this issue? If possible, providing exact step-by-step instructions or even recording a video could significantly help us in identifying and addressing the problem.

Your cooperation and additional information will allow us to offer a more precise solution. I appreciate your patience and look forward to resolving this issue for you.

Best regards,
Dasha from HiveMQ Team