Sparkplug extension does not save Groov EPIC measurements

jsorensen · May 26, 2021, 6:05pm

I’m trying to get measurements from a Groov EPIC controller to load in an influx historian.
I have the hivemq_sparkplug_extension working but it does not save the measurements. It does however load what appear to be more than 60 metrics monitoring the hivemq broker with names like:
com.hivemq.cache.internal.export-states.loadExceptionCount
How can I get this to load the Groov EPIC readings?
Can I put the hiveMQ metrics into a different bucket?

hivemq-support · May 31, 2021, 3:35pm

Hi @jsorensen,

Nice to see your interest in HiveMQ and Sparkplug.

Typically this should work out of the box. It is possible that some topic or setting is not matched correctly.
Set your HiveMQ Log level to TRACE and share the outcome of your log after some publishes from your device that are not written to Influx, so we can identify what is going wrong.

To set the log level to TRACE go to {HIVEMQ_HOME}/conf/logback.xml and change line 69 from ~~${HIVEMQ_LOG_LEVEL:-INFO}~~ to ${HIVEMQ_LOG_LEVEL:-TRACE}
Can you also please share your sparkplug.properties file?

Regards,
Florian from the HiveMQ Team

jsorensen · May 31, 2021, 5:58pm

Hello Florian,

Thanks for getting back to me. We are a new company and have no legacy code, I really want to implement the MQTT broker from the beginning. I see how it can help us manage complicated data pathways and simplify security.

I’ve attached the sparkplug.properties file and a log file. I’m looking forward to getting this working.

What I see in Influx is 50+ parameters that look to be HiveMQ metrics. I don’t see any of the data from the Groov EPIC. I’ve verified this by deleting the bucket and recreating it. I’ve also tried it with HiveMQ and InfluxDB on the same computer and with them on different computers and get the same results.

James Sorensen

(Attachment hivemq.log is missing)

(Attachment sparkplug.properties is missing)

jsorensen · May 31, 2021, 6:28pm

I’ve been having troubles getting Outlook to send this email. My apologies for any duplicates I’ve sent.

Hello Florian,

Thanks for getting back to me. I’ve attached the sparkplug.properties file and a log file. I’m looking forward to getting this working.

What I see in Influx is 50+ parameters that look to be HiveMQ metrics. I don’t see any of the data from the Groov EPIC. I’ve verified this by deleting the bucket and recreating it. I’ve also tried it with HiveMQ and InfluxDB on the same computer and with them on different computers and get the same results.

James Sorensen

(Attachment hive.zip is missing)

jsorensen · May 31, 2021, 6:37pm

Hello Florian,

Thank you for responding. I’ve attached a log file and the sparkplug.properties file below. I tried sending you them as attachments but they keep getting rejected.

James Sorensen

========================== sparkplug.properties ==========================

–

Copyright 2021 HiveMQ GmbH

–

influxdb properties

influxdb.mode:tcp

influxdb.host:dragon

influxdb.port:8086

influxdb.protocol:http

influxdb.auth:-Vhp02epIxXu-dEjZcBkOrIQkZBNr-O_DoZW1TT0PwksfKw_XLRJ5xb0R5amJnam5nfwItOdAWeHcbRip06Paw==

#influxdb.prefix:

#influxdb.database:hivemq

influxdb.reportingInterval:1

influxdb.connectTimeout:5000

#influxdb.tags:host=localhost

InfluxDB cloud options

influxdb.bucket:lab

influxdb.organization:safire

sparkplug properties

sparkplug.version:spBv1.0

=============================== log file ================================

jsorensen · June 7, 2021, 8:18pm

Hello Florian,

Any news on why I’m can’t save Sparkplug messages into InfluxDB using the Hive MQ sparkplug extension.
I’m scheduled to present to management a working solution this Wednesday. .

As requested I attached the sparkplug.properties file and the trace log below.

Any help you can give me is greatly appreciated,

James Sorensen

(970) 227-4231

hivemq-support · June 8, 2021, 12:00pm

Hi @jsorensen,

I cannot find a log file attached.
Can you try uploading it directly on our forum?

Regards,
Florian

jsorensen · June 8, 2021, 3:15pm

Hello Florian,
I’ve attached a log file to this message.
James Sorensen

2021-06-08 08:40:38,808 TRACE - Increasing connection count by one. New value: 2
2021-06-08 08:40:38,809 TRACE - Suspending read operations for MQTT client with id dragon and IP 192.168.0.18
2021-06-08 08:40:38,826 TRACE - Client dragon specified a keepAlive value of 10s. Using keepAlive of 10s. The maximum timeout before disconnecting is 15.0s
2021-06-08 08:40:38,829 TRACE - Received ClientSessionSharedSubscriptionGetRequest request from nQ8fD for key dragon.
2021-06-08 08:40:38,829 TRACE - Restarting read operations for MQTT client with id dragon and IP 192.168.0.18
2021-06-08 08:40:38,830 TRACE - Received InflightMessagesRequest from node nQ8fD
2021-06-08 08:40:38,830 TRACE - Received NewMessagesRequest request from nQ8fD for key dragon.
2021-06-08 08:40:38,996 TRACE - Checking SUBSCRIBE message of client ‘dragon’ if topics are valid
2021-06-08 08:40:38,997 TRACE - Applied all subscriptions for client [dragon]
2021-06-08 08:40:39,001 TRACE - Adding subscriptions for client [dragon] and topic [spBv1.0/cabin/NCMD/reactor1/#] with qos [AT_MOST_ONCE]
2021-06-08 08:40:39,001 TRACE - Received retained message GET request for topic spBv1.0/cabin/NCMD/reactor1/#
2021-06-08 08:40:39,001 TRACE - Checking SUBSCRIBE message of client ‘dragon’ if topics are valid
2021-06-08 08:40:39,002 TRACE - Applied all subscriptions for client [dragon]
2021-06-08 08:40:39,003 TRACE - Adding subscriptions for client [dragon] and topic [spBv1.0/cabin/DCMD/reactor1/#] with qos [AT_MOST_ONCE]
2021-06-08 08:40:39,007 TRACE - Received retained message GET request for topic spBv1.0/cabin/DCMD/reactor1/#
2021-06-08 08:40:39,007 TRACE - Incoming publish from spBv1.0/cabin/NBIRTH/reactor1
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=bdSeq)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Control/Rebirth)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Control/Next MQTT Server)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Control/Current MQTT Server)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Properties/Hardware Make)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Properties/Device Type)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Properties/Software Version)
2021-06-08 08:40:39,021 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Control/Next MQTT Server)
2021-06-08 08:40:39,022 TRACE - Add Metric Mapping (Alias=0, MetricName=Node Control/Restart Edge Node)
2021-06-08 08:40:39,022 TRACE - Sparkplug Message type & structure TopicStructure{namespace=‘spBv1.0’, groupId=‘cabin’, messageType=‘NBIRTH’, eonId=‘reactor1’, deviceId=‘null’, scadaId=‘null’}
2021-06-08 08:40:39,027 DEBUG - Register SettableDoubleGauge metric for: sparkplug.reactor1.status
2021-06-08 08:40:39,028 TRACE - Metrics gauge [sparkplug.reactor1.status] added
2021-06-08 08:40:39,028 TRACE - Metrics counter [sparkplug.eons.current.count] added
2021-06-08 08:40:39,028 TRACE - Incoming publish from spBv1.0/cabin/DBIRTH/reactor1/epiclc
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=5, MetricName=Strategy/IO/Alarm)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=6, MetricName=Strategy/IO/FreezerDoor)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=7, MetricName=Strategy/IO/FreezerLight)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=8, MetricName=Strategy/IO/ParkingLotHeaterTPO)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=9, MetricName=Strategy/IO/Slushy_HLevel)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=10, MetricName=Strategy/IO/Slushy_LLevel)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=11, MetricName=Strategy/IO/SlushyHLevel_Control)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=12, MetricName=Strategy/IO/SlushyLLevel_Control)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=13, MetricName=Strategy/IO/SlushyPump1_Red_LED)
2021-06-08 08:40:39,032 TRACE - Add Metric Mapping (Alias=14, MetricName=Strategy/IO/SlushyPump2)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=15, MetricName=Strategy/IO/SlushyStartButton)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=16, MetricName=Strategy/IO/SlushyValve_Green_LED)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=17, MetricName=Strategy/IO/FuelLevelPot)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=18, MetricName=Strategy/IO/ParkingLotHeaterTemp)
2021-06-08 08:40:39,033 TRACE - Received cluster PUBLISH for client ‘hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70’ from node nQ8fD.
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=19, MetricName=Strategy/Numeric Variables/FuelLevelReading)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=0, MetricName=Device Properties/Part Number)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=0, MetricName=Device Properties/Controller Firmware Version)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=0, MetricName=Device Properties/Strategy Title)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=0, MetricName=Device Properties/Strategy CRC)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=1, MetricName=Device Control/Scanner Interval Ms)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=2, MetricName=Device Control/Additional Host Tasks)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=3, MetricName=Device Properties/Strategy Tag Access Time Ms)
2021-06-08 08:40:39,033 TRACE - Add Metric Mapping (Alias=4, MetricName=Device Properties/Strategy Write Queue Depth)
2021-06-08 08:40:39,033 TRACE - Sparkplug Message type & structure TopicStructure{namespace=‘spBv1.0’, groupId=‘cabin’, messageType=‘DBIRTH’, eonId=‘reactor1’, deviceId=‘epiclc’, scadaId=‘null’}
2021-06-08 08:40:39,034 DEBUG - Register SettableDoubleGauge metric for: sparkplug.reactor1.epiclc.status
2021-06-08 08:40:39,034 TRACE - Metrics gauge [sparkplug.reactor1.epiclc.status] added
2021-06-08 08:40:39,034 TRACE - Metrics counter [sparkplug.devices.current.count] added
2021-06-08 08:40:39,037 TRACE - Received PublishAvailableRequest
2021-06-08 08:40:39,038 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:39,038 TRACE - Received cluster PUBLISH for client ‘hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70’ from node nQ8fD.
2021-06-08 08:40:39,039 TRACE - Received PublishAvailableRequest
2021-06-08 08:40:39,040 TRACE - Client hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70: Sending PUBLISH QoS 0 Message with packet id 0
2021-06-08 08:40:39,040 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:39,041 TRACE - Client hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70: Sending PUBLISH QoS 0 Message with packet id 0
2021-06-08 08:40:39,041 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:39,116 TRACE - Added 14399 credits across 1 clients.
2021-06-08 08:40:39,316 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:39,517 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:39,717 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:39,917 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:40,011 TRACE - Incoming publish from spBv1.0/cabin/DDATA/reactor1/epiclc
2021-06-08 08:40:40,011 TRACE - Add Metric Mapping (Alias=17, MetricName=)
2021-06-08 08:40:40,012 TRACE - Add Metric Mapping (Alias=18, MetricName=)
2021-06-08 08:40:40,012 TRACE - Add Metric Mapping (Alias=19, MetricName=)
2021-06-08 08:40:40,012 TRACE - Sparkplug Message type & structure TopicStructure{namespace=‘spBv1.0’, groupId=‘cabin’, messageType=‘DDATA’, eonId=‘reactor1’, deviceId=‘epiclc’, scadaId=‘null’}
2021-06-08 08:40:40,012 DEBUG - Register SettableDoubleGauge metric for: sparkplug.reactor1.epiclc.
2021-06-08 08:40:40,012 TRACE - Metrics gauge [sparkplug.reactor1.epiclc.] added
2021-06-08 08:40:40,013 TRACE - Received cluster PUBLISH for client ‘hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70’ from node nQ8fD.
2021-06-08 08:40:40,014 TRACE - Received PublishAvailableRequest
2021-06-08 08:40:40,015 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:40,016 TRACE - Client hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70: Sending PUBLISH QoS 0 Message with packet id 0
2021-06-08 08:40:40,016 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:40,118 TRACE - Added 13700 credits across 1 clients.
2021-06-08 08:40:40,318 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:40,519 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:40,719 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:40,919 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:41,011 TRACE - Incoming publish from spBv1.0/cabin/DDATA/reactor1/epiclc
2021-06-08 08:40:41,012 TRACE - Add Metric Mapping (Alias=17, MetricName=)
2021-06-08 08:40:41,012 TRACE - Add Metric Mapping (Alias=18, MetricName=)
2021-06-08 08:40:41,012 TRACE - Add Metric Mapping (Alias=19, MetricName=)
2021-06-08 08:40:41,012 TRACE - Sparkplug Message type & structure TopicStructure{namespace=‘spBv1.0’, groupId=‘cabin’, messageType=‘DDATA’, eonId=‘reactor1’, deviceId=‘epiclc’, scadaId=‘null’}
2021-06-08 08:40:41,013 TRACE - Received cluster PUBLISH for client ‘hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70’ from node nQ8fD.
2021-06-08 08:40:41,014 TRACE - Received PublishAvailableRequest
2021-06-08 08:40:41,014 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:41,015 TRACE - Client hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70: Sending PUBLISH QoS 0 Message with packet id 0
2021-06-08 08:40:41,015 TRACE - Received NewMessagesRequest request from nQ8fD for key hmq_nQ8fD_0_33abe57d8a2186f39b05fe47046ceb70.
2021-06-08 08:40:41,120 TRACE - Added 7100 credits across 1 clients.
2021-06-08 08:40:41,320 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:41,520 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:41,721 TRACE - Added 0 credits across 0 clients.
2021-06-08 08:40:41,922 TRACE - Added 0 credits across 0 clients.

jsorensen · June 8, 2021, 3:34pm

Hello Florian,
Please let me know if the above log file is what you need. I tried uploading a longer file but the forum does not allow more than 32000 characters. Also you have to embed the log in a message, you cannot upload it as an attachment. Attachments can only be image files.

In the message above I show the sparkplug.properties - influxdb.mode as tcp. I’ve tried both. The log file was generated with influxdb.mode set to cloud.
James Sorensen

seti · June 10, 2021, 7:47am

Hi @jsorensen

Four things which come to my mind here:

Which InfluxDB version are you using?
The extension seems to be working fine on first sight, but the publishing to InfluxDB may be the problem. Can you look for any InfluxDB related log messages?
Furthermore, can you confirm that you are not receiving any metric starting with sparkplug? The metrics sparkplug.devices.current.count and sparkplug.eons.current.count should be present if the publishing to InfluxDB works.
Also, I’m a bit concerned about some empty metric names sent by the device Add Metric Mapping (Alias=17, MetricName=) - any ideas?

If you are using InfluxDB 2.0, the following configuration should be definetly fine:

influxdb.mode:cloud
influxdb.host:dragon
influxdb.port:8086
influxdb.protocol:http
influxdb.auth:<your token>

influxdb.bucket:<your-bucket>
influxdb.organization:<your company>

influxdb.reportingInterval:1
influxdb.connectTimeout:5000

influxdb.tags:host=dragon

Regards,
Till from the HiveMQ Team

BOU · February 3, 2023, 9:10am

Hello @seti
I hope you are doing well

I have a problem with the Sparkplug InfluxDB Extension for HiveMQ. I am using a Windows OS
After I run Influxdb and HiveMQ broker I have an error message: “Unable to report to InfluxDB with error ‘Unsupported or unrecognized SSL message’. Discarding data.”

but I want to note that in the HiveMQ broker I am using TLS with self-certification

the picture below is the influxdb.properties file of the extension

please, I need some help
thank you in advance

Daria_H · February 3, 2023, 10:28am

@BOU ,

You forgot to provide this info

Kind regards,
Dasha from HiveMQ Team

BOU · February 3, 2023, 10:46am

Hi Daria
thank you for your reply
I am using a local influxdb2-2.4.0-windows-amd64, an OSS version

Topic		Replies	Views
Serialization of kafka metrics failed HiveMQ Extensions	6	618	August 25, 2022
How come my access.log is empty? HiveMQ Extension SDK	1	200	September 27, 2023
Logback setting to archive event log HiveMQ Community Edition	5	929	May 19, 2021
Error in log - Serialization of Kafka metrics?	1	308	August 24, 2022
Hive MQ Logging : Assistance Required HiveMQ Extensions	8	529	June 14, 2023

Sparkplug extension does not save Groov EPIC measurements

–

Copyright 2021 HiveMQ GmbH

–

influxdb properties

InfluxDB cloud options

sparkplug properties

Related Topics