Getting start with Arduino ESP8266 certificate generation error

[manone]

I’m trying to implement ESP 8266 connection to HiveMq cloud cluster, and I’m following instructions in the relative page: HiveMQ Cloud?

I take the script this script

and I compile it in VisualCode, and I take this error:
You need the program ‘ar’ from xtensa-lx106-elf found here: (esp8266-arduino-core)/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/bin/ar
File “C:\Users\Francesco\Desktop\cert.py”, line 28, in
raise Exception(“You need the program ‘ar’ from xtensa-lx106-elf found here: (esp8266-arduino-core)/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/bin/ar”)

I check and in dir
C:\Program Files (x86)\Arduino\hardware\esp8266com\esp8266\tools\xtensa-lx106-elf\xtensa-lx106-elf\bin
I have ar.exe file,

someone know the reason?
or could suggest any other way/method ?

best regards
fn

[Daria_H]

Hello francesco,

Glad to meet a new HiveMQ enthusiast!

Your exception comes from the following line of the Python code:

# check if ar and openssl are available
if which('ar') is None and not os.path.isfile('./ar') and not os.path.isfile('./ar.exe'):
    raise Exception("You need the program 'ar' from xtensa-lx106-elf found here: (esp8266-arduino-core)/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/bin/ar")

Basically, the script either cannot find the file ar.exe or it does not recognise it as a file.

Is this only an issue when you Run this Python script from the Visual Code? or it is also the issue when you run it from the command line:

python3 certs-from-mozilla.py

Please check and see if the error is the same.

Thank you ,
Kind regards,
Dasha from HiveMQ Team

[manone]

@Daria_H

certs-from-mozilla.py

Hello Dasha,
many thanks for your suggestion,

I tested, and the error is the same:
"File “certs-from-mozilla.py”, line 28, in
raise Exception(“You need the program ‘ar’ from xtensa-lx106-elf found here: (esp8266-arduino-core)/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/bin/ar”)
Exception: You need the program ‘ar’ from xtensa-lx106-elf found here: (esp8266-arduino-core)/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/bin/ar "

in really I 've the ar.exe file in my pc in path:
C:\Program Files (x86)\Arduino\hardware\esp8266com\esp8266\tools\xtensa-lx106-elf\xtensa-lx106-elf\bin

how I can make it available to the python script?

kind regards
fn

[Daria_H]

Hello fn,

Thank you for the update. Let’s see now, which one of the conditions is not met and causing the exception to get thrown.

Run a simple script and see the output. The output will indicate whether the problem is with the function which() or with the function os.path.isfile().

Copy the script to the file test.py:

import os
from shutil import which

if which('ar') is None:
    print("Function which('ar') returned 'None'")
else:
    print("Function which('ar') returned: " + which('ar'))
if not os.path.isfile('./ar.exe'):
    print("Function os.path.isfile('./ar.exe') returned 'false'")
else:
    print("Function os.path.isfile('./ar.exe') returned 'true'")

Save the file as test.py and run it with your Python:

python3 test.py

Check out the output and see what does each function return.

Kind regards,
Dasha

[manone]

@manone

Hello Dasha,

Hello Dasha,

the response is:
python test.py
Function which(‘ar’) returned ‘None’
Function os.path.isfile(‘./ar.exe’) returned ‘false’

thanks for your help,
so could you also suggest me a tutorial link about CA certificate and/or hiveMq certificates system
to understand better functioning…
because this is my first IOT project with ESP8266 and cloud Mqtt system, but in my mind I would expand my know how also in other way, hjava , python etc…
to use it in my work if possible…

kind regards
fn

[Daria_H]

Hi fn,

Thank you for checking this! The output indicates that neither Python os.path module nor the which function cannot find the file. Perhaps if you put the python script into the same folder as your ar.exe file, then for the Python it will be easier to find it.

1. Copy the Python script file into the same directory as ar.exe (i.e. C:\Program Files (x86)\Arduino\hardware\esp8266com\esp8266\tools\xtensa-lx106-elf\xtensa-lx106-elf\bin)
2. In the terminal, change your current directory to that directory
3. From the terminal, Run the Python script with the python3 command.

Kind regards,
Dasha

[baslam.em20ceme]

There is no such directory on my PC. What should do?

[manone]

Goodmorning Dasha,

thanks for your help again,
I tested and I made a littele step.
I executed scritp as per your indication and now give me another error:

"C:\Program Files (x86)\Arduino\hardware\esp8266com\esp8266\tools\xtensa-lx106-elf\xtensa-lx106-elf\bin>python certs-from-mozilla.py
Traceback (most recent call last):
File “C:\Program Files (x86)\Arduino\hardware\esp8266com\esp8266\tools\xtensa-lx106-elf\xtensa-lx106-elf\bin\certs-from-mozilla.py”, line 30, in
raise Exception(“You need to have openssl in PATH, installable from https://www.openssl.org/”)
Exception: You need to have openssl in PATH, installable from https://www.openssl.org/ "

seem that I have not openssl.exe PATH (wich path exactely ?)
but in really I have it in C:\Program Files\OpenSSL-Win64\bin, that is differente from openssl.org software.

Sincerely I am a little bit discouraged, by all this complications,
It’s possible to generate the ssl certificate in other way?

many htanks for your patient,
and best regards
fn

[Daria_H]

Hi fn,

PATH is an environment variable on Microsoft Windows, specifying a set of directories where executable programs are located.

You can add the path to your openssl.exe to the current PATH variable in the following way:

set PATH=%PATH%;C:\your\path\here\

Note, that this will only update the PATH for your current terminal session.

To update the PATH permanently, follow these steps below:

Setting the path and variables in Windows 10

1. From the desktop, right-click the very bottom-left corner of the screen to access the Power User Task Menu.
2. In the Power User Task Menu, select the System option.
3. In the Settings window, scroll down to the Related settings section and click the System info link.
4. In the System window, click the Advanced system settings link in the left navigation pane.
5. In the System Properties window, click the Advanced tab, then click the Environment Variables button near the bottom of that tab.
6. In the Environment Variables window (pictured below), highlight the Path variable in the System variables section and click the Edit button. Add or modify the path lines with the paths you want the computer to access. Each different directory is separated with a semicolon, as shown below.

C:\Program Files;C:\Winnt;C:\Winnt\System32 ; C:\your\path\here\

You can also add the path to your ar.exe to the PATH.

Kind regards,
Dasha from HiveMQ Team

[henk]

Hello.
I am new to hivemq and wanted to use it with a esp32/esp8266. But as above I folowed all steps off the instruction on hivemq but I did not manage to get a certificate.
Visual code / Python will not find application “ar.exe”. I keep getting the warning: “Exception: You need the program ‘ar’ from xtensa-lx106-elf found here: (esp8266-arduino-core)/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/xtensa-lx106-elf/bin/ar”
I have been trying al kind off things as suggested above. No result. Is there a way to modify the path to this ar.exe? or better,:
Is there an other way to generate this certificate.? Else Hivemq is not useable for me :(.

Kind Regards
Henk.

[Daria_H]

Hello Henk

and welcome to our HiveMQ Cloud community. Probably you are contacting us regarding Python script certs-from-mozilla.py from the ESP8266 Community Forum?

The Exception “You need the program ‘ar’ from xtensa-lx106-elf found here…” indicates that your Python cannot find the “ar.exe” file that is used in the script. To help your Python to find the file, please add its path to the PATH environment variable before running the certs-from-mozilla.py script.

If you our need help on how to update the PATH variable please let us know.

Kind regards,
Dasha from HiveMQ Team

[henk]

Hello Daria,
Thanks for your reply.
I am not a software enthusiast. I try to get a esp32 (so not a esp8266) node (TTGO WIFILORA32) to work with Hivemq. I just want it to publish sensordata as voltage, temp etc.Up to now I used Lorawan but this node is near a wifi router so MQTT should be better,
I have tried to follow the instructions on the website off HIVEMQ but it is in my opinion too complicated to "just"get a certificate. Up to the part of the cert-from-python file I got it all done. Took me some days up to now.Then I do not get it to work.
I have installed:Arduino 1.18.19 , Phyton3 and Visual studio on my desktop and on a notebook. Same result: after doing everything as in the guide there is no directory …/esp8266com/etc to be found. So also no “ar.exe” AND if I could get that sollved the next problem is waiting: the “Openssl” which I do not have installed.
I am quite willing to try other software to program the node.
So is there a way to programm a node that a “normal” user can do? Is it possible to use e.g.PlatformIO or anothet ide without this kind of external dependicies?

[henk]

Hello Daria
After writing my reply I found this: Arduino example with server signed certificate?
I tried this code and I succeeded to have the node to contact HiveMQ. But after some time (about a minute) it loses connection again and I get the error:… rc = -2 … When I restart the connectien it will work for a short time. About 10 messages come through.
The problem is, i think, again the certificate. SO Is there an other way to get a valid certificate so I can use the example .ino provided by HiveMQ?

[Daria_H]

Hi Henk,

When you device requires a server CA file to connect via TLS, you can generate it from your HiveMQ cloud instance using openssl s_client command:

openssl s_client -connect XXXXXXXXXXX.s2.eu.hivemq.cloud:8883 -showcerts < /dev/null 2> /dev/null | sed -n '/BEGIN/,/END/p' > server.pem

Please let me know if this helped,
Thank you ,
Dasha from HiveMQ Team

[henk]

Hello
Txs but another stupid question .
I have downloaded HiveMQ client. (for windows10) started it witt the Mqtt-cli-shell.cmd. (there is no run.bat in the directory).
When I input the cmd you gave I get this: (I put xxxxx over my identifiers.).What did I wrong?

KInd regards
Henk

Usage: mqtt shell [-hV] [COMMAND]

Starts MqttCLI in shell mode, to enable interactive mode with further sub commands.

Options:
-h, --help display this help message
-V, --version display version info

Commands:
help Displays help information about the specified command
version Prints version information
con, connect Connects an mqtt client
dis, disconnect Disconnects an mqtt client
switch Switch the current context
ls, list List all connected clients with their respective identifieres
cls, clear Clears the screen
exit Exit the shell

Press Ctl-C to exit.

Using default values from properties file C:\Users\hpebo.mqtt-cli\config.properties:
Host: localhost, Port: 1883, Mqtt-Version MQTT_5_0, Shell-Debug-Level: VERBOSE
Writing Logfile to C:\Users\xxxxxx.mqtt-cli\logs\hmq-cli.2022-01-05.log
mqtt> con -h xxxxxxxxxxxxxe8.s2.eu.hivemq.cloud -p 8883 -s -u -pw xxxxxxxxxx

Server closed connection without DISCONNECT.
mqtt> openssl s_client -connect xxxxxxxxxxxxxxxxxxxxxxxxxxxxx.eu.hivemq.cloud:8883 -showcerts < /dev/null 2> /dev/null | sed -n ‘/BEGIN/,/END/p’ > server.pem

Unmatched arguments from index 0: ‘openssl’, ‘s_client’, ‘-connect’, ‘xxxxxxxxxxxxxxxxxxxxxxxxe8.s2.eu.hivemq.cloud:8883’, ‘-showcerts’, ‘<’, ‘/dev/null’, ‘2>’, ‘/dev/null’, ‘|’, ‘sed’, ‘-n’, ‘/BEGIN/,/END/p’, ‘>’, ‘server.pem’

[rickqinj]

About the exception “ar” not found.
If you get your ESP8266 boards installed via Arduino IDE Board Manager, you may not have the path in your Arduino IDE home folder which the cert download script certs-from-mozilla.py requires. However, you can find the folder somewhere else on your system. In my case, I used the software Everything to look for the folder xtensa-lx106-elf and found the ar.exe finally.

About the “openssl” not found.
Actually, it is not necessary to have OpenSSL installed if you’ve had Git installed on your system. As the cert download script needs to access the openssl.exe to execute some certificate-related commands, the only thing you need to do is to add the path of the openssl.exe to your system PATH variables to make it accessible to the cert download script, that’s it.

Finally, execute the cert download script
Copy the cert download script certs-from-mozilla.py to the folder where the ar.exe is located. And open the script with Python IDLE and run. After it is completed, you will get a folder data with a certs.ar file in it. Then, move the folder data into your project folder and go on the HiveMQ tutorial.

[henk]

Hi
Thanks for the respons . I have managed to get a certificate by running “certs-from-mozila.py” on a raspberry. Here it worked ok.
I also replaced the esp32 with an esp8266 Nodemcu. With this changed I got the MQTT connection running and stable with the help off the example of Virtuino i found on youtube.
It cost me a lot off time to get a relative simple node running because I am just starting to learn MQTT. It would help if there were some more examples on Hivemq how to get things running on different boards (esp32, esp8266, arduino ) to implement a secure connection.

[Daria_H]

Hi Henk,

Thank you for the feedback! We do realise how important it is to have an example for every popular use case. We are continously working on adding more code examples to our list.

Kind regards,
Dasha from HiveMQ Team

[henk]

Hello
Good to hear🙂.
BUT where can I find these list of examples? I have been searching in the forum but I did find questions but few examples or instructions. I found on the web enough info on how to connect to a public MQTT (port 1883) but the problems starts when you must use a secure connection ( port 8883 ) .

[Daria_H]

Hi Henk,

Guides for various MQTT clients are collected on the “Getting Started” page of your HiveMQ Cloud cluster. Your example comes from the HiveMQ Client documentation (look for simpleAuth()) .

Kind regards,
Dasha from HiveMQ Team