Hi,
I followed the documentation with the link Concepts and Components of the HiveMQ Enterprise Security Extension :: HiveMQ Documentation. I have created the database, filled data into the tables, downloaded the driver, put the configure file into conf folder which is under the enterprise security extension folder. I tested the broker, and the broker does not read and use the config I made. The broker now can allow the unauthenticated users to connect the broker, and allow authorized clients to pulish/subscirbe the topics.
welcome to the HiveMQ Community! We’re excited to have you join us, especially because of your interest in MQTT and the HiveMQ broker. It’s great to see new users like you.
Looking at your description of your issue, most probably, you have the Enterprise Security Extension disabled and the Allow All Extension enabled. To verify, check the log /opt/hivemq/log/hivemq.log and see which extensions have started successfully and which did not. After checking the log, check each extension’s directory for the file “DISABLED”, for example:
If the “DISABLED” file is present, it indicates that the extension is disabled.
If an extension was originally enabled, but during HiveMQ broker startup got disabled, this might indicate an issue within the extension’s configuration. Error messages in the log /opt/hivemq/log/hivemq.log will provide more details on possible issue.
All extensions are enabled and disabled during the hivemq startup.
Starting HiveMQ extension system.
2023-11-16 13:15:59,485 INFO - Starting extension with id “hivemq-allow-all-extension” at \hivemq-4.21.0\extensions\hivemq-allow-all-extension
2023-11-16 13:15:59,485 WARN -
################################################################################################################
This HiveMQ deployment is not secure! You are lacking Authentication and Authorization.
Right now any MQTT client can connect to the broker with a full set of permissions.
For production usage, add an appropriate security extension and remove the hivemq-allow-all extension.
################################################################################################################
2023-11-16 13:15:59,485 INFO - Extension “Allow All Extension” version 1.0.0 started successfully.
2023-11-16 13:16:02,149 INFO - EiCja: no members discovered after 2014 ms: creating cluster as first member
2023-11-16 13:16:02,169 INFO - No user for HiveMQ Control Center configured. Starting with default user
2023-11-16 13:16:02,173 INFO - Starting HiveMQ Control Center on address 127.0.0.1 and port 8080
2023-11-16 13:16:02,376 INFO - Control Center Audit Logging started.
2023-11-16 13:16:02,377 INFO - Started HiveMQ Control Center in 208ms
The following is from the log file.
which configure file should I modify then? I did not find any useful information from the log.
As per @Daria_H 's message, based on the provided log message, only the “Allow All Extension” is currently enabled for this broker. These extensions can be enabled and disabled at runtime by adding an empty file named “DISABLED” in the extension folder.
For example, to disable the “Allow All Extension”, there must be file named “DISABLED” within the extension folder.
An easy way to complete this on, for example, a Linux machine, you could simply navigate to the /opt/hivemq/extensions/hivemq-allow-all-extension folder, and execute the following command :
touch DISABLED
Similarly, for the Enterprise Security Extension, we will need to verify that this disabled file does not exist within the extension installation directory.
I have set up the postgresql and the security extension is enabled. While I got the following issue and the client is not authorized when it tries to connect to the broker. The error is: Connection failed: ConnectResult[Code: NotAuthorized] . My client is this:
var options = new HiveMQClientOptions
{
ClientId = “superuser”,
Host = hostName,
Port = 1883,
UseTLS = false,
UserName = “superuser”,
Password = “hivemq”,
LastWillAndTestament = lastWillAndTestament
};
var client = new HiveMQClient(options);
The configure is user role based, the superuser has all permission. The configure set the use-authorization-role-key as true.
Can you help me with that?
2023-11-16 22:23:49,772 INFO - Starting HiveMQ Enterprise Server
2023-11-16 22:23:49,774 INFO - HiveMQ version: 4.21.0
2023-11-16 22:23:49,774 INFO - HiveMQ home directory: \hivemq-4.21.0
2023-11-16 22:23:49,779 INFO - Log Configuration was overridden by \hivemq-4.21.0\conf\logback.xml
2023-11-16 22:23:50,093 INFO - Successfully loaded configuration from ‘hivemq-4.21.0\conf\config.xml’.
2023-11-16 22:23:50,219 INFO - This node’s ID is 7BGU6
2023-11-16 22:23:50,220 INFO - Clustering is disabled
2023-11-16 22:23:54,426 INFO - No valid license file found. Using trial license, restricted to 25 connections.
2023-11-16 22:23:55,617 INFO - No valid license file for Data Hub found. Using free license, restricted to 1 policy.
2023-11-16 22:23:55,838 INFO - This node uses ‘24’ CPU cores.
2023-11-16 22:23:55,866 INFO - Starting HiveMQ extension system.
2023-11-16 22:23:55,927 WARN -
###############################################################################
No security extension present, MQTT clients can not connect to this broker.
###############################################################################
2023-11-16 22:23:58,747 INFO - 7BGU6: no members discovered after 2010 ms: creating cluster as first member
2023-11-16 22:23:58,779 INFO - No user for HiveMQ Control Center configured. Starting with default user
2023-11-16 22:23:58,779 INFO - Starting HiveMQ Control Center on address 127.0.0.1 and port 8080
2023-11-16 22:23:58,986 INFO - Control Center Audit Logging started.
2023-11-16 22:23:58,987 INFO - Started HiveMQ Control Center in 209ms
2023-11-16 22:23:59,002 INFO - Starting TCP listener on address 0.0.0.0 and port 1883
2023-11-16 22:23:59,222 INFO - Started TCP Listener on address 0.0.0.0 and on port 1883.
2023-11-16 22:23:59,222 INFO - Started HiveMQ in 9455ms
Based on the logs provided, the Enterprise Security Extension is not starting when the broker is started, hence the warning regarding the lack of a present security extension. To confirm, has the ‘DISABLED’ file been removed from the Enterprise Security Extension folder at the following directory :
I removed the Disabled file under the /opt/hivemq/extensions/hivemq-enterprise-security-extension/. but it would be generated immediately. Thus, the extension will never be enabled. How can I stop the file generation process?
If you are experiencing a situation where the Enterprise Security Extension starts enabled but becomes disabled during startup, it could potentially point to a configuration issue with the extension.
To assist you further in resolving this matter, I kindly suggest reviewing the hivemq.log for any pertinent statements related to the Enterprise Security Extension being disabled. This log file should provide insights into the reasons behind the extension’s deactivation.
I got the error: Registered driver with driverClassName=org.postgresql.Driver was not found, trying direct instantiation… While I have the postgresql-42.6.0.jar under the folder: hivemq-4.21.0\extensions\hivemq-enterprise-security-extension\drivers\jdbc, which I followed the documentation step by step.
Thanks
The correct location to add the driver file is within the directory path: hivemq-4.21.0\extensions\hivemq-enterprise-security-extension\drivers. Please move the file to the correct location and start the extension. This should resolve the observed error.
while if I move the driver jar file to the folder hivemq-4.21.0\extensions\hivemq-enterprise-security-extension\drivers, and it is not what the documenation says. I get the error if I move the file to the driver folder instead of jdbc folder :
java.lang.RuntimeException: Failed to load driver class org.postgresql.Driver in either of HikariConfig class loader or Thread context classloader
at com.zaxxer.hikari.HikariConfig.setDriverClassName(Source.java:488)
at hse.al.a.a(Source.java:103)
at hse.aj.a.h(Source.java:87)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at hmq.fN.f.run(Source.java:39)
at com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(Source.java:241)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2023-11-17 15:23:34,325 WARN - Could not create datasource for realm postgres-backend
2023-11-17 15:23:34,326 ERROR - The HiveMQ Enterprise Security Extension could not start because of a configuration error: No realm could be started.
2023-11-17 15:23:34,326 INFO - Startup of extension with id “hivemq-enterprise-security-extension” was prevented by the extension itself, reason: No realm could be started. Extension will be disabled.
2023-11-17 15:23:34,329 WARN -
###############################################################################
No security extension present, MQTT clients can not connect to this broker.
###############################################################################
2023-11-17 15:23:36,729 INFO - 2pbi4: no members discovered after 2017 ms: creating cluster as first member
2023-11-17 15:23:36,760 INFO - No user for HiveMQ Control Center configured. Starting with default user
2023-11-17 15:23:36,761 INFO - Starting HiveMQ Control Center on address 127.0.0.1 and port 8080
2023-11-17 15:23:36,898 INFO - Control Center Audit Logging started.
2023-11-17 15:23:36,899 INFO - Started HiveMQ Control Center in 139ms
2023-11-17 15:23:36,910 INFO - Starting TCP listener on address 0.0.0.0 and port 1883
2023-11-17 15:23:37,046 INFO - Started TCP Listener on address 0.0.0.0 and on port 1883.
2023-11-17 15:23:37,046 INFO - Started HiveMQ in 8625ms
Hi all,
I got the error message: " Registered driver with driverClassName=org.postgresql.Driver was not found, trying direct instantiation." Do you know what happened? I already put the driver in to the location $HIVEMQ_HOME/extensions/hivemq-enterprise-security-extension/drivers/jdbc.
2023-11-20 15:28:40,248 INFO - com.hivemq.extensions.ese.postgres.postgres-backend - Starting…
2023-11-20 15:28:40,254 WARN - Registered driver with driverClassName=org.postgresql.Driver was not found, trying direct instantiation.
2023-11-20 15:28:40,361 INFO - com.hivemq.extensions.ese.postgres.postgres-backend - Added connection org.postgresql.jdbc.PgConnection@7181554e
2023-11-20 15:28:40,365 INFO - com.hivemq.extensions.ese.postgres.postgres-backend - Start completed.
2023-11-20 15:28:40,399 INFO - Started HiveMQ Enterprise Security Extension successfully in 365ms.
Thank you for reaching out. I appreciate your diligence in providing the log statement:
This log entry signifies a successful initiation of the HiveMQ Enterprise Security Extension. To further assist you, kindly examine and confirm the contents of the directory: $HIVEMQ_HOME/extensions/hivemq-enterprise-security-extension. Specifically, check for the presence or absence of the file named “DISABLED.”
I tried with file realm, it works well.
With postgresql, it keeps having the issue: Registered driver with driverClassName=org.postgresql.Driver was not found, trying direct instantiation.
I have put the postgresql-42.6.0.jar in the drivers folder as you suggested.
The log statement you are referring to is a warning. What are your next statements in the log?
DEBUG - validationTimeout...............5000
INFO - com.hivemq.extensions.ese.postgres.postgres-backend - Starting...
WARN - Registered driver with driverClassName=org.postgresql.Driver was not found, trying direct instantiation.
DEBUG - Driver class org.postgresql.Driver found in Thread context class loader hse.al.a@482b609e
INFO - com.hivemq.extensions.ese.postgres.postgres-backend - Added connection org.postgresql.jdbc.PgConnection@676b56cd
INFO - com.hivemq.extensions.ese.postgres.postgres-backend - Start completed.
DEBUG - Registering Authentication Expirer Provider.
DEBUG - Registering HiveMQ Authenticator Provider.
DEBUG - Simple authenticator added by extension 'hivemq-enterprise-security-extension'.
DEBUG - Set Control Center authenticator for Pipeline control-center-pipeline-2.
DEBUG - Control Center authenticator added by extension 'hivemq-enterprise-security-extension'.
DEBUG - REST API authenticator added by extension 'hivemq-enterprise-security-extension'.
INFO - Started HiveMQ Enterprise Security Extension successfully in 302ms.
INFO - Extension "HiveMQ Enterprise Security Extension" version 4.22.0 started successfully.
What hash method does hivemq use for hash the password?
I created one method for hashing the password and save the hashed password value into database. The authentication fails, the response reason is notArhorized. Do you know why?
The hash password method is this:
static string HashPassword(string password, string saltString, int iterations)
{
byte saltBytes = Convert.FromBase64String(saltString);
using (var pbkdf2 = new Rfc2898DeriveBytes(password, saltBytes, iterations, HashAlgorithmName.SHA512))
{
byte[] hash = pbkdf2.GetBytes(64);
return Convert.ToBase64String(hash);
}
}
The password, iteraction and algorithm is
string enteredPassword = “mqtt-password-1”;
string storedSaltFromDB = “Nv6NU9XY7tvHdSGaKmNTOw==”;
string hashedPassword = HashPassword(enteredPassword, storedSaltFromDB, 100);
The hasdedPassword in the database is “iFOP4jBqt9FqskKSHGdgIuzRa1sWXNHBhFR7EprYIroGJfOWTqXZ9BlYgfjg2jZ1rYsp8qed1AsvhPmBLFdnSg==”
The options is:
var options = new HiveMQClientOptions
{
ClientId = “Client_1”,
Host = hostName,
Port = 1883,
UseTLS = false,
UserName = enteredUsername,
Password = hashedPassword,//“mqtt-password-1”,
LastWillAndTestament = lastWillAndTestament
};
var client = new HiveMQClient(options);
Why would you try connecting to the broker using hashed password and not the plain text password in the HiveMQClientOptions, is there any reason for that? Usually you simply specify the password:
var options = new HiveMQClientOptions
{
Host = "derstarter-2gombb.a01.euc1.aws.hivemq.cloud",
Port = 8883,
UseTLS = true,
UserName = "Superuser1",
Password = "Superuser1",
};
var client = new HiveMQClient(options);
The password hash in the database is typically generated using the following ESE Helper command:
java -jar hivemq-ese-helper.jar db create mqtt-client-user mqtt-user-1 mqtt-password-1
To assist you better, could you please share the specific parameters you used when executing this command to generate the password hash in your example? This information will help us identify any potential differences and address the authentication issue you’re experiencing.