HiveMQ Docker logback file

HiveMQ Community Edition
1

Hi Team,

I have deployed HiveMQ on Kubernetes (Opnestack Paltform) using below link.

https://github.com/hivemq/hivemq4-docker-images/tree/master/hivemq4/dns-image

My requirement is that, i would like to redirect both event.log and hivemq.log in single file. but not successful. I have modified the logback file but event logs not redirecting in hivemq.log file.

below is logback file:

<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
        <pattern>%-30(%d %level)- %msg%n%ex</pattern>
    </encoder>
</appender>

<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${hivemq.log.folder}/hivemq.log</file>
    <append>true</append>

    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- daily rollover -->
        <fileNamePattern>${hivemq.log.folder}/hivemq.%d{yyyy-MM-dd}.log</fileNamePattern>

        <!-- keep 30 days' worth of history -->
        <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder>
        <pattern>%-30(%d %level)- %msg%n%ex</pattern>
    </encoder>
</appender>

<appender name="EVENT-FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${hivemq.log.folder}/hivemq.log</file>
    <append>true</append>
    <encoder>
        <!--pattern>%-24(%d)- %msg%n%ex</pattern> -->
        <pattern>%-30(%d %level)- %msg%n%ex</pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
        <fileNamePattern>${hivemq.log.folder}/hivemq-%i.log.gz</fileNamePattern>
        <minIndex>1</minIndex>
        <maxIndex>5</maxIndex>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
        <maxFileSize>100MB</maxFileSize>
    </triggeringPolicy>
</appender>

Could you please help me out, how to do this with logback file.

Currently only hivemq.log printing on docker console. I need both log file in single and should be printed on console and should store in /var/lib/docker/conatiners/*/json.log, so i can easily get all logs with log driver and visualize on kibana dashboard.

Thanks
Rahul

2

Hey Rahul,

please set your appender ref to “FILE” instead of “EVENT-FILE” in order to log to console and a single file.

     <root level="INFO">
         <appender-ref ref="FILE"/>
         <appender-ref ref="CONSOLE"/>
     </root>

The path and filename are defined in:

<appender name="FILE" class...
    <file>/path/desired/filename.json</file>

I hope this helps you achieve your desired output!

Best regards,
Finn from the HiveMQ team

3

@zingiber,

Thanks for your input.

I have already make mentioned practice and logs for both files (event.log and hivemq.log) in single file.Its working fine, but issue is that, when i running command on Kubernetes (kubectl logs pod/hivemq-replica-dfwwx) it’s only printing hivemq.log on console. It should be print both event and hivemq logs on console. Is there any way how to print both logs on docker console.

Pods printing only STDOUT hivemq logs on kubectl console because pods log getting stored in /var/lib/docker/containers//-json.log.

The requirement is to get all container logs with filebeat and visualize on Kibana dashboard.
It will be easy for me to get exact count of no. of connect, disconnect, session expiry and more.

Please guide me further.
Thanks

4

In order to output to console in addition to the files, add

<appender-ref ref="CONSOLE"/>
to each logger.

Here is a logger for the client-connected event as an example:

<logger name="event.client-connected" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
    <appender-ref ref="CONSOLE"/>
</logger>

Now both the defined file as well as the console should receive these events.

Best regards,
Finn

5

@zingiber/Finn

Nope, Its not working still not getting on console. Please have a look below file which i have made changes in logback.xml

<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
        <pattern>%-30(%d %level)- %msg%n%ex</pattern>
    </encoder>
</appender>

<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${hivemq.log.folder}/hivemq.log</file>
    <append>true</append>

    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- daily rollover -->
        <fileNamePattern>${hivemq.log.folder}/hivemq.%d{yyyy-MM-dd}.log</fileNamePattern>

        <!-- keep 30 days' worth of history -->
        <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder>
        <pattern>%-30(%d %level)- %msg%n%ex</pattern>
    </encoder>
</appender>

<appender name="MIGRATIONS-FILE" class="ch.qos.logback.core.FileAppender">
    <file>${hivemq.log.folder}/migration.log</file>
    <append>true</append>
    <encoder>
        <pattern>%-30(%d %level)- %msg%n%ex</pattern>
    </encoder>
</appender>

<logger name="event.client-connected" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
    <appender-ref ref="CONSOLE"/>
</logger>

<logger name="event.client-disconnected" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
    <appender-ref ref="CONSOLE"/>
</logger>

<logger name="event.message-dropped" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
    <appender-ref ref="CONSOLE"/>
</logger>

<logger name="event.client-session-expired" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
    <appender-ref ref="CONSOLE"/>
</logger>

<logger name="migrations" level="DEBUG" additivity="false">
    <appender-ref ref="MIGRATIONS-FILE"/>
</logger>

<root level="INFO">
    <appender-ref ref="FILE"/>
    <appender-ref ref="CONSOLE"/>
</root>

<logger name="jetbrains.exodus" level="WARN"/>
<logger name="org.eclipse.jetty" level="ERROR"/>
<logger name="com.google.common.util.concurrent.Futures.CombinedFuture" level="OFF"/>
<logger name="oshi" level="ERROR"/>
<logger name="org.jgroups" level="INFO"/>
6

@zingiber/Finn,

Its working now. I have used STDOUT instead of CONSOLE in logback file.

<logger name="event.message-dropped" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
    <appender-ref ref="STDOUT"/>
</logger>

Thank you so much for your support.

1 Like
7

Hey rp85,

glad I could be of assistance!