Setting up a generic MQTT broker as a synchronization destination

Introduction

A generic MQTT broker can be configured as a data destination in IIH Essentials to synchronize live and historical data. Consider the following points before starting the synchronization:

• MQTT version:

  IIH Essentials requires the MQTT broker to support version 3.1 or 3.1.1.

• Data prioritization:

  IIH Essentials synchronizes data for an attribute in chronological order. This means it starts with the oldest but not yet synchronized data and continues until all data up to the current timestamp is synchronized.

• Attribute-based synchronization:

  The outgoing MQTT packages from IIH Essentials will contain data from all the attributes contained in a hierarchy. When creating a hierarchy, it is possible to use assets or aspects that already exist in the asset model of IIH Essentials or create assets or aspects that only exist within this destination hierarchy.

  The outgoing MQTT packages from IIH Essentials only contain data from a single attribute. It is not possible to send data from multiple attributes within the same MQTT package. The synchronization only starts once at least an attribute has been added to the entity. If the synchronization is started for an entity without attributes, no MQTT packages will be sent out.

• Synchronization without a model:

  In contrast to the data synchronization to Senseye or Insights Hub, the synchronization to a generic MQTT broker is done without model synchronization. This means that the model hierarchy will not be synchronized to the data destination. Instead, the hierarchy path of the attribute being synchronized can be sent as part of the topic or payload.

• Payload:

  The payload structure can be specified using the configuration file, please see the section Creating a configuration file for further details on how to create a configuration file. While creating the synchronization destination through IIH Essential's UI, you can indicate further parameters linked to the payload of the MQTT packages. As a whole, the following values were taken as upper limits for the payload:

  • Maximum 10 MB for payload size per package (including headers).

  Note

  The payload size strongly depends on what the payload structure specified in the configuration file looks like.

• Unidirectional synchronization:

  Data is always synchronized in one direction – from IIH Essentials to the generic MQTT broker. For this synchronization with a generic MQTT broker, IIH Essentials is set up as an MQTT client.

Procedure

To set up a generic MQTT broker as a synchronization destination, follow these steps:

1. Create a new data destination and select "Generic MQTT Broker" as the destination type.

   

   Adding a generic MQTT broker as synchronization destination in IIH Essentials

2. Enter the relevant information.

   • "Name": Identifier for the synchronization destination. This name must be unique. It will be used for creating synchronization of entities or attributes.

   • "MQTT Protocol": Which protocol should be used to establish the connection to the broker.

     NOTICE

     For security reasons, it is advisable to use the secure protocols "SSL" or "WebSocket Secure" if the broker supports this.

     • "TCP/MQTT": The broker's URL will be built using tcp://
     • "SSL/TLS/MQTTS": The broker's URL will be built using ssl://
     • "WebSocket": The broker's URL will be built using ws://
     • "WebSocket Secure": The broker's URL will be built using wss://

   • "MQTT Host": Host address of the MQTT broker to which the data should be sent.

   • "MQTT Port": Port through which the MQTT broker will receive the data published from IIH Essentials.

     Note

     This port should be open in the proxy settings, in case that the application is running in an Industrial Edge Device.

   • "MQTT Path": Which connection path should be appended to the broker's URL when connecting. For instance, /mqtt.

     Note

     This is an optional field and is only available when "WebSocket" or "WebSocket Secure" are chosen as MQTT protocols.

   • "MQTT Client ID": Which name should be used as client ID when establishing the connection to the MQTT broker.

     Note

     This is an optional field. If nothing is specified as MQTT client ID, a generated ID will be used when connecting to the MQTT broker. For instance, DataDestination[69ef0711e9994bd6bdedb1138de0421d], where the string between brackets is the internal ID of the data destination created in IIH Essentials.

   • "Bulk Data": When this is set, data points of an attribute are grouped in one MQTT package for the period specified as "Message Publishing Interval". Deactivating this sets the publishing interval to 1 s and QoS to level 0. Therefore, one MQTT package will only contain one single datapoint of an attribute. This causes a higher resource consumption.

   • "Message Publishing Interval (s)": How often the data of the attributes should be sent out. This interval will define the time span of an attribute contained within a single MQTT package. Since one MQTT package is sent out per attribute it is recommended to have relatively large message publishing interval. Depending on the amount of values being synchronized, value ranges between 1-10 minutes would be beneficial.

     Note

     This field is only available if "Bulk Data" is set.

   • "QOS": MQTT Quality of Service for time series data, birth message and last will and testament message.

     Note

     This field is only available if "Bulk Data" is set.

   • "Maximum MQTT Packages per Second": Amount of single MQTT packages containing data of a single attribute that can be sent within a second. If the maximum quota is reached, it will waited until the next period of one second starts.

   • "Maximum MQTT Package Size in KB": The maximum size of an MQTT package after the payload has been created based on the configuration file uploaded by the user. This size includes the headers needed for MQTT as well. The maximum package size is limited to 10 MB.

     Note

     This field is only available if "Bulk Data" is set.

   • "Name Replacement": replaces names of entities or attributes by regular expressions. See name replacement for more details.

3. Create the configuration file following the steps portrayed in Creating a configuration file, save it as a JSON file and upload it into IIH Essentials.

4. Choose the desired authentication method to establish a connection with the MQTT broker.

   Please make sure to select the MQTT protocol "(using the selector MQTT Protocol)" that matches the authentication method chosen here and the configuration of the MQTT broker.

   Four authentication methods are supported:

   

   Authentication methods of a generic MQTT broker as synchronization destination in IIH Essentials
   • "No authentication": Neither username nor certificates will be used to establish the connection to the MQTT broker.
   • "User Credentials": The given username and password will be used to connect to the MQTT broker.
   • "Client Certificate": X.509 certificates based authentication will be used to connect to the MQTT broker.
   • "User Credentials with Client Certificate": X.509 certificates based authentication will be combined with user credentials to connect to the MQTT broker.

   Both the username and password will be required if "User Credentials" or "User Credentials with Client Certificate" are chosen as the authentication method. If your broker does not require a password, type in an arbitrary password in the password field, e.g. dummypassword. If "Client Certificate" or "User Credentials with Client Certificate" are chosen as the authentication method, the client certificate, client key and the CA certificate will be required fields.

5. More advanced settings can be accessed by activating the "Advanced Settings" option.

   

   Advanced settings of a generic MQTT broker as synchronization destination in IIH Essentials

   The advanced settings will always be used for establishing a connection to the MQTT broker. With the slider the user can decide whether he prefers to use the default values or specify them through the UI. The following default values are used then the slider is set to inactive:

   • "Retained Flag": Active. This property will be used for the time series data, birth message and last will and testament.
   • "Clean Session Flag": Inactive
   • "Keep Alive Interval": 60 seconds
   • "MQTT Connection Timeout": 15 seconds
   • "Birth Message": Inactive. Therefore, the birth message topic and birth message content will be left empty. This message will be sent whenever a connection is established between the client, i.e., IIH Essentials, and the MQTT broker.
   • "Last Will and Testament": Inactive. Therefore, the last will topic and last will message will be left empty. This message will be sent if the client, i.e., IIH Essentials, disconnects unexpectedly.

   When the advanced settings are active it is possible to leave some optional fields empty and fill out those of interest. The optional values that were left empty will be replaced by the default values that were portrayed above. Mandatory values are represented using an asterisk right after the property name.

6. Enable the data destination.

7. Click "Save".

Result

The added MQTT broker is now available as a data destination and can be specified as the storage location. You can now build a destination hierarchy for this data destination.

Status of a Generic MQTT Broker as synchronization destination in IIH Essentials