Geek Answers Handbook

Creating a cluster with Mosquitto broker

I use the Mosquitto broker to implement the MQTT protocol. But I cannot find how clustering can be done in the case of mosquito brokers. There is also a limit on the number of clients that can be served by one broker.

+4
source share
6 answers

You cannot cluster with a mosquito.

Some other MQTT brokers that support clustering, including HiveMQ . HiveMQ has flexible auto-discovery cluster capability and distributed illiterate architecture and works great with cloud providers such as AWS or Azure.

You can see the full list of all brokers that support clustering here .

Disclaimer: I am a HiveMQ developer, so this answer may be biased.

+12
source

Original source of this answer: Horizontal scaling for brokers . I just added plugin support.

To configure the n node cluster, you can use two functions of the combined mosquito broker.

  • Mosquito bridge support.
  • Plugin support - mosquitto-auth-plug ( 1 )

The functionality of the bridge is used ONLY to synchronize messages between all mosquito brokers, while the mosquito-out-plugin can be used to save authorization and ACLs in one database.

When setting up the bridge, pay attention to the use of I / O bridges to avoid forwarding loops ( 2 ). The template has a format

 topic pattern [[[ out | in | both ] qos-level] local-prefix remote-prefix]

Specifying the man page for mosquitto.conf ( 3 ). For incoming topics, the bridge will add the template using the remote prefix and subscribe to the resulting theme of the remote broker. When the corresponding incoming message is received, the deleted prefix will be removed from the topic, and then the local prefix will be added. And vice versa for an outbound topic

An example of mosquitto.conf for a 3 node cluster is given below. To test the load with mqtt-malaria ( 4 ) and, more importantly, the effect on connecting clients with the clean_session flag to false.

  +-------------------+ +------------------> BRIDGE BROKER <------------+ | | 192.168.1.1 | | | +--------^----------+ | | | | | | | | | | Broker A | Broker B | | Broker C | | | +----------------------+--+ +----------------+--------+ +--+----------------------+ | connection A | | connection B | | connection C | | | | | | | | address 192.168.1.1:1883| | address 192.168.1.1:1883| | address 192.168.1.1:1883| | | | | | | | topic # out 2 "" A/ | | topic # out 2 "" B/ | | topic # out 2 "" C/ | | topic # in 2 "" B/ | | topic # in 2 "" A/ | | topic # in 2 "" A/ | | topic # in 2 "" C/ | | topic # in 2 "" C/ | | topic # in 2 "" B/ | +----------------------^--+ +----------------^--------+ +--+----------------------+ | | ^ | | | | | | | +-------+---------+ | +-------------------+ HA PROXY +-------------+ +-----^--^--------+ | | | | + +
+7
source

This is not on the list mentioned by Dominik, but Solace Systems has an MQTT device that supports clustering. Each broker supports several hundred thousand concurrent client connections.

Disclaimer: I'm a system engineer at Solace

+2
source

The way I turned to clustering for mosquitoes was to configure the haproxy interface, which calls proxy servers to several broker brokers.

I also took another step to use DNS SD, which supports the latest mosquitto support with the -S flag.

However, there were some headaches in this design, namely in supporting applications for dnssd and libresolv on most distributions, the domain and search fields in resolv.conf are not respected.

Therefore, if you are working on an outdated environment, I would suggest looking into rabbitmq, hivemq or redis pubsub for a higher level of performance that can be tied to existing fingerprints.

+2
source

There are 2 (for this time of writing) main open source projects supporting large-scale deployments and clustering of MQTT

VerneMQ and EMQ are written in Erlang, which is very convenient for distributed messages, and all replication is built on part of the toolbox (Erlang).

VerneMQ has commercial support for the enterprise, while EMQ plans in the near future, and both projects have large companies supporting them as sponsors.

You can also look at HiveMQ, which is the oldest child in the block, but cannot be used without a paid license.

+1
source

You can watch the Beviwise IoT Platform . This is a commercial offer. This has the opportunity to add as many brokers as possible to increase. It supports the MQTT and MQTT-SN protocols.

Disclaimer: I work for Bevywise networks.

0
source

More articles:


All Articles