Apache BifroMQ (Incubating)
Apache BifroMQ is a high-performance, distributed MQTT broker that natively supports multi-tenancy. It is designed to enable the building of large-scale IoT device connectivity and messaging systems.
Features
- Full support for MQTT 3.1, 3.1.1 and 5.0 features over TCP, TLS, WS, WSS
- Native support for multi-tenancy resource sharing and workload isolation
- Built-in distributed storage engine optimized for MQTT workloads, with no third-party middleware dependencies.
- Extension mechanism for supporting:
- Authentication/Authorization
- Tenant-level Runtime Setting
- Tenant-level Resource Throttling
- Event
- System/Tenant-level Metrics
Documentation
You can access the documentation on the
official website.
Additionally, contributions to the documentation are welcome in the
GitHub repository.
Getting Started
Docker
docker run -d -m <MEM_LIMIT> -e MEM_LIMIT='<MEM_LIMIT_IN_BYTES>' --name bifromq -p 1883:1883 apache/bifromq:${TAG}
Substitute <MEM_LIMIT> and <MEM_LIMIT_IN_BYTES> with the actual memory allocation for the Docker process, for
example, 2G for <MEM_LIMIT> and 2147483648 for <MEM_LIMIT_IN_BYTES>. If not specified, Apache BifroMQ defaults to using
the hosting server’s physical memory for determining JVM parameters. This can result in the Docker process being
terminated by the host’s Out-of-Memory (OOM) Killer. Refer to here
for more information.
You can build an Apache BifroMQ cluster using Docker Compose on a single host for development and testing. Suppose you want to create a cluster with three nodes: node1,
node2, and node3. The directory structure should be as follows:
|- docker-compose.yml
|- node1
|- node2
|- node3
Each node should have a configuration file, it is defined as follows:
clusterConfig:
env: "Test"
host: bifromq-node1 # Change this to bifromq-node2 for node2 and bifromq-node3 for node3
port: 8899
seedEndpoints: "bifromq-node1:8899,bifromq-node2:8899,bifromq-node3:8899"
The docker-compose.yml file defines the services for the three nodes:
services:
bifromq-node1:
image: apache/bifromq:${TAG}
container_name: bifromq-node1
volumes:
- ./node1/standalone.yml:/home/bifromq/conf/standalone.yml
ports:
- "1883:1883"
environment:
- MEM_LIMIT=2147483648 # Adjust the value according to the actual host configuration.
networks:
- bifromq-net
bifromq-node2:
image: apache/bifromq:${TAG}
container_name: bifromq-node2
volumes:
- ./node2/standalone.yml:/home/bifromq/conf/standalone.yml
ports:
- "1884:1883"
environment:
- MEM_LIMIT=2147483648
networks:
- bifromq-net
bifromq-node3:
image: apache/bifromq:${TAG}
container_name: bifromq-node3
volumes:
- ./node3/standalone.yml:/home/bifromq/conf/standalone.yml
ports:
- "1885:1883"
environment:
- MEM_LIMIT=2147483648
networks:
- bifromq-net
networks:
bifromq-net:
driver: bridge
To launch the cluster, run the following command:
docker compose up -d
Build from source
Prerequisites
- JDK 17+
- Maven 3.5.0+
- (Optional for native TLS) OpenSSL available on the host; if absent, TLS falls back to JDK implementation.
Get Source & Build
Clone the repository to your local workspace:
cd <YOUR_WORKSPACE>
git clone https://github.com/apache/bifromq bifromq
Navigate to the project root folder and execute the following commands to build the entire project:
cd bifromq
./mvnw -v
./mvnw -U clean verify -DskipTests -Pbuild-release
The build output consists of several archive files with sha512 checksum located under /target/output
apache-bifromq-<VERSION>-src.tar.gzapache-bifromq-<VERSION>.tar.gzapache-bifromq-<VERSION>-windows.zip
Native TLS options
- Default binary bundles
netty-tcnative-classes only; if system OpenSSL is unavailable, TLS falls back to JDK TLS automatically.
- To build with system OpenSSL (requires OpenSSL installed on the host):
mvn -Pbuild-release -Pwith-tcnative -Dtcnative.classifier=<your_platform_classifier> clean verify -DskipTests
- Example classifiers:
linux-x86_64, linux-aarch_64, osx-aarch_64, osx-x86_64, windows-x86_64.
- To build with BoringSSL static bundle
mvn -Pbuild-release -Pwith-boringssl-static -Dtcnative.classifier=<your_platform_classifier> clean verify -DskipTests
- Example classifiers:
linux-x86_64, linux-aarch_64, osx-aarch_64, osx-x86_64, windows-x86_64.
Running the tests
Prepare the test environment (compile and download dependencies without running tests):
./mvnw clean install -DskipTests
Run unit tests only:
./mvnw test
Generate coverage (includes unit tests and integration tests, takes longer):
./mvnw test -Pbuild-coverage
Build the Docker Image
With the binary apache-bifromq-<version>.tar.gz and its .sha512 under target/output, build the Docker image using the helper script:
./release/docker-build.sh target/output/apache-bifromq-<version>.tar.gz
Optional: override the tag or target architecture:
./release/docker-build.sh -t apache-bifromq:<version> target/output/apache-bifromq-<version>.tar.gz
./release/docker-build.sh -a arm64 target/output/apache-bifromq-<version>.tar.gz
Quick Start
To quickly set up an Apache BifroMQ server, extract the apache-bifromq-<VERSION>.tar.gz file into a directory. You will see the
following directory structure:
|- bin
|- conf
|- lib
|- plugins
To start or stop the server, execute the respective command in the bin directory:
The configuration file, standalone.yml, can be found in the conf directory. The settings within this file are named
in a self-explanatory manner. By default, the standalone server stores persistent data in the data directory.
Plugin Development
To jump start your Apache BifroMQ plugin development, execute the following Maven command:
mvn archetype:generate \
-DarchetypeGroupId=org.apache.bifromq \
-DarchetypeArtifactId=bifromq-plugin-archetype \
-DarchetypeVersion=<BIFROMQ_VERSION> \
-DgroupId=<YOUR_GROUP_ID> \
-DartifactId=<YOUR_ARTIFACT_ID> \
-Dversion=<YOUR_PROJECT_VERSION> \
-DpluginName=<YOUR_PLUGIN_CLASS_NAME> \
-DpluginContextName=<YOUR_PLUGIN_CONTEXT_CLASS_NAME> \
-DbifromqVersion=<BIFROMQ_VERSION> \
-DinteractiveMode=false
Replace <YOUR_GROUP_ID>, <YOUR_ARTIFACT_ID>, <YOUR_PROJECT_VERSION>, <YOUR_PLUGIN_CLASS_NAME>,
and < YOUR_PLUGIN_CONTEXT_CLASS_NAME> with your specific details. This command generates a ready-to-build multi-module
project structured for Apache BifroMQ plugin development.
Important Note: The archetype version should be 3.2.0 or higher as the archetype is compatible starting from version
3.2.0. Ensure that <BIFROMQ_VERSION> is set accordingly.
Cluster Deployment
BifroMQ has two cluster deployment modes: Standard Cluster, Independent-Workload Cluster
Standard Cluster
The standard cluster deployment mode is suitable for small to medium-sized production environments that require
reliability and scalability. It comprises several fully functional standalone nodes working together as a logical MQTT
broker
instance, ensuring high availability. You can also scale up the concurrent mqtt connection workload by adding more
nodes, while some types of messaging related workload are not horizontally scalable in this mode.
Independent Workload Cluster
The Independent Workload Cluster deployment mode is designed for building large-scale, multi-tenant serverless clusters.
In this mode, the cluster consists of several specialized sub-clusters, each focusing on a particular ‘independent type’
of workload. These sub-clusters work together coherently to form a logical MQTT broker instance.
We welcome you to connect with the Apache BifroMQ community:
- Mailing Lists – Stay informed, discuss development topics, and collaborate with other contributors via our public mailing lists.
- Discord server – Join us to chat, share ideas, and get real-time updates on ongoing work.
ASF Incubator disclaimer
Apache BifroMQ™ is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4)
评论