• Stars
    star
    514
  • Rank 86,040 (Top 2 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created over 1 year ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A MQTT broker implementation adopting serverless architecture

BifroMQ

English | 中文简体


BifroMQ is a high-performance, distributed MQTT broker implementation that seamlessly integrates native multi-tenancy support. It is designed to support building large-scale IoT device connections and messaging systems, Currently, it serves as the foundational technology for Baidu AI Cloud IoTCore , a public serverless cloud service.

Features

  • Full support for MQTT 3.1, 3.1.1 (MQTT5 support coming soon) features over TCP, TLS, WS, WSS
  • Native support for multi-tenancy resource sharing and workload isolation
  • Built-in storage engine. Optimized for critical load targeting, no third-party middleware dependencies
  • Extension mechanism for supporting:
    • Authentication/Authorization
    • Runtime Setting
    • Bridging
    • Event
    • System/Tenant-level Monitoring

Documentation

You can view the documentation on the official website BifroMQ Docs .

And you can contribute to the documentation in the GitHub repository bifromq-docs.

Getting Started

Docker

docker run -d --name bifromq -p 1883:1883 bifromq/bifromq:latest

Build from source

Prerequisites

  • JDK 17+
  • Maven 3.5.0+

Get source & Build

Clone the repository to your local workspace:

cd <YOUR_WORKSPACE>
git clone https://github.com/baidu/bifromq bifromq

Navigate to the project root folder and execute the following commands to build the entire project:

cd bifromq
mvn wrapper:wrapper
./mvnw -U clean package

The build output consists of several archive files located under /build/build-bifromq-starters/target/

  • bifromq-<VERSION>-windows-standalone.zip
  • bifromq-<VERSION>-standalone.tar.gz

Running the tests

Execute the following command in the project root folder to run all test cases, including unit tests and integration tests. Note: The tests may take some time to finish

mvn test

Deployment

BifroMQ has three deployment modes: Standalone, Standard Cluster, Independent Workload Cluster

Standalone

The standalone deployment mode is ideal for the development stage or production environments that do not require immediate recovery from downtime.

To start a standalone bifromq server, extract bifromq-xxx-standalone.tar.gz into a directory. The directories are like:

|-bin
|-conf
|-lib
|-plugins

Execute the following command in the bin directory:

start server:

./standalone.sh start // the server process will start in background

stop server:

./standalone.sh stop

The configuration file, 'standalone.yml', is located in the conf directory. Many of the settings are self-described via name. By default, the standalone server will save persistent data in data directory.

Standard Cluster(Will be available publicly soon)

The standard cluster deployment mode is suitable for small to medium-sized production environments that require reliability and scalability. It comprises several fully functional nodes working together as a logical MQTT broker instance, ensuring high availability. You can also scale up the concurrent mqtt connection workload by adding more nodes, while some types of messaging related workload are not horizontal scalable in this mode.

Independent Workload Cluster

The Independent Workload Cluster deployment mode is designed for building large-scale, multi-tenant serverless clusters. In this mode, the cluster consists of several specialized sub-clusters, each focusing on a particular 'independent type' of workload. These sub-clusters work together coherently to form a logical MQTT broker instance. This is the most complex deployment mode and requires additional non-open-sourced building blocks. Feel free to contact us for commercial support.

Discussion

Join our Discord or WeChat group if you are interested in our work.

Discord

BifroMQ Discord server

WeChat group

More Repositories

1

amis

前端低代码框架,通过 JSON 配置就能生成各种页面。
TypeScript
17,235
star
2

uid-generator

UniqueID generator
Java
5,429
star
3

san

A fast, portable, flexible JavaScript component framework
JavaScript
4,708
star
4

lac

百度NLP:分词,词性标注,命名实体识别,词重要性
C++
3,864
star
5

braft

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.
C++
3,499
star
6

dperf

dperf is a DPDK based 100Gbps network performance and load testing software.
C
3,273
star
7

bfs

The Baidu File System.
C++
2,853
star
8

openrasp

🔥Open source RASP solution
C++
2,774
star
9

Familia

A Toolkit for Industrial Topic Modeling
C++
2,638
star
10

AnyQ

FAQ-based Question Answering System
C++
2,584
star
11

sofa-pbrpc

A light-weight RPC implement of google protobuf RPC framework.
C++
2,130
star
12

Senta

Baidu's open-source Sentiment Analysis System.
Python
1,889
star
13

tera

An Internet-Scale Database.
C++
1,887
star
14

bfe-book

In-depth Understanding of BFE《深入理解BFE》(Book for BFE, a CNCF open source project. both in English and in Chinese)
1,212
star
15

BaikalDB

BaikalDB, A Distributed HTAP Database.
C++
1,169
star
16

bigflow

Baidu Bigflow is an interface that allows for writing distributed computing programs and provides lots of simple, flexible, powerful APIs. Using Bigflow, you can easily handle data of any scale. Bigflow processes 4P+ data inside Baidu and runs about 10k jobs every day.
C++
1,142
star
17

DuReader

Baseline Systems of DuReader Dataset
Python
1,133
star
18

DDParser

百度开源的依存句法分析系统
Python
973
star
19

starlight

Java implementation for Baidu RPC, multi-protocol & high performance RPC.
Java
961
star
20

CUP

CUP, common useful python-lib. (Currently, Most popular python lib in baidu). Python 开发底层库, 涵盖util、service(threadpool/generator/executor/cache等等)、logging、monitoring、增强型配置 等等库支持
Python
938
star
21

ICE-BA

C++
700
star
22

NoahV

An efficient front-end application framework based on vue.js
JavaScript
639
star
23

EasyFaaS

EasyFaaS是一个依赖轻、适配性强、资源占用少、无状态且高性能的函数计算服务引擎
Go
620
star
24

Curve

An Integrated Experimental Platform for time series data anomaly detection.
JavaScript
530
star
25

Jprotobuf-rpc-socket

Protobuf RPC是一种基于TCP协议的二进制RPC通信协议的Java实现
Java
516
star
26

fast_rgf

Multi-core implementation of Regularized Greedy Forest
C++
466
star
27

babylon

High-Performance C++ Fundamental Library
C++
457
star
28

Dialogue

Python
444
star
29

Elasticsearch

Baidu Elasticsearch
Java
432
star
30

brcc

BRCC(better remote config center)是一个分布式配置中心,用于统一管理应用服务的配置信息,避免各类资源散落在各个项目中,简化资源配置的维护成本。作为一种轻量级的解决方案,部署简单,同时支持多环境、多版本、多角色的资源管理,可以在不改变应用源码的情况下无缝切换和实时生效配置信息。
Java
390
star
31

Cafe

A powerful test framework for Android
Java
370
star
32

mix-img

A fast mix image javascript tool libary
JavaScript
332
star
33

puck

Puck is a high-performance ANN search engine
Jupyter Notebook
331
star
34

unit-dmkit

C++
327
star
35

galaxy

Galaxy is a cluster management system.
C++
326
star
36

information-extraction

Python
325
star
37

knowledge-driven-dialogue

baseline system of knowledge driven dialogue competition
Python
270
star
38

CarbonGraph

A Swift dependency injection / lookup framework for iOS
Swift
254
star
39

unit-uskit

unit-uskit
C++
251
star
40

BIPlatform

JavaScript
219
star
41

dlock

An effective and reliable Distributed Lock
Java
216
star
42

ins

iNexus, coordinate large scale services
C++
214
star
43

boteye

C++
212
star
44

titan-dex

Java
201
star
45

m-git

MGit 是一款基于 Git 的多仓库管理工具,可以安全的、高效的管理多个 Git 仓库; 适合于在多个仓库中进行关联开发的项目,实现批量的版本管理功能,提高 Git 操作的效率,避免逐个执行 Git 命令带来的误操作风险。
Ruby
166
star
46

Rubik

An Android platform component management tool chain, based on Kotlin language.
Kotlin
154
star
47

common

Common library
C++
132
star
48

go-lib

Go
126
star
49

titan-hotfix

Java
125
star
50

wx2

小程序互转工具
JavaScript
124
star
51

iot-sdk-c

device sdk for baidu IoT Core service, in c. Including MQTT client
C
118
star
52

Youtube-8M

PaddlePaddle models for Youtube-8M Video Understanding Challenge
Python
114
star
53

ar-sdk

DuMix AR SDK for Developer
GLSL
107
star
54

broc

Python
101
star
55

ITEST

Web service interface test framework
97
star
56

ote-stack

OTE-Stack is an edge computing platform for 5G and AI
Go
96
star
57

GPT

Java
87
star
58

redis

Baidu Ksarch Redis - a production solution of redis cluster
87
star
59

san-devtools

Browser developer tools extension for debugging San.
TypeScript
82
star
60

terminator

Service Virtualization
Java
76
star
61

QCompute

QCompute is a Python-based quantum software development kit (SDK). It provides a full-stack programming experience for advanced users via hybrid quantum programming language features and a high-performance simulator.
Python
76
star
62

spring-cloud-baidu

70
star
63

shuttle

A fast computing framework based on Galaxy
C++
64
star
64

iot-edge-sdk-for-iot-parser

C
64
star
65

baidu-iot-samples

C
61
star
66

san-store

Application States Management for San
JavaScript
59
star
67

ARK

Development framework of intelligent operation
Python
57
star
68

san-update

Object immutable update utility for san solution
JavaScript
56
star
69

logcover

轻量级异常日志测试覆盖率度量工具
Python
56
star
70

palo

A fast MPP database for all modern analytics on big data. Powered by Apache Doris(Incubating)
50
star
71

speech-samples

百度语音示例
Java
48
star
72

ntripcaster

C
43
star
73

san-router

Official Router for San
JavaScript
38
star
74

Quanlse

Jupyter Notebook
38
star
75

san-ssr

San SSR framework and utils
TypeScript
37
star
76

dm-kit-php

PHP
36
star
77

boteye_sensor

C
35
star
78

ipipe-agent

Java
33
star
79

OASP

OASP (Online App Status Protocol)
Java
32
star
80

san-composition

JavaScript
30
star
81

duedge-recipes

DuEdge百度边缘网络计算样例代码
JavaScript
27
star
82

paddle-on-k8s-operator

Kubernetes operator for managing the lifecycle of PaddlePaddle job.
Go
24
star
83

baiducloud-sdk-go

Go SDK for Baidu Cloud
Go
24
star
84

san-website

JavaScript
21
star
85

baiduads-sdk

Baidu Ads API SDK
Python
19
star
86

du1906_esp

DUHOME AIOT platform based on du1906 and esp32
C
18
star
87

highflip

HIGHFLIP: An easy way to bridge different federal learning platforms
18
star
88

smartapp-openapi-java

百度智能小程序服务端 OpenAPI SDK for java,是基于小程序服务端 OpenAPI 封装的一套让开发者方便使用的 SDK, 它可以帮开发者减少理解和使用 OpenAPI 的成本, 减少开发者直接调用服务端接口不当而引起的错误, 避免在开发中走弯路。
Java
16
star
89

san-factory

JavaScript
15
star
90

ttm

C
14
star
91

cluster-api-provider-baiducloud

Kubernetes cluster-api for Baidu Cloud
Go
13
star
92

minions

Baidu 100G Chasiss Switch hardware spec
11
star
93

signet

签章系统
JavaScript
10
star
94

sgxray

SGXRay: a bounded verifier for Intel SGX enclaves
C
10
star
95

grafana-tsdb-datasource

JavaScript
9
star
96

iotcore-sdk-java

Java SDK for baidu IoT Core service
Java
9
star
97

bce-fpga-dev-kit

VHDL
8
star
98

iot

for all code about Internet of Things
8
star
99

smartapp-openapi-go

百度智能小程序服务端 OpenAPI SDK for go,是基于小程序服务端 OpenAPI 封装的一套让开发者方便使用的 SDK, 它可以帮开发者减少理解和使用 OpenAPI 的成本, 减少开发者直接调用服务端接口不当而引起的错误, 避免在开发中走弯路。
Go
8
star
100

duedge-cli

DuEdge Command Line
Python
6
star