įŽäŊä¸æ
GravityGravity is used to replicate data between various inputs and outputs (databases, message queues).
It is designed to be a customizable data replication tool that:
-
Supports multiple sources and destinations
-
Supports Kubernetes-based replication cluster
Use Cases
- Data Bus: Use Change Data Capture (MySQL binlog, MongoDB Oplog) and batch table scan to publish data to message queue like Kafka.
- Unidirectional data replication: Replicates data from one MySQL cluster to another MySQL cluster.
- Bidirectional data replication: Replicates data between two MySQL clusters bidirectionally.
- Synchronization of shards to the merged table: Synchronizes MySQL sharded tables to the merged table. You can specify the corresponding relationship between the source table and the target table.
- Online data mutation: Data can be changed during the replication. For example, rename the column, encrypt/decrypt data columns.
Features
Input support
Input | Status |
---|---|
MySQL Binlog | |
MySQL Scan | |
Mongo Oplog | |
Mongo Scan | |
TiDB Binlog | Doing |
PostgreSQL WAL | Doing |
output support
Output | Status |
---|---|
MySQL/TiDB | |
Kafka | |
Elasticsearch | Beta |
Data mutation support
Gravity supports the following data mutations:
- Ignore specific data
- Renaming columns
- Deleting columns
Documentation
Special thanks to @dantin, @LiangShang, and @liwink for the early support to this project, and thanks @CaitinChen for the initial translation of the documentation.