• Stars
    star
    909
  • Rank 50,251 (Top 1.0 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created almost 6 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Data Replication Center

Gravity įŽ€äŊ“中文


Build Status

2.0 Product

Gravity is used to replicate data between various inputs and outputs (databases, message queues).

It is designed to be a customizable data replication tool that:

  • Supports multiple sources and destinations

  • Supports Kubernetes-based replication cluster

Use Cases

  • Data Bus: Use Change Data Capture (MySQL binlog, MongoDB Oplog) and batch table scan to publish data to message queue like Kafka.
  • Unidirectional data replication: Replicates data from one MySQL cluster to another MySQL cluster.
  • Bidirectional data replication: Replicates data between two MySQL clusters bidirectionally.
  • Synchronization of shards to the merged table: Synchronizes MySQL sharded tables to the merged table. You can specify the corresponding relationship between the source table and the target table.
  • Online data mutation: Data can be changed during the replication. For example, rename the column, encrypt/decrypt data columns.

Features

Input support

Input Status
MySQL Binlog ✅
MySQL Scan ✅
Mongo Oplog ✅
Mongo Scan ✅
TiDB Binlog Doing
PostgreSQL WAL Doing

output support

Output Status
MySQL/TiDB ✅
Kafka ✅
Elasticsearch Beta

Data mutation support

Gravity supports the following data mutations:

  • Ignore specific data
  • Renaming columns
  • Deleting columns

Documentation


Special thanks to @dantin, @LiangShang, and @liwink for the early support to this project, and thanks @CaitinChen for the initial translation of the documentation.