• Stars
    star
    3,301
  • Rank 13,064 (Top 0.3 %)
  • Language
    C++
  • License
    Other
  • Created about 8 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

The Paxos library implemented in C++ that has been used in the WeChat production environment.

简体中文README

PhxPaxos is a state-synchronization lib based on Paxos protocol, it is totally designed by Wechat independently. It can help your services in synchronizing the state from a single node to the other nodes to form a multi-copy cluster and handling fail-over automatically by calling functions in our lib.

This lib has been used in Wechat production environment, we have also tested it in a large number of harsh environments to guarantee a stable consistency.

Authors: Haochuan Cui, Ming Chen, Junchao Chen and Duokai Huang

Contact us: [email protected]

Principle details(Chinese)

PhxPaxos Build Status

Features

  • Purely based on Paxos Made Simple by Lamport.
  • Transfering message in a async mechanism architecture.
  • Using fsync to guarantee the correctness in every IO write operations.
  • Latency of a successful Propose is one RTT and one IO write.
  • Using P2P stream protocol to learn paxos log.
  • Cleaning Checkpoint and PaxosLog automatically.
  • Pulling Checkpoint across nodes automatically.
  • Supporting more than one state-machines in a single PhxPaxos Instance.
  • Supporting recover checkpoint by snapshot+paxoslog automatically.
  • Implementing Master election as a state-machine embedded in PhxPaxos
  • Implementing reconfiguration as a state-machine embedded in PhxPaxos
  • Using signature algorithm embedded in PhxPaxos to recognise invalid hosts.
  • Using checksum to verifying the data consistency of increment data in realtime.
  • Implementing Network, Stroage, Monitor, Logging module as a plugin, they can be implemented customly
  • Supporting overload protection in a self-adaption way.

Limitations

  • Only a single process (possibly multi-threaded) can run a PhxPaxos instance at a time.
  • There is no client-server support builtin to the library. An application that needs such support will have to wrap their own server around the library.
  • PhxPaxos was only tested on Linux 64bit platform so far.

Performance

Setup

CPU: 24 x Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz
Memory: 2 GB
Disk: ssd raid5
Network: Gigabit Ethernet
Cluster Nodes: 3
Ping: 0.05ms
Parallel client: 100 Threads

Performance Test Result(QPS)

Request latency small than 10ms.

Data set with small size(100B)
1 Group: 1171
20 Groups: 11931
50 Groups: 13424
100 Groups: 13962
Data set with larse size(100KB)
1 Group: 280
20 Groups: 984
50 Groups: 1054
100 Groups: 1067
BatchPropose(2KB)
100 Groups: 150000

Code Directory Introduction

include: This directory includes all head files while using PhxPaxos. You may make some mistakes if you don't understand all the functions in this directory completely.

NOTE: The public interface is in include/*.h. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

src: This directory includes all implementation of Phapaxos, You can figure out the working principle of PhxPaoxs by reading this directorys. No neccessary to read it if you are only using PhxPaxos.

third_party: This directory is designed to place all third party libs for compiling and running PhxPaxos. You can get a detail in the following compilation section. We have only two libs requirement: Protobuf and LevelDB.

plugin: This directory provides the plugin of Logging module. Loggins is an important way to debug a program. But different organazations always implement it in a totally different way. So PhxPaxos does not provide a specific implementation of Logging, instead, it provides the mechanism of Logging so everyone can implement it customly. We also implement a specific Logging by using GLOG for you. If you are using GLOG this may help you in saving your development time.

sample: This directory provides 3 samples based on PhxPaoxs, They representive different depth of using PhxPaxos, from easy to hard.

  • PhxElection: This is a very simple program. It implements a Master Election Program by a Master Election state-machine which is embedded in PhxPaxos.
  • PhxEcho: This shows how to program a status-machine and combine it with PhxPaxos.
  • PhxKV: This is a more complete system. which implements a KV state-machine. It shows how to implement a distributed KV storage system by PhxPaxos and how to delete PaxosLog by implementing the Checkpoint API. It also shows how to combine this code into a RPC(etc: GRPC) framework to get a complete distributed KV storage system.

Guide to Header Files:

  • include/node.h The beginning of PhxPaxos. We strongly suggest you to begin here.
  • include/options.h Some configurations and options needed while running PhxPaxos.
  • include/sm.h A base class of state-machine.
  • include/def.h Sets of return code.
  • include/network.h Abstract function of Network Module. You can use your own network protocol by overloading these functions.
  • include/storage.h Abstract function of Storage Module.
  • include/log.h Abstract function of Logging Module.
  • include/breakpoint.h Abstract function of breaking points, add your ownMonitor Module here to monitor these break points.

How to Compile

Third party libs preparation

Following is a dependency relationship tablet of all directories.

Directories compilation target inner dependency third party dependency
root libphxpaxos.a None protobuf,leveldb
plugin libphxpaxos_plugin.a libphxpaxos.a glog
sample/phxelection Executable program libphxpaxos.a,libphxpaxos_plugin.a None
sample/phxecho Executable program libphxpaxos.a,libphxpaxos_plugin.a None
sample/phxkv Executable program libphxpaxos.a,libphxpaxos_plugin.a grpc
src/ut Unit tests None gtest,gmock

We only need 2 third party libs: Protobuf and Leveldb while compiling PhxPaxos library(target is libphxpaoxs.a).But we need GLOG and GRPC while compiling other directories. All these third party libs should be prepared and palced in our third_party directory before PhxPaxos compilation. You can git clone them by adding --recurse-submodules on just download and link them.

Compilation Enviroment

  • Linux
  • GCC-4.8 or above

Compilation and Installation

How to Complie libphxpaxos.a.

Execute following shell commands in root directory of PhxPaxos

./autoinstall.sh
make
make install
How to Complie libphxpaxos_plugin.a.

Execute following shell commands in plugin directory.

make
make install

How to Wrap Your Own Code Around PhxPaxos.

First choose a single node service.

We will show you this by a PhxEcho service in our sample directory, Echo is a common test functions while writing an RPC service. We will wrap this service's code around PhxPaxos to make Echo into a multi-node service.

First, Assume following is the definition fo PhxEchoServer

class PhxEchoServer
{
public:
    PhxEchoServer();
    ~PhxEchoServer();

    int Echo(const std::string & sEchoReqValue, std::string & sEchoRespValue);
};

Let's wrap this code around PhxPaxos.

Second, implement a state-machine

We now define a PhxEchoSM state-machine which inherit from class StateMachine as the following

class PhxEchoSM : public phxpaxos::StateMachine
{
public:
    PhxEchoSM();

    bool Execute(const int iGroupIdx, const uint64_t llInstanceID, 
            const std::string & sPaxosValue, phxpaxos::SMCtx * poSMCtx);

    const int SMID() const { return 1; }
};

SMID() functions should return an unique identifier since PhxPaxos support more than 1 state-machines at the same time.

Execute() is a state transition function of this state-machine. PhxPaxos guarantees all nodes will execute sPaxosValue in the same order to achieve strong consistency (sPaxosValue is one of input arguments of Execute()). Following is the implementation of this functions:

bool PhxEchoSM :: Execute(const int iGroupIdx, const uint64_t llInstanceID, 
        const std::string & sPaxosValue, SMCtx * poSMCtx)
{
    printf("[SM Execute] ok, smid %d instanceid %lu value %s\n", 
            SMID(), llInstanceID, sPaxosValue.c_str());

    //only commiter node have SMCtx.
    if (poSMCtx != nullptr && poSMCtx->m_pCtx != nullptr)
    {   
        PhxEchoSMCtx * poPhxEchoSMCtx = (PhxEchoSMCtx *)poSMCtx->m_pCtx;
        poPhxEchoSMCtx->iExecuteRet = 0;
        poPhxEchoSMCtx->sEchoRespValue = sPaxosValue;
    }   

    return true;
}

We only print it on the screen as a prove of broadcasting this Echo message.

Here we got a strange class SMCtx, following is the definition.

class SMCtx
{
public:
    SMCtx();
    SMCtx(const int iSMID, void * pCtx);

    int m_iSMID;
    void * m_pCtx;
};

SMCtx is a context argument which is provide by proposer(we will offer more details in the following introduction), transmitted to Execute() function by PhxPaxos and finally callback to proposer.

m_iSMID is related to SMID() function mentioned above.PhxPaxos will transmit this to a specific state-machine which owns the same id.

m_pCtx is a customly context point address provided by proposer.

The context arguments of Execute() functions is a nullptr in all nodes except the one which propose it. Developer should judge if it is NULL while implementing, otherwise it will cause a segment fault.

Following shows the context definition of Echo:

class PhxEchoSMCtx
{
public:
    int iExecuteRet;
    std::string sEchoRespValue;

    PhxEchoSMCtx()
    {   
        iExecuteRet = -1; 
    }   
};

iExecuteRet represents the return code of Execute() execution.

sEchoRespValue represents the sEchoReqValue transmit by Execute().

We finally construct a state-machine and a state transmittion function by classes above.

HINT: What you should do to make a service into a replicated service is to abstract the logic of it. And then implement it in a Execute() fuction. That's all:)

Running PhxPaxos

After the implementation, We will try to run PhxPaxos with PhxEchoSM loaded.

We will do some modifications for EchoServer class first:

class PhxEchoServer
{
public:
    PhxEchoServer(const phxpaxos::NodeInfo & oMyNode, const phxpaxos::NodeInfoList & vecNodeList);
    ~PhxEchoServer();

    int RunPaxos();
    int Echo(const std::string & sEchoReqValue, std::string & sEchoRespValue);

    phxpaxos::NodeInfo m_oMyNode;
    phxpaxos::NodeInfoList m_vecNodeList;

    phxpaxos::Node * m_poPaxosNode;
    PhxEchoSM m_oEchoSM;
};

We add some arguments in construction function, oMyNode represents the information(IP, Port, etc...) of this node. vecNodeList represents informations of all nodes in this cluster. These 2 arguments is pre-defined by PhxPaxos.

Except m_oMyNode and m_vecNodeList, m_oEchoSM is a state-machine we just finished, m_poPaxosNode represents the instance pointer of PhxPaxos running this time.

PhxPaxos instance is actived by executing RunPaxos() function, following is the implementation of this function:

int PhxEchoServer :: RunPaxos()
{
    Options oOptions;

    int ret = MakeLogStoragePath(oOptions.sLogStoragePath);
    if (ret != 0)
    {   
        return ret;
    }   

    //this groupcount means run paxos group count.
    //every paxos group is independent, there are no any communicate between any 2 paxos group.
    oOptions.iGroupCount = 1;

    oOptions.oMyNode = m_oMyNode;
    oOptions.vecNodeInfoList = m_vecNodeList;

    GroupSMInfo oSMInfo;
    oSMInfo.iGroupIdx = 0;
    //one paxos group can have multi state machine.
    oSMInfo.vecSMList.push_back(&m_oEchoSM);
    oOptions.vecGroupSMInfoList.push_back(oSMInfo);

    //use logger_google to print log
    LogFunc pLogFunc;
    ret = LoggerGoogle :: GetLogger("phxecho", "./log", 3, pLogFunc);
    if (ret != 0)
    {   
        printf("get logger_google fail, ret %d\n", ret);
        return ret;
    }   

    //set logger
    oOptions.pLogFunc = pLogFunc;

    ret = Node::RunNode(oOptions, m_poPaxosNode);
    if (ret != 0)
    {   
        printf("run paxos fail, ret %d\n", ret);
        return ret;
    }   

    printf("run paxos ok\n");
    return 0;
}

All arguments and options have been included in Option variable while running PhxPaxos instance.

MakeLogStoragePath() function genereate the path where to storage paxos data and it will be set to oOptions.sLogStoragePath. oOptions.iGroupCount represents how many groups running at the same time. They are identified by GroupIdx (the range is [0, oOptions.iGroupCount) ). Different groups run in a independent space and have no connection with the others groups. Why we put them into a single instance is to make them share the IP/Port.

After configuring all IP/Port informations we will configure the state-machine we just finished.

oOptions.vecGroupSMInfoList is a array of GroupSMInfo class, it represents a list of status-machines corresponding to every specific group.

GroupSMInfo is a class which is used to describe a state-machine info for a specific group. GroupSMInfo.iGroupIdx is the identifier of this group. It will set to be 0 since we have only 1 group.

vecSMList is an array of state-machine class. It represents a list of state-machines which we want to load into PhxPaxos.

Config the Logging functions next, we used GLOG here.

At last, Execute Node::RunNode(oOptions, m_poPaxosNode) to run PhxPaxos. Return code 0 means we have active it successfully and m_poPaxosNode point to this PhxPaxos instance.

Proposing Requests

The following codes shows how to reform Echo() in EchoServer to propose a value in PhxPaxos:

int PhxEchoServer :: Echo(const std::string & sEchoReqValue, std::string & sEchoRespValue)
{
    SMCtx oCtx;
    PhxEchoSMCtx oEchoSMCtx;
    //smid must same to PhxEchoSM.SMID().
    oCtx.m_iSMID = 1;
    oCtx.m_pCtx = (void *)&oEchoSMCtx;

    uint64_t llInstanceID = 0;
    int ret = m_poPaxosNode->Propose(0, sEchoReqValue, llInstanceID, &oCtx);
    if (ret != 0)
    {   
        printf("paxos propose fail, ret %d\n", ret);
        return ret;
    }   

    if (oEchoSMCtx.iExecuteRet != 0)
    {   
        printf("echo sm excute fail, excuteret %d\n", oEchoSMCtx.iExecuteRet);
        return oEchoSMCtx.iExecuteRet;
    }   

    sEchoRespValue = oEchoSMCtx.sEchoRespValue.c_str();

    return 0;
}

First we define a context variable oEchoSMCtx and assigned to oCtx.m_pCtx.

oCtx.m_iSMID set to 1 which corresponding to SMID() above, indicate this request will be executed in the state-machine which SMID is 1.

Then call m_poPaxosNode->Propose with following arguments:

GroupIdx: it indicates which group we will propose this request. The value is 0 here. sEchoReqValue: indicates what we want to propose. llInsanceID: A return arugments we got after proposed successfully, it is global incremented. oCtx: The context variable which will be transmitted to Execute() function.

The propose will return 0 if it was proposed successfully, You can get sEchoRespValue in the context variable.

After all the steps above, we enhanced a Echo service from a single node into a cluster:)

The Running Perfomance

The output of Echo cluster is shown in the following. You can implement it by yourself or just compile sample/phxeco directory to get this program.

We have 3 nodes in this cluster. Following output came from The node which proposed the value:

run paxos ok
echo server start, ip 127.0.0.1 port 11112

please input: <echo req value>
hello phxpaxos:)
[SM Execute] ok, smid 1 instanceid 0 value hello phxpaxos:)
echo resp value hello phxpaxos:)

We listened on 127.0.0.1:11112. Add we sent "hello phxpaxos" as an input. Then Execute() funtion printed [SM Execute] ok... as the code we write above, we also got the same EchoRespValue at the context variable at the same time.

Let's see what happend in the other nodes:

run paxos ok
echo server start, ip 127.0.0.1 port 11113

please input: <echo req value>
[SM Execute] ok, smid 1 instanceid 0 value hello phxpaxos:)

This one listend on 127.0.0.1:11113, The Execute() function also printed "hello phxpaxos".

Got a election feature for your service by using Master Election in PhxPaxos.

Here we want to explain the exact meaning of Master: Master is a special role in a cluster.At any given moment, there is only one node that considers itself as master at most (remember no master exists is legal.).

This is a very pratical. Assume there is a cluster of multi machines and we wish only one machines to serve at any given moment. The common way to achive this is to build up a Zookeeper cluster and implement a distributed lock service. But now dozens of lines of code will help you to implement this feature by using our Master Election feature. You don't any extra big modules now.

Following will show you how to embed Master Election into your own service.

First we construct a election class PhxElection for existing modules.

class PhxElection
{
public:
    PhxElection(const phxpaxos::NodeInfo & oMyNode, const phxpaxos::NodeInfoList & vecNodeList);
    ~PhxElection();

    int RunPaxos();
    const phxpaxos::NodeInfo GetMaster();
    const bool IsIMMaster();

private:
    phxpaxos::NodeInfo m_oMyNode;
    phxpaxos::NodeInfoList m_vecNodeList;
    phxpaxos::Node * m_poPaxosNode;
};

It has two APIs: GetMaster to get current master of this cluster and IsIMMaster indicate whether I(the node) am master now.

Then implement RunPaxos() function:

int PhxElection :: RunPaxos()
{
    Options oOptions;

    int ret = MakeLogStoragePath(oOptions.sLogStoragePath);
    if (ret != 0)
    {   
        return ret;
    }   

    oOptions.iGroupCount = 1;

    oOptions.oMyNode = m_oMyNode;
    oOptions.vecNodeInfoList = m_vecNodeList;

    //open inside master state machine
    GroupSMInfo oSMInfo;
    oSMInfo.iGroupIdx = 0;
    oSMInfo.bIsUseMaster = true;

    oOptions.vecGroupSMInfoList.push_back(oSMInfo);

    ret = Node::RunNode(oOptions, m_poPaxosNode);
    if (ret != 0)
    {   
        printf("run paxos fail, ret %d\n", ret);
        return ret;
    }   

    //you can change master lease in real-time.
    m_poPaxosNode->SetMasterLease(0, 3000);

    printf("run paxos ok\n");
    return 0;
}

The difference between MasterElection and Echo is there is no neccessary to implement your own state-machine this time, instead, you can set oSMInfo.bIsUseMaster to true to enable our embedded MasterElection state-machine.

Then, run Node::RunNode() to get the pointer of PhxPaxos instance. You can set the lease length of Master by using SetMasterLease API at any moment.

Finally, We can get master information from this pointer like following shows:

const phxpaxos::NodeInfo PhxElection :: GetMaster()
{
    //only one group, so groupidx is 0.
    return m_poPaxosNode->GetMaster(0);
}

const bool PhxElection :: IsIMMaster()
{
    return m_poPaxosNode->IsIMMaster(0);
}

Now, every node in the cluster can get master information now by the codes above.

Welcome to have a try and give us your suggestion :)

More Repositories

1

weui

A UI library by WeChat official design team, includes the most useful widgets/modules in mobile web applications.
Less
27,053
star
2

wepy

小程序组件化开发框架
JavaScript
22,396
star
3

ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform
C++
19,310
star
4

mars

Mars is a cross-platform network component developed by WeChat.
C++
17,137
star
5

tinker

Tinker is a hot-fix solution library for Android, it supports dex, library and resources update without reinstall apk.
Java
17,056
star
6

MMKV

An efficient, small mobile key-value storage framework developed by WeChat. Works on Android, iOS, macOS, Windows, and POSIX.
C++
16,913
star
7

APIJSON

🏆 零代码、全功能、强安全 ORM 库 🚀 后端接口和文档零代码,前端(客户端) 定制返回 JSON 的数据和结构。 🏆 A JSON Transmission Protocol and an ORM Library 🚀 provides APIs and Docs without writing any code.
Java
16,681
star
8

vConsole

A lightweight, extendable front-end developer tool for mobile web page.
TypeScript
16,379
star
9

weui-wxss

A UI library by WeChat official design team, includes the most useful widgets/modules.
Less
14,966
star
10

QMUI_Android

提高 Android UI 开发效率的 UI 库
Java
14,336
star
11

rapidjson

A fast JSON parser/generator for C++ with both SAX/DOM style API
C++
13,803
star
12

secguide

面向开发人员梳理的代码安全指南
13,093
star
13

omi

Web Components Framework - Web组件框架
TypeScript
12,926
star
14

VasSonic

VasSonic is a lightweight and high-performance Hybrid framework developed by tencent VAS team, which is intended to speed up the first screen of websites working on Android and iOS platform.
Java
11,779
star
15

matrix

Matrix is a plugin style, non-invasive APM system developed by WeChat.
Java
11,417
star
16

wcdb

WCDB is a cross-platform database framework developed by WeChat.
C
10,509
star
17

xLua

xLua is a lua programming solution for C# ( Unity, .Net, Mono) , it supports android, ios, windows, linux, osx, etc.
C
9,133
star
18

libco

libco is a coroutine library which is widely used in wechat back-end service. It has been running on tens of thousands of machines since 2013.
C++
7,998
star
19

Hippy

Hippy is designed to easily build cross-platform dynamic apps. 👏
C++
7,840
star
20

Shadow

零反射全动态Android插件框架
Java
7,316
star
21

QMUI_iOS

QMUI iOS——致力于提高项目 UI 开发效率的解决方案
Objective-C
7,030
star
22

MLeaksFinder

Find memory leaks in your iOS app at develop time.
Objective-C
5,399
star
23

lemon-cleaner

腾讯柠檬清理是针对macOS系统专属制定的清理工具。主要功能包括重复文件和相似照片的识别、软件的定制化垃圾扫描、可视化的全盘空间分析、内存释放、浏览器隐私清理以及设备实时状态的监控等。重点聚焦清理功能,对上百款软件提供定制化的清理方案,提供专业的清理建议,帮助用户轻松完成一键式清理。
Objective-C
5,188
star
24

kbone

一个致力于微信小程序和 Web 端同构的解决方案
JavaScript
4,742
star
25

libpag

The official rendering library for PAG (Portable Animated Graphics) files that renders After Effects animations natively across multiple platforms.
C++
4,729
star
26

puerts

PUER(普洱) Typescript. Let's write your game in UE or Unity with TypeScript.
C++
4,661
star
27

GT

GT (Great Tit) is a portable debugging tool for bug hunting and performance tuning on smartphones anytime and anywhere just as listening music with Walkman. GT can act as the Integrated Debug Environment by directly running on smartphones.
Java
4,385
star
28

TNN

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
C++
4,297
star
29

westore

小程序项目分层架构
JavaScript
4,216
star
30

tmagic-editor

TypeScript
4,037
star
31

wujie

极致的微前端框架
TypeScript
3,801
star
32

vap

VAP是企鹅电竞开发,用于播放特效动画的实现方案。具有高压缩率、硬件解码等优点。同时支持 iOS,Android,Web 平台。
Objective-C
3,794
star
33

WeFlow

A web developer workflow tool by WeChat team based on tmt-workflow, with cross-platform supported and environment ready.
JavaScript
3,224
star
34

cherry-markdown

✨ A Markdown Editor
JavaScript
3,195
star
35

weui.js

A lightweight javascript library for WeUI.
JavaScript
3,157
star
36

spring-cloud-tencent

Spring Cloud Tencent is a Spring Cloud based Service Governance Framework provided by Tencent.
Java
3,116
star
37

tencent-ml-images

Largest multi-label image database; ResNet-101 model; 80.73% top-1 acc on ImageNet
Python
3,046
star
38

tdesign

Enterprise Design System
Vue
3,010
star
39

VasDolly

Android V1 and V2 Signature Channel Package Plugin
Java
2,999
star
40

FaceDetection-DSFD

腾讯优图高精度双分支人脸检测器
Python
2,863
star
41

PhoenixGo

Go AI program which implements the AlphaGo Zero paper
C++
2,863
star
42

Tendis

Tendis is a high-performance distributed storage system fully compatible with the Redis protocol.
C++
2,837
star
43

behaviac

behaviac is a framework of the game AI development, and it also can be used as a rapid game prototype design tool. behaviac supports the behavior tree, finite state machine and hierarchical task network(BT, FSM, HTN)
C#
2,784
star
44

PocketFlow

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.
Python
2,782
star
45

MSEC

Mass Service Engine in Cluster(MSEC) is opened source by QQ team from Tencent. It is a backend DEV &OPS engine, including RPC,name finding,load balance,monitoring,release and capacity management.
Java
2,745
star
46

phxsql

A high availability MySQL cluster that guarantees data consistency between a master and slaves.
C++
2,463
star
47

OOMDetector

OOMDetector is a memory monitoring component for iOS which provides you with OOM monitoring, memory allocation monitoring, memory leak detection and other functions.
Objective-C++
2,298
star
48

tsf

coroutine and Swoole based php server framework in tencent
PHP
2,179
star
49

tmt-workflow

A web developer workflow used by WeChat team based on Gulp, with cross-platform supported and solutions prepared.
CSS
2,175
star
50

Hardcoder

Hardcoder is a solution which allows Android APP and Android System to communicate with each other directly, solving the problem that Android APP could only use system standard API rather than the hardware resource of system.
C++
2,145
star
51

LKImageKit

A high-performance image framework, including a series of capabilities such as image views, image downloader, memory caches, disk caches, image decoders and image processors.
Objective-C
2,079
star
52

UnLua

A feature-rich, easy-learning and highly optimized Lua scripting plugin for UE.
C++
2,053
star
53

TubeMQ

TubeMQ has been donated to the Apache Software Foundation and renamed to InLong, please visit the new Apache repository: https://github.com/apache/incubator-inlong
2,027
star
54

ObjectDetection-OneStageDet

单阶段通用目标检测器
Python
1,962
star
55

cloudbase-framework

腾讯云开发云原生一体化部署工具 🚀 CloudBase Framework:一键部署,不限框架语言,云端一体化开发,基于Serverless 架构。A front-end and back-end integrated deployment tool. One-click deploy to serverless architecture. https://docs.cloudbase.net/framework/index
JavaScript
1,936
star
56

InjectFix

InjectFix is a hot-fix solution library for Unity
C#
1,933
star
57

TscanCode

A static code analyzer for C++, C#, Lua
C++
1,916
star
58

phxrpc

A simple C++ based RPC framework.
C++
1,905
star
59

soter

A secure and quick biometric authentication standard and platform in Android held by Tencent.
Java
1,902
star
60

phxqueue

A high-availability, high-throughput and highly reliable distributed queue based on the Paxos algorithm.
C++
1,891
star
61

plato

腾讯高性能分布式图计算框架Plato
C++
1,889
star
62

GameAISDK

基于图像的游戏AI自动化框架
C++
1,861
star
63

MedicalNet

Many studies have shown that the performance on deep learning is significantly affected by volume of training data. The MedicalNet project provides a series of 3D-ResNet pre-trained models and relative code.
Python
1,837
star
64

TSW

Tencent Server Web
TypeScript
1,802
star
65

NeuralNLP-NeuralClassifier

An Open-source Neural Hierarchical Multi-label Text Classification Toolkit
Python
1,781
star
66

QMUI_Web

An efficient front-end framework for developers building UI on the web.
JavaScript
1,719
star
67

Biny

Biny is a tiny, high-performance PHP framework for web applications
PHP
1,690
star
68

sluaunreal

lua dev plugin for unreal engine 4 or 5
C++
1,687
star
69

paxosstore

PaxosStore has been deployed in WeChat production for more than two years, providing storage services for the core businesses of WeChat backend. Now PaxosStore is running on thousands of machines, and is able to afford billions of peak TPS.
C++
1,658
star
70

Metis

Metis is a learnware platform in the field of AIOps.
Python
1,644
star
71

CodeAnalysis

Static Code Analysis - 静态代码分析
Python
1,585
star
72

TurboTransformers

a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
C++
1,442
star
73

TencentOS-kernel

腾讯针对云的场景研发的服务器操作系统
1,401
star
74

nohost

基于 Whistle 实现的多账号多环境远程配置及抓包调试平台
JavaScript
1,392
star
75

TBase

TBase is an enterprise-level distributed HTAP database. Through a single database cluster to provide users with highly consistent distributed database services and high-performance data warehouse services, a set of integrated enterprise-level solutions is formed.
C
1,372
star
76

WeDemo

WeDemo为微信团队开源项目,用于帮助微信开发者完成微信登录、微信分享等功能的接入和开发。开发者可参考源代码完成开发,也可以直接将代码应用到自己的App开发中,安全、便捷地在App中实现微信分享、微信登录功能。
Objective-C
1,365
star
77

feflow

🚀 A command line tool aims to improve front-end engineer workflow and standard, powered by TypeScript.
TypeScript
1,354
star
78

GAutomator

Automation for mobile games
Objective-C
1,318
star
79

tdesign-vue-next

A Vue3.x UI components lib for TDesign.
TypeScript
1,316
star
80

flare

Flare是广泛投产于腾讯广告后台的现代化C++开发框架,包含了基础库、RPC、各种客户端等。主要特点为易用性强、长尾延迟低。
C++
1,264
star
81

TFace

A trusty face analysis research platform developed by Tencent Youtu Lab
Python
1,236
star
82

LuaPanda

lua debug and code tools for VS Code
Lua
1,219
star
83

FeatherCNN

FeatherCNN is a high performance inference engine for convolutional neural networks.
C++
1,209
star
84

tdesign-miniprogram

A Wechat MiniProgram UI components lib for TDesign.
HTML
1,084
star
85

tgfx

A lightweight 2D graphics library for rendering texts, geometries, and images with high-performance APIs that work across various platforms.
C++
1,011
star
86

TencentPretrain

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo
Python
985
star
87

RapidView

RapidView is an android ui and lightapp development framework
Java
977
star
88

FAutoTest

A UI automated testing framework for H5 and applets
Python
930
star
89

TencentKona-8

Tencent Kona is a no-cost, production-ready distribution of the Open Java Development Kit (OpenJDK), Long-term support(LTS) with quarterly updates. Tencent Kona serves as the default JDK internally at Tencent Cloud for cloud computing and other Java applications.
Java
909
star
90

tquic

A high-performance, lightweight, and cross-platform QUIC library
Rust
900
star
91

hel

A module federation SDK which is unrelated to tool chain for module consumer. 工具链无关的运行时模块联邦sdk.
JavaScript
888
star
92

tdesign-vue

A Vue.js UI components lib for TDesign.
TypeScript
872
star
93

Pebble

Pebble分布式开发框架
C++
861
star
94

mxflutter

使用 TypeScript/JavaScript 来开发 Flutter 应用的框架。
Dart
834
star
95

Face2FaceTranslator

面对面翻译小程序是微信团队针对面对面沟通的场景开发的流式语音翻译小程序,通过微信同声传译插件提供了语音识别,文本翻译等功能。
JavaScript
822
star
96

tdesign-react

A React UI components lib for TDesign.
TypeScript
787
star
97

LightDiffusionFlow

This extension is developed for AUTOMATIC1111's Stable Diffusion web UI that provides import/export options for parameters.
JavaScript
764
star
98

Real-SR

Real-World Super-Resolution via Kernel Estimation and Noise Injection
Python
753
star
99

DCache

A distributed in-memory NOSQL system based on TARS framework, support LRU algorithm and data persists on back-end database. Users can easily deploy, publish, and scale services on the web interface.
C++
746
star
100

PatrickStar

PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP and democratizes AI for everyone.
Python
741
star