• This repository has been archived on 17/Dec/2020
  • Stars
    star
    639
  • Rank 70,174 (Top 2 %)
  • Language
    C
  • License
    Apache License 2.0
  • Created almost 10 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

pdo and redis tcp connect proxy

php-cp(php-connect-pool),redis和pdo的本地代理 Build Status

提供连接池,读写分离,负载均衡,慢查询日志,大数据块日志等功能

要求

(目前项目正在维护中,请先不要使用)

  • PHP 5.3 + (no zts)
  • linux 2.6+
  • pdo and redis extension install

使用Docker安装

可以使用Docker编译,需要在项目的根目录下运行:

  1. 根据自己的配置,复制 config.ini.example 文件为 pool.ini 文件,修改 pool.ini 文件
  2. docker build -t php-cp .

技术特性:

  • 提供了release方法,在每次fetch数据后(redis的get set) 调用,将连接放回到池子里面,避免其他耗时操作导致的db层连接数过高问题。
  • 提供最大最小连接数配置支持。
  • 连接自动ping 数据库, 防止压力小长时间不请求导致的gone away问题
  • 根据压力自动获取(最大到最大连接数)或者释放(释放最小到最小连接数)池子里面的连接。
  • 做了大量优化虽然请求经过代理进程转发但基本没有性能损耗.
  • 当池子里面的连接被占用没了,接下来的挣钱连接的进程将会排队,直到持有连接的进程release连接.
  • 使用透明化,相对于传统的pdo和redis操作,只需要修改new的类名,以及适当时机release连接即可(可以集成到db层框架)
  • 支持pdo的读写分离和从库的负载均衡。
  • 支持cli模式下的pdo和redis异步查询。
  • 支持慢查询日志(max_hold_time_to_log)以及大的数据块(max_data_size_to_log)日志功能。

提示

  • 请求结束(rshutdown/mshutdown阶段)会调用自动调用release,不要依赖于这个release,否则连接利用率会很低
  • 关于异常: pdoProxy和原生Pdo不一样的一点是 默认pdo是静默模式 不抛异常 pdoProxy是抛异常的(且是用Exception类抛出的 不是PDOException)
  • pool_server 必须以root用户启动
  • redis不支持pub/sub方法
  • 当你用完一个连接后(例如:fetchAll调用结束),请调用release来马上释放连接到池子里面(如果事务需要在事务commit或者rollback后release),如果不想改业务代码可以在框架层每次fetch(或者get/set)用完之后调用release方法。

集成好的框架

安装使用

  • 安装扩展 安装步骤跟其它PHP扩展无差别
$ phpize && ./configure && make && make install //如果报phpize命令找不到请安装php-devel包
$ echo "extension=connect_pool.so" >> php.ini
//如果PHP启用了目录配置 上一步骤可以换为下面的方式 配置目录可以通过 php --info|grep 'Scan'获取
$ echo "extension=connect_pool.so" > /etc/php.d/20-connection_pool.ini
  • 初始化配置(一次性)
$ cp ./config.ini.example /etc/pool.ini //根据需求修改配置内容
$ mkdir -m 755 /var/log/php-connection-pool //创建日志目录 目录文件夹不存在或没权限会导致日志写不起
$ chmod +x ./pool_server //x权限git已经设置 为稳妥再设置一次 pool_server为php脚本 可自行修改
$ [ -f /bin/env ] || sudo ln -s /usr/bin/env /bin/env  //deb系的系统(如debian、ubuntu)env的路径为/usr/bin/env做软链接兼容处理
$ cp ./pool_server /usr/local/bin/pool_server
  • 日常运维使用
$ pool_server start //启动服务 如果配置文件的daemonize开启则后台运行 否则为前台运行 Ctrl+c结束服务
$ pool_server stop //停止服务
$ pool_server restart //重启服务
$ pool_server status //查看服务状态
  • 日常开发使用
    将该项目源码加入IDE的外部库中, 即可有代码提示, 切记不要加入php配置的include path中
<?php
/* * ****************don't use pool(不用连接池 最原始的方式)************************ */
$obj = new Redis();
$rs = $obj->connect("192.168.20.130");
$obj->select(5);
$obj->set("test", '1111');
var_dump($obj->get("test"));

$obj = new PDO('mysql:host=192.168.20.130;dbname=test1', "admin", "admin");
$rs = $obj->query("show tables");
var_dump($rs->fetchAll());

//*****************use pool(使用了连接池)*********************************/
$obj = new redisProxy();
$rs = $obj->connect("192.168.20.130");
$obj->select(5);
$obj->set("test", '1111');
var_dump($obj->get("test"));
$obj->release();

$obj1 = new pdoProxy('mysql:host=192.168.20.131;dbname=db1', "admin", "admin");
$rs = $obj1->query("show tables");
var_dump($rs->fetchAll());
$obj1->release();


/* * ****************异步 pdo和redis操作**********************************************
 * 依赖 swoole的event函数
 */
include './asyncClass.php';
$obj = new asyncRedisProxy();
$obj->connect("127.0.0.1", "6379");
$obj->set("a", 11111, function($obj, $ret) {
    $obj->get("a", function($obj, $data) {
        var_dump($data);
        $obj->release(); //release to con pool
    });
});


$obj2 = new asyncPdoProxy('mysql:host=192.168.1.19;dbname=mz_db', "public_user", "1qa2ws3ed");
$obj2->query("select 1 from mz_user where user_id=299", function($obj, $stmt) {
    $arr = $stmt->fetchAll();
    var_dump($arr);
    $obj->query("select 2 from mz_user where user_id=299", function($obj, $stmt) {
        $arr = $stmt->fetchAll();
        var_dump($arr);
        $obj->release(); //release to con pool
    });
});


$obj3 = new asyncPdoProxy('mysql:host=192.168.1.19;dbname=mz_db', "public_user", "1qa2ws3ed");
$obj3->exec("insert into t1(name) values('111111')", function($obj, $data) {
    var_dump($data);
    $obj->release(); ////release to con pool
});


$obj4 = new asyncPdoProxy('mysql:host=192.168.1.19;dbname=mz_db', "public_user", "1qa2ws3ed");
$stmt = $obj4->prepare("select * from mz_account where user_id=:user_id");
$stmt->bindParam(':user_id', "311");
$stmt->execute(function($stmt, $ret) {
    $data = $stmt->fetchAll();
    var_dump($data);
    $stmt->release();
});

//*******************use master slave(最新版本支持了读写分离和从库的负载均衡 用法如下)***********************/
$config = array(
    'master' => array(
        'data_source' => "mysql:host=192.168.1.19;dbname=db1",
        'username' => "public_user",
        'pwd' => "1qa2ws3ed",
        'options' => array(
            PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
            PDO::ATTR_TIMEOUT => 3,
            PDO::ATTR_CASE => PDO::CASE_UPPER,
        ),
    ),
    'slave' => array(
        "0" => array(
            'data_source' => "mysql:host=192.168.1.20;dbname=db2",
            'username' => "public_user",
            'pwd' => "1qa2ws3ed",
            'options' => array(
                PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
                PDO::ATTR_TIMEOUT => 3,
                PDO::ATTR_CASE => PDO::CASE_UPPER,
            ),
        ),
        "1" => array(
            'data_source' => "mysql:host=192.168.1.21;dbname=db3",
            'username' => "public_user",
            'pwd' => "1qa2ws3ed",
            'options' => array(
                PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
                PDO::ATTR_TIMEOUT => 3,
                PDO::ATTR_CASE => PDO::CASE_LOWER,
            ),
        ),
    ),
);
/* * *************************"select"和"show"开头的语句 走随机从库********** */
$obj1 = new pdoProxy($config);
$rs = $obj1->query("select * from test limit 1");
var_dump($rs->fetchAll()); //走随机从库
$obj1->release();

/* * **************************读强行走主库*************************** */
$obj1->enable_slave = false;
$rs = $obj1->query("select * from test limit 1");
var_dump($rs->fetchAll()); //读主库
$obj1->release();

/* * *************************除了"select"和"show"开头的语句 都走主库********** */
$sql = "insert into `test` (tid) values (5)";
$rs = $obj1->exec($sql); //走主库
$obj1->release();

测试

运行命令

php tests/RunTest.php --host 172.17.0.2 --class RedisTest --test test_set_get

执行测试,该命令接受三个参数:

  • --host 设置主机地址,可选,默认值为"127.0.0.1"
  • --class 设置要运行的测试类名称,可选,默认值为"RedisTest"
  • --test 设置要运行的测试函数名称,可选,如果不设置则执行所有测试

contact us

More Repositories

1

swoole-src

🚀 Coroutine-based concurrency library for PHP
C++
18,406
star
2

phpx

💗 C++ wrapper for Zend API
C++
823
star
3

docker-swoole

🏄 Official Docker Image of Swoole
Dockerfile
499
star
4

ide-helper

📘 Swoole IDE Helper
PHP
469
star
5

yasd

Yet Another Swoole Debugger
PHP
356
star
6

swoole-wiki

📖Swoole全量Markdown文档, Swoole-Doc, Swoole-Wiki
Shell
270
star
7

phpkafka

PHP Kafka client is used in PHP-FPM and Swoole. PHP Kafka client supports 50 APIs, which might be one that supports the most message types ever.
PHP
266
star
8

awesome-swoole

💎 A curated list of awesome things related to Swoole.
230
star
9

library

📚 Swoole Library
PHP
221
star
10

grpc

💎 Grpc client based on Swoole Coroutine
PHP
199
star
11

ext-zookeeper

🧑 Coroutine-based ZooKeeper Client for PHP
C
150
star
12

swoole-cli

SWOOLE-CLI is a php binary distribution composed swoole & php-core & cli & fpm and mostly of common extensions.
C
119
star
13

community-chinese

Swoole 提案
117
star
14

thrift-rpc-server

Thrift RPC Server based on swoole
PHP
115
star
15

auto_reload

Inotify监控文件变更自动重启swoole_server
PHP
105
star
16

ext-serialize

the fastest serialize function bound for php7
C
93
star
17

ext-async

Asynchronous callback client
PHP
87
star
18

proxy-server

Full asynchronous proxy server can support over a large number of concurrent.
PHP
84
star
19

debugger

Swoole 远程调试器
PHP
79
star
20

ext-postgresql

🐘 Coroutine-based client for PostgreSQL
C++
64
star
21

swoole-docs

📗 Please check the latest version: https://www.swoole.co.uk/docs/
61
star
22

phpy

PHP
43
star
23

zmq

ZeroMQ bindings for Swoole
PHP
37
star
24

dashboard

Vue
22
star
25

php-docker

php docker file
Dockerfile
20
star
26

docs

PHP
10
star
27

benchmark

PHP
9
star
28

make-library

Convert PHP code to C/C++ header file
PHP
6
star
29

documents

2
star
30

swoole.github.io

Swoole's website, docs & blog mainly focused on the english-speaking community.
CSS
2
star
31

v4.4-lts

C++
1
star
32

golang-h2demo

Go
1
star