• Stars
    star
    1,803
  • Rank 25,652 (Top 0.6 %)
  • Language
    TypeScript
  • License
    Other
  • Created over 6 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Tencent Server Web

Tencent Server Web 2.0

license Build Status tested with jest codecov

What is it

Tencent Server Web(TSW) 是一套面向 WEB 前端开发者,以提升问题定位效率为初衷,提供 染色抓包全息日志 的 Node.js 基础设施。TSW 关注业务的运维监控能力,适用于 http、https 协议的业务场景,可无缝与现有应用(Koa、Express)进行整合。

TSW 2.0 在 1.0 的基础上抽丝剥茧,辅以现代化的设计模式,去除了 1.0 中的大量糟粕,同时对容器化、云原生更加友好。做到了无侵入、低成本接入。

Highlights

🚀

0 侵入

📒

全息日志

🛠

请求抓包

通过 Hack NodeJS 底层代码实现功能。对原有业务代码 0 侵入。 按照请求聚类的显微镜级别的全息日志,给开发者完美的现场还原。 可抓取 Server 端向外部发送的所有请求的完整包体内容,与后台沟通再无障碍。

特别提醒

TSW 1.0 迁移到 2.0 需要注意的地方

  • TSW 1.0 服务包含进程管理,日志管理等多项内置模块,在进行往 2.0 迁移的时候,请选择其他组件完成替代,如引入 PM2 管理 Node 进程,使用 winston 进行日志管理
  • 另外 2.0 没有包含日志清理工具,建议选择 winston-rotating-file 类似工具来完成清理,避免日志存放过多,导致磁盘空间不足

Quick Start

1. 安装

npm install --save @tswjs/tsw
// yarn add @tswjs/tsw

2. 添加配置文件

配置文件是 TSW 启动时加载进运行时的配置文件,主要声明需要使用的 插件 列表。默认会加载项目根目录下的 tswconfig.js 文件,也可以通过启动参数 -c 或者 --config 来手动指定配置文件路径。

注意事项: 2.0 中没有集成开放平台相关逻辑,而是封装成了一个插件让用户按需使用,详情见插件章节。

配置文件示例:

module.exports = {
  plugins: [
    new MyPlugin({})
  ]
}

参数列表:

Name Type default Optional Description
plugins Array<Plugin> - yes 插件列表
cleanLog boolean false yes 是否关闭默认打印
logLevel DEBUG/INFO/WARN/ERROR DEBUG yes 设置 log level
winstonTransports Array<TransportStream> - yes Winston日志通道

3. 启动

npx @tswjs/tsw ./index.js

注意事项:原先 node --inspect ./index.js 中的 CLI 参数如 --inspect 需要转化为环境变量 NODE_OPTIONS 来执行,如 NODE_OPTIONS="--inspect" npx @tswjs/tsw ./index.js

使用 ts: 在保证项目有 ts-node 依赖包的情况下,按照如下方式执行即可直接加载 ts 文件。

NODE_OPTIONS="--require=ts-node/register" npx @tswjs/tsw ./index.ts

CLI (Command Line Interface)

使用 npx @tswjs/tsw --help 来获取 CLI 选项。

Examples

我们提供了一些示例项目以让大家尽快了解该项目。

  1. cd ~
  2. git clone https://github.com/Tencent/TSW.git
  3. cd TSW

Koa

  1. cd examples/koa
  2. yarn
  3. yarn serve 或者 npm run serve
  4. curl -v localhost:4443/path/to/foo -X POST -d "hello, server"

Plugins

插件是什么?

TSW 核心的实现方式是 Hack NodeJS 自身的 http.request 以及 http.createServer, 以此来实现抓包机制。在服务器处理请求的前后,在服务器向其他服务器发包的前后,等等,都会有相应的事件抛出,以供用户来进行自定义处理。为了让用户更加方便地复用、传播这样一组组自定义处理,我们将他们抽象出来,形成了插件机制。

一个最简单的插件

export.modules = class MyPlugin() {
  constructor() {
    this.name = "MyPlugin"
  }

  async init(eventBus, config) {
    eventBus.on("RESPONSE_CLOSE", (payload) => {
      console.log(payload);
    })
  }
}

init 方法是必须的,这个方法在插件加载开始时会被调用,可以是同步也可以是异步。

eventBus

eventBus 是通过 new EventEmitter() 得到的。TSW 核心会在各个关键时机触发上面的事件。

key 含义(触发时机) payload
DNS_LOOKUP_SUCCESS 在每次 DNS 查询成功之后触发 string | dns.LookupAddress[]
DNS_LOOKUP_ERROR 在每次 DNS 查询失败之后触发 NodeJS.ErrorException
RESPONSE_START 在每次服务器开始返回响应(执行 writeHead)时触发 ResponseEventPayload
RESPONSE_FINISH 在响应结束时(res.on("finish"))触发 ResponseEventPayload
RESPONSE_CLOSE 在底层链接关闭时 (res.on("close"))触发 ResponseEventPayload
REQUEST_START 在每次服务器接受到新的请求时触发 RequestEventPayload

Open Platform

在默认的情况下,TSW 只是会把所有的日志和抓包内容抓取到并且送到事件总线上,以供 插件 消费。所以将日志和抓包内容落地查看一般需要用户自己编写插件以及提供存储,使用成本过于高昂。
因此,TSW 官方提供了公共的服务平台(https://tswjs.org),让用户低成本、更快、更方便地使用 TSW 的特性,详情见 开放平台使用指引

Cluster

TSW 2.0 是面对容器化和云原生设计的,所以没有内置 Cluster 相关功能,推荐直接使用容器的健康检查来完成服务的无损重启和故障重启机制。对于没有使用容器化方案的场景来说,我们推荐使用 pm2 类似工具来实现多进程模式。

pm2

使用 Ecosystem File

// ecosystem.config.json

{
  "apps": [
    {
      "name": "app-name",
      "script": "built/index.js",
      "interpreter": "node",
      "interpreter_args": "./node_modules/@tswjs/tsw/dist/cli.js",
      // other options
    }
  ]
}
// package.json

{
  ...
  "scripts": {
    "start": "pm2 start ecosystem.config.json"
  },
  ...
}

Winston

winston 是什么?

winston 是一个通用且轻量的日志包。winston 支持多个日志通道,并且可以分别定义日志优先级。除了内置的三个日志传输通道ConsoleFileHTTP,在 Winston 项目外部还会维护一些传输模块。查看 winston 官方文档

TSW 2.0 支持使用 winston 传输通道记录日志信息,用户在配置文件中可以添加 winston.transports 实例,日志会落到对应配置中。

一个简单的示例

使用 winston 记录 error 级别 以及 debug 级别以下的日志信息到对应文件中,当前 config 文件配置如下:

module.exports = {
  winstonTransports: [
    new winston.transports.File({ filename: 'error.log', level: 'error'}),
    new winston.transports.File({ filename: 'debug.log', level: 'debug'})
  ]
}

日志记录

log

Users

tsw    tsw    tsw     
 
tsw    tsw    tsw      
 
tsw    tsw    tsw      
 
tsw    tsw    tsw      
 
tsw    tsw    tsw      
 
tsw    tsw    tsw      
 
tsw    tsw    tsw      
 
tsw    tsw    tsw      
 
tsw    tsw    tsw

License

Tencent Server Web 的开源协议为 MIT, 详情参见 LICENSE

More Repositories

1

weui

A UI library by WeChat official design team, includes the most useful widgets/modules in mobile web applications.
Less
27,140
star
2

wepy

小程序组件化开发框架
JavaScript
22,491
star
3

ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform
C++
19,861
star
4

mars

Mars is a cross-platform network component developed by WeChat.
C++
17,249
star
5

MMKV

An efficient, small mobile key-value storage framework developed by WeChat. Works on Android, iOS, macOS, Windows, and POSIX.
C++
17,138
star
6

tinker

Tinker is a hot-fix solution library for Android, it supports dex, library and resources update without reinstall apk.
Java
17,137
star
7

APIJSON

🏆 实时 零代码、全功能、强安全 ORM 库 🚀 后端接口和文档零代码,前端(客户端) 定制返回 JSON 的数据和结构 🏆 Real-Time coding-free, powerful and secure ORM 🚀 providing APIs and Docs without coding by Backend, and the returned JSON of API can be customized by Frontend(Client) users
Java
17,052
star
8

vConsole

A lightweight, extendable front-end developer tool for mobile web page.
TypeScript
16,716
star
9

weui-wxss

A UI library by WeChat official design team, includes the most useful widgets/modules.
Less
15,070
star
10

QMUI_Android

提高 Android UI 开发效率的 UI 库
Java
14,423
star
11

rapidjson

A fast JSON parser/generator for C++ with both SAX/DOM style API
C++
14,163
star
12

secguide

面向开发人员梳理的代码安全指南
13,203
star
13

omi

Web Components Framework - Web组件框架
TypeScript
13,001
star
14

VasSonic

VasSonic is a lightweight and high-performance Hybrid framework developed by tencent VAS team, which is intended to speed up the first screen of websites working on Android and iOS platform.
Java
11,801
star
15

matrix

Matrix is a plugin style, non-invasive APM system developed by WeChat.
Java
11,544
star
16

wcdb

WCDB is a cross-platform database framework developed by WeChat.
C
10,655
star
17

xLua

xLua is a lua programming solution for C# ( Unity, .Net, Mono) , it supports android, ios, windows, linux, osx, etc.
C
9,296
star
18

libco

libco is a coroutine library which is widely used in wechat back-end service. It has been running on tens of thousands of machines since 2013.
C++
8,223
star
19

Hippy

Hippy is designed to easily build cross-platform dynamic apps. 👏
C++
7,955
star
20

Shadow

零反射全动态Android插件框架
Java
7,382
star
21

QMUI_iOS

QMUI iOS——致力于提高项目 UI 开发效率的解决方案
Objective-C
7,085
star
22

MLeaksFinder

Find memory leaks in your iOS app at develop time.
Objective-C
5,417
star
23

lemon-cleaner

腾讯柠檬清理是针对macOS系统专属制定的清理工具。主要功能包括重复文件和相似照片的识别、软件的定制化垃圾扫描、可视化的全盘空间分析、内存释放、浏览器隐私清理以及设备实时状态的监控等。重点聚焦清理功能,对上百款软件提供定制化的清理方案,提供专业的清理建议,帮助用户轻松完成一键式清理。
Objective-C
5,402
star
24

libpag

The official rendering library for PAG (Portable Animated Graphics) files that renders After Effects animations natively across multiple platforms.
C++
4,943
star
25

puerts

PUER(普洱) Typescript. Let's write your game in UE or Unity with TypeScript.
C++
4,902
star
26

kbone

一个致力于微信小程序和 Web 端同构的解决方案
JavaScript
4,772
star
27

GT

GT (Great Tit) is a portable debugging tool for bug hunting and performance tuning on smartphones anytime and anywhere just as listening music with Walkman. GT can act as the Integrated Debug Environment by directly running on smartphones.
Java
4,387
star
28

TNN

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
C++
4,381
star
29

westore

小程序项目分层架构
JavaScript
4,237
star
30

tmagic-editor

TypeScript
4,190
star
31

wujie

极致的微前端框架
TypeScript
4,023
star
32

vap

VAP是企鹅电竞开发,用于播放特效动画的实现方案。具有高压缩率、硬件解码等优点。同时支持 iOS,Android,Web 平台。
Objective-C
3,884
star
33

cherry-markdown

✨ A Markdown Editor
JavaScript
3,505
star
34

phxpaxos

The Paxos library implemented in C++ that has been used in the WeChat production environment.
C++
3,351
star
35

WeFlow

A web developer workflow tool by WeChat team based on tmt-workflow, with cross-platform supported and environment ready.
JavaScript
3,224
star
36

spring-cloud-tencent

Spring Cloud Tencent is a Spring Cloud based Service Governance Framework provided by Tencent.
Java
3,171
star
37

weui.js

A lightweight javascript library for WeUI.
JavaScript
3,165
star
38

tdesign

Enterprise Design System
Vue
3,139
star
39

tencent-ml-images

Largest multi-label image database; ResNet-101 model; 80.73% top-1 acc on ImageNet
Python
3,051
star
40

VasDolly

Android V1 and V2 Signature Channel Package Plugin
Java
3,048
star
41

HunyuanDiT

Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
Python
2,945
star
42

Tendis

Tendis is a high-performance distributed storage system fully compatible with the Redis protocol.
C++
2,919
star
43

FaceDetection-DSFD

腾讯优图高精度双分支人脸检测器
Python
2,885
star
44

PhoenixGo

Go AI program which implements the AlphaGo Zero paper
C++
2,871
star
45

behaviac

behaviac is a framework of the game AI development, and it also can be used as a rapid game prototype design tool. behaviac supports the behavior tree, finite state machine and hierarchical task network(BT, FSM, HTN)
C#
2,829
star
46

PocketFlow

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.
Python
2,783
star
47

MSEC

Mass Service Engine in Cluster(MSEC) is opened source by QQ team from Tencent. It is a backend DEV &OPS engine, including RPC,name finding,load balance,monitoring,release and capacity management.
Java
2,746
star
48

phxsql

A high availability MySQL cluster that guarantees data consistency between a master and slaves.
C++
2,470
star
49

OOMDetector

OOMDetector is a memory monitoring component for iOS which provides you with OOM monitoring, memory allocation monitoring, memory leak detection and other functions.
Objective-C++
2,312
star
50

tsf

coroutine and Swoole based php server framework in tencent
PHP
2,179
star
51

tmt-workflow

A web developer workflow used by WeChat team based on Gulp, with cross-platform supported and solutions prepared.
CSS
2,173
star
52

UnLua

A feature-rich, easy-learning and highly optimized Lua scripting plugin for UE.
C++
2,169
star
53

Hardcoder

Hardcoder is a solution which allows Android APP and Android System to communicate with each other directly, solving the problem that Android APP could only use system standard API rather than the hardware resource of system.
C++
2,155
star
54

LKImageKit

A high-performance image framework, including a series of capabilities such as image views, image downloader, memory caches, disk caches, image decoders and image processors.
Objective-C
2,080
star
55

GameAISDK

基于图像的游戏AI自动化框架
C++
2,030
star
56

TubeMQ

TubeMQ has been donated to the Apache Software Foundation and renamed to InLong, please visit the new Apache repository: https://github.com/apache/incubator-inlong
2,022
star
57

phxrpc

A simple C++ based RPC framework.
C++
1,974
star
58

TscanCode

A static code analyzer for C++, C#, Lua
C++
1,972
star
59

ObjectDetection-OneStageDet

单阶段通用目标检测器
Python
1,966
star
60

InjectFix

InjectFix is a hot-fix solution library for Unity
C#
1,961
star
61

cloudbase-framework

腾讯云开发云原生一体化部署工具 🚀 CloudBase Framework:一键部署,不限框架语言,云端一体化开发,基于Serverless 架构。A front-end and back-end integrated deployment tool. One-click deploy to serverless architecture. https://docs.cloudbase.net/framework/index
JavaScript
1,937
star
62

soter

A secure and quick biometric authentication standard and platform in Android held by Tencent.
Java
1,919
star
63

phxqueue

A high-availability, high-throughput and highly reliable distributed queue based on the Paxos algorithm.
C++
1,899
star
64

plato

腾讯高性能分布式图计算框架Plato
C++
1,895
star
65

MedicalNet

Many studies have shown that the performance on deep learning is significantly affected by volume of training data. The MedicalNet project provides a series of 3D-ResNet pre-trained models and relative code.
Python
1,888
star
66

NeuralNLP-NeuralClassifier

An Open-source Neural Hierarchical Multi-label Text Classification Toolkit
Python
1,807
star
67

sluaunreal

lua dev plugin for unreal engine 4 or 5
C++
1,734
star
68

QMUI_Web

An efficient front-end framework for developers building UI on the web.
JavaScript
1,720
star
69

Biny

Biny is a tiny, high-performance PHP framework for web applications
PHP
1,687
star
70

Metis

Metis is a learnware platform in the field of AIOps.
Python
1,682
star
71

paxosstore

PaxosStore has been deployed in WeChat production for more than two years, providing storage services for the core businesses of WeChat backend. Now PaxosStore is running on thousands of machines, and is able to afford billions of peak TPS.
C++
1,665
star
72

CodeAnalysis

Static Code Analysis - 静态代码分析
Python
1,639
star
73

MimicMotion

High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance
Python
1,475
star
74

TurboTransformers

a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
C++
1,464
star
75

tdesign-vue-next

A Vue3.x UI components lib for TDesign.
TypeScript
1,428
star
76

nohost

基于 Whistle 实现的多账号多环境远程配置及抓包调试平台
JavaScript
1,416
star
77

TencentOS-kernel

腾讯针对云的场景研发的服务器操作系统
1,408
star
78

TBase

TBase is an enterprise-level distributed HTAP database. Through a single database cluster to provide users with highly consistent distributed database services and high-performance data warehouse services, a set of integrated enterprise-level solutions is formed.
C
1,380
star
79

WeDemo

WeDemo为微信团队开源项目,用于帮助微信开发者完成微信登录、微信分享等功能的接入和开发。开发者可参考源代码完成开发,也可以直接将代码应用到自己的App开发中,安全、便捷地在App中实现微信分享、微信登录功能。
Objective-C
1,371
star
80

feflow

🚀 A command line tool aims to improve front-end engineer workflow and standard, powered by TypeScript.
TypeScript
1,360
star
81

GAutomator

Automation for mobile games
Objective-C
1,331
star
82

flare

Flare是广泛投产于腾讯广告后台的现代化C++开发框架,包含了基础库、RPC、各种客户端等。主要特点为易用性强、长尾延迟低。
C++
1,308
star
83

TFace

A trusty face analysis research platform developed by Tencent Youtu Lab
Python
1,289
star
84

LuaPanda

lua debug and code tools for VS Code
Lua
1,235
star
85

FeatherCNN

FeatherCNN is a high performance inference engine for convolutional neural networks.
C++
1,208
star
86

tdesign-miniprogram

A Wechat MiniProgram UI components lib for TDesign.
HTML
1,193
star
87

tgfx

A lightweight 2D graphics library for rendering texts, geometries, and images with high-performance APIs that work across various platforms.
C++
1,057
star
88

tquic

A high-performance, lightweight, and cross-platform QUIC library
Rust
1,020
star
89

TencentPretrain

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo
Python
1,005
star
90

RapidView

RapidView is an android ui and lightapp development framework
Java
979
star
91

hel

A module federation SDK which is unrelated to tool chain for module consumer. 工具链无关的运行时模块联邦sdk.
JavaScript
959
star
92

FAutoTest

A UI automated testing framework for H5 and applets
Python
932
star
93

TencentKona-8

Tencent Kona is a no-cost, production-ready distribution of the Open Java Development Kit (OpenJDK), Long-term support(LTS) with quarterly updates. Tencent Kona serves as the default JDK internally at Tencent Cloud for cloud computing and other Java applications.
Java
928
star
94

tdesign-vue

A Vue.js UI components lib for TDesign.
TypeScript
899
star
95

Pebble

Pebble分布式开发框架
C++
864
star
96

mxflutter

使用 TypeScript/JavaScript 来开发 Flutter 应用的框架。
Dart
857
star
97

Face2FaceTranslator

面对面翻译小程序是微信团队针对面对面沟通的场景开发的流式语音翻译小程序,通过微信同声传译插件提供了语音识别,文本翻译等功能。
JavaScript
836
star
98

tdesign-react

A React UI components lib for TDesign.
TypeScript
821
star
99

LightDiffusionFlow

This extension is developed for AUTOMATIC1111's Stable Diffusion web UI that provides import/export options for parameters.
JavaScript
798
star
100

Real-SR

Real-World Super-Resolution via Kernel Estimation and Noise Injection
Python
769
star