• Stars
    star
    431
  • Rank 98,240 (Top 2 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created about 7 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

pytextclassifier is a toolkit for text classification. 文本分类,LR,Xgboost,TextCNN,FastText,TextRNN,BERT等分类模型实现,开箱即用。

PyTextClassifier: Python Text Classifier

PyPI version Downloads Contributions welcome GitHub contributors License Apache 2.0 python_vesion GitHub issues Wechat Group

Introduction

PyTextClassifier: Python Text Classifier. It can be applied to the fields of sentiment polarity analysis, text risk classification and so on, and it supports multiple classification algorithms and clustering algorithms.

pytextclassifier is a python Open Source Toolkit for text classification. The goal is to implement text analysis algorithm, so to achieve the use in the production environment.

文本分类器,提供多种文本分类和聚类算法,支持句子和文档级的文本分类任务,支持二分类、多分类、多标签分类、多层级分类和Kmeans聚类,开箱即用。python3开发。

Guide

Feature

pytextclassifier has the characteristics of clear algorithm, high performance and customizable corpus.

Functions:

Classifier

  • LogisticRegression
  • Random Forest
  • Decision Tree
  • K-Nearest Neighbours
  • Naive bayes
  • Xgboost
  • Support Vector Machine(SVM)
  • TextCNN
  • TextRNN
  • Fasttext
  • BERT

Cluster

  • MiniBatchKmeans

While providing rich functions, pytextclassifier internal modules adhere to low coupling, model adherence to inert loading, dictionary publication, and easy to use.

Install

  • Requirements and Installation
pip3 install torch # conda install pytorch
pip3 install pytextclassifier

or

git clone https://github.com/shibing624/pytextclassifier.git
cd pytextclassifier
python3 setup.py install

Usage

Text Classifier

English Text Classifier

Including model training, saving, predict, evaluate, for example examples/lr_en_classification_demo.py:

import sys

sys.path.append('..')
from pytextclassifier import ClassicClassifier

if __name__ == '__main__':
    m = ClassicClassifier(output_dir='models/lr', model_name_or_model='lr')
    # ClassicClassifier support model_name:lr, random_forest, decision_tree, knn, bayes, svm, xgboost
    print(m)
    data = [
        ('education', 'Student debt to cost Britain billions within decades'),
        ('education', 'Chinese education for TV experiment'),
        ('sports', 'Middle East and Asia boost investment in top level sports'),
        ('sports', 'Summit Series look launches HBO Canada sports doc series: Mudhar')
    ]
    # train and save best model
    m.train(data)
    # load best model from model_dir
    m.load_model()
    predict_label, predict_proba = m.predict([
        'Abbott government spends $8 million on higher education media blitz'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')

    test_data = [
        ('education', 'Abbott government spends $8 million on higher education media blitz'),
        ('sports', 'Middle East and Asia boost investment in top level sports'),
    ]
    acc_score = m.evaluate_model(test_data)
    print(f'acc_score: {acc_score}')

output:

ClassicClassifier instance (LogisticRegression(fit_intercept=False), stopwords size: 2438)
predict_label: ['education'], predict_proba: [0.5378236358492112]
acc_score: 1.0

Chinese Text Classifier(中文文本分类)

Text classification compatible with Chinese and English corpora.

example examples/lr_classification_demo.py

import sys

sys.path.append('..')
from pytextclassifier import ClassicClassifier

if __name__ == '__main__':
    m = ClassicClassifier(output_dir='models/lr-toy', model_name_or_model='lr')
    # 经典分类方法,支持的模型包括:lr, random_forest, decision_tree, knn, bayes, svm, xgboost
    data = [
        ('education', '名师指导托福语法技巧:名词的复数形式'),
        ('education', '中国高考成绩海外认可 是“狼来了”吗?'),
        ('education', '公务员考虑越来越吃香,这是怎么回事?'),
        ('sports', '图文:法网孟菲尔斯苦战进16强 孟菲尔斯怒吼'),
        ('sports', '四川丹棱举行全国长距登山挑战赛 近万人参与'),
        ('sports', '米兰客场8战不败国米10年连胜'),
    ]
    m.train(data)
    print(m)
    # load best model from model_dir
    m.load_model()
    predict_label, predict_proba = m.predict(['福建春季公务员考试报名18日截止 2月6日考试',
                                              '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')

    test_data = [
        ('education', '福建春季公务员考试报名18日截止 2月6日考试'),
        ('sports', '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜'),
    ]
    acc_score = m.evaluate_model(test_data)
    print(f'acc_score: {acc_score}')  # 1.0

    #### train model with 1w data
    print('-' * 42)
    m = ClassicClassifier(output_dir='models/lr', model_name_or_model='lr')
    data_file = 'thucnews_train_1w.txt'
    m.train(data_file)
    m.load_model()
    predict_label, predict_proba = m.predict(
        ['顺义北京苏活88平米起精装房在售',
         '美EB-5项目“15日快速移民”将推迟'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')

output:

ClassicClassifier instance (LogisticRegression(fit_intercept=False), stopwords size: 2438)
predict_label: ['education' 'sports'], predict_proba: [0.5, 0.598941806741534]
acc_score: 1.0
------------------------------------------
predict_label: ['realty' 'education'], predict_proba: [0.7302956923617372, 0.2565005445322923]

Visual Feature Importance

Show feature weights of model, and prediction word weight, for example examples/visual_feature_importance.ipynb

import sys

sys.path.append('..')
from pytextclassifier import ClassicClassifier
import jieba

tc = ClassicClassifier(output_dir='models/lr-toy', model_name_or_model='lr')
data = [
    ('education', '名师指导托福语法技巧:名词的复数形式'),
    ('education', '中国高考成绩海外认可 是“狼来了”吗?'),
    ('sports', '图文:法网孟菲尔斯苦战进16强 孟菲尔斯怒吼'),
    ('sports', '四川丹棱举行全国长距登山挑战赛 近万人参与'),
    ('sports', '米兰客场8战不败国米10年连胜')
]
tc.train(data)
import eli5

infer_data = ['高考指导托福语法技巧国际认可',
              '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜']
eli5.show_weights(tc.model, vec=tc.feature)
seg_infer_data = [' '.join(jieba.lcut(i)) for i in infer_data]
eli5.show_prediction(tc.model, seg_infer_data[0], vec=tc.feature,
                     target_names=['education', 'sports'])

output:

img.png

Deep Classification model

本项目支持以下深度分类模型:FastText、TextCNN、TextRNN、Bert模型,import模型对应的方法来调用:

from pytextclassifier import FastTextClassifier, TextCNNClassifier, TextRNNClassifier, BertClassifier

下面以FastText模型为示例,其他模型的使用方法类似。

FastText 模型

训练和预测FastText模型示例examples/fasttext_classification_demo.py

import sys

sys.path.append('..')
from pytextclassifier import FastTextClassifier, load_data

if __name__ == '__main__':
    m = FastTextClassifier(output_dir='models/fasttext-toy')
    data = [
        ('education', '名师指导托福语法技巧:名词的复数形式'),
        ('education', '中国高考成绩海外认可 是“狼来了”吗?'),
        ('education', '公务员考虑越来越吃香,这是怎么回事?'),
        ('sports', '图文:法网孟菲尔斯苦战进16强 孟菲尔斯怒吼'),
        ('sports', '四川丹棱举行全国长距登山挑战赛 近万人参与'),
        ('sports', '米兰客场8战不败保持连胜'),
    ]
    m.train(data, num_epochs=3)
    print(m)
    # load trained best model
    m.load_model()
    predict_label, predict_proba = m.predict(['福建春季公务员考试报名18日截止 2月6日考试',
                                              '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')
    test_data = [
        ('education', '福建春季公务员考试报名18日截止 2月6日考试'),
        ('sports', '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜'),
    ]
    acc_score = m.evaluate_model(test_data)
    print(f'acc_score: {acc_score}')  # 1.0

    #### train model with 1w data
    print('-' * 42)
    data_file = 'thucnews_train_1w.txt'
    m = FastTextClassifier(output_dir='models/fasttext')
    m.train(data_file, names=('labels', 'text'), num_epochs=3)
    # load best trained model from model_dir
    m.load_model()
    predict_label, predict_proba = m.predict(
        ['顺义北京苏活88平米起精装房在售',
         '美EB-5项目“15日快速移民”将推迟']
    )
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')
    x, y, df = load_data(data_file)
    test_data = df[:100]
    acc_score = m.evaluate_model(test_data)
    print(f'acc_score: {acc_score}')

BERT 类模型

多分类模型

训练和预测BERT多分类模型,示例examples/bert_classification_zh_demo.py

import sys

sys.path.append('..')
from pytextclassifier import BertClassifier

if __name__ == '__main__':
    m = BertClassifier(output_dir='models/bert-chinese-toy', num_classes=2,
                       model_type='bert', model_name='bert-base-chinese', num_epochs=2)
    # model_type: support 'bert', 'albert', 'roberta', 'xlnet'
    # model_name: support 'bert-base-chinese', 'bert-base-cased', 'bert-base-multilingual-cased' ...
    data = [
        ('education', '名师指导托福语法技巧:名词的复数形式'),
        ('education', '中国高考成绩海外认可 是“狼来了”吗?'),
        ('education', '公务员考虑越来越吃香,这是怎么回事?'),
        ('sports', '图文:法网孟菲尔斯苦战进16强 孟菲尔斯怒吼'),
        ('sports', '四川丹棱举行全国长距登山挑战赛 近万人参与'),
        ('sports', '米兰客场8战不败国米10年连胜'),
    ]
    m.train(data)
    print(m)
    # load trained best model from model_dir
    m.load_model()
    predict_label, predict_proba = m.predict(['福建春季公务员考试报名18日截止 2月6日考试',
                                              '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')

    test_data = [
        ('education', '福建春季公务员考试报名18日截止 2月6日考试'),
        ('sports', '意甲首轮补赛交战记录:米兰客场8战不败国米10年连胜'),
    ]
    acc_score = m.evaluate_model(test_data)
    print(f'acc_score: {acc_score}')

    # train model with 1w data file and 10 classes
    print('-' * 42)
    m = BertClassifier(output_dir='models/bert-chinese', num_classes=10,
                       model_type='bert', model_name='bert-base-chinese', num_epochs=2,
                       args={"no_cache": True, "lazy_loading": True, "lazy_text_column": 1, "lazy_labels_column": 0, })
    data_file = 'thucnews_train_1w.txt'
    # 如果训练数据超过百万条,建议使用lazy_loading模式,减少内存占用
    m.train(data_file, test_size=0, names=('labels', 'text'))
    m.load_model()
    predict_label, predict_proba = m.predict(
        ['顺义北京苏活88平米起精装房在售',
         '美EB-5项目“15日快速移民”将推迟',
         '恒生AH溢指收平 A股对H股折价1.95%'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')

PS:如果训练数据超过百万条,建议使用lazy_loading模式,减少内存占用

多标签分类模型

分类可以分为多分类和多标签分类。多分类的标签是排他的,而多标签分类的所有标签是不排他的。

多标签分类比较直观的理解是,一个样本可以同时拥有几个类别标签, 比如一首歌的标签可以是流行、轻快,一部电影的标签可以是动作、喜剧、搞笑等,这都是多标签分类的情况。

训练和预测BERT多标签分类模型,示例examples/bert_multilabel_classification_zh_demo.py.py

import sys
import pandas as pd

sys.path.append('..')
from pytextclassifier import BertClassifier


def load_jd_data(file_path):
    """
    Load jd data from file.
    @param file_path: 
        format: content,其他,互联互通,产品功耗,滑轮提手,声音,APP操控性,呼吸灯,外观,底座,制热范围,遥控器电池,味道,制热效果,衣物烘干,体积大小
    @return: 
    """
    data = []
    with open(file_path, 'r', encoding='utf-8') as f:
        for line in f:
            line = line.strip()
            if line.startswith('#'):
                continue
            if not line:
                continue
            terms = line.split(',')
            if len(terms) != 16:
                continue
            val = [int(i) for i in terms[1:]]
            data.append([terms[0], val])
    return data


if __name__ == '__main__':
    # model_type: support 'bert', 'albert', 'roberta', 'xlnet'
    # model_name: support 'bert-base-chinese', 'bert-base-cased', 'bert-base-multilingual-cased' ...
    m = BertClassifier(output_dir='models/multilabel-bert-zh-model', num_classes=15,
                       model_type='bert', model_name='bert-base-chinese', num_epochs=2, multi_label=True)
    # Train and Evaluation data needs to be in a Pandas Dataframe containing at least two columns, a 'text' and a 'labels' column. The `labels` column should contain multi-hot encoded lists.
    train_data = [
        ["一个小时房间仍然没暖和", [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]],
        ["耗电情况:这个没有注意", [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
    ]
    data = load_jd_data('multilabel_jd_comments.csv')
    train_data.extend(data)
    print(train_data[:5])
    train_df = pd.DataFrame(train_data, columns=["text", "labels"])

    print(train_df.head())
    m.train(train_df)
    print(m)
    # Evaluate the model
    acc_score = m.evaluate_model(train_df[:20])
    print(f'acc_score: {acc_score}')

    # load trained best model from model_dir
    m.load_model()
    predict_label, predict_proba = m.predict(['一个小时房间仍然没暖和', '耗电情况:这个没有注意'])
    print(f'predict_label: {predict_label}, predict_proba: {predict_proba}')

Evaluation

Dataset

  1. THUCNews中文文本数据集(1.56GB):官方下载地址,抽样了10万条THUCNews中文文本10分类数据集(6MB),地址:examples/thucnews_train_10w.txt
  2. TNEWS今日头条中文新闻(短文本)分类 Short Text Classificaiton for News,该数据集(5.1MB)来自今日头条的新闻版块,共提取了15个类别的新闻,包括旅游,教育,金融,军事等,地址:tnews_public.zip

Evaluation Result

在THUCNews中文文本10分类数据集(6MB)上评估,模型在测试集(test)评测效果如下:

模型 acc 说明
LR 0.8803 逻辑回归Logistics Regression
TextCNN 0.8809 Kim 2014 经典的CNN文本分类
TextRNN_Att 0.9022 BiLSTM+Attention
FastText 0.9177 bow+bigram+trigram, 效果出奇的好
DPCNN 0.9125 深层金字塔CNN
Transformer 0.8991 效果较差
BERT-base 0.9483 bert + fc
ERNIE 0.9461 比bert略差

在中文新闻短文本分类数据集TNEWS上评估,模型在开发集(dev)评测效果如下:

模型 acc 说明
BERT-base 0.5660 本项目实现
BERT-base 0.5609 CLUE Benchmark Leaderboard结果 CLUEbenchmark
  • 以上结果均为分类的准确率(accuracy)结果
  • THUCNews数据集评测结果可以基于examples/thucnews_train_10w.txt数据用examples下的各模型demo复现
  • TNEWS数据集评测结果可以下载TNEWS数据集,运行examples/bert_classification_tnews_demo.py复现

命令行调用

提供分类模型命令行调用脚本,文件树:

pytextclassifier
├── bert_classifier.py
├── fasttext_classifier.py
├── classic_classifier.py
├── textcnn_classifier.py
└── textrnn_classifier.py

每个文件对应一个模型方法,各模型完全独立,可以直接运行,也方便修改,支持通过argparse 修改--data_path等参数。

直接在终端调用fasttext模型训练:

python -m pytextclassifier.fasttext_classifier -h

Text Cluster

Text clustering, for example examples/cluster_demo.py

import sys

sys.path.append('..')
from pytextclassifier.textcluster import TextCluster

if __name__ == '__main__':
    m = TextCluster(output_dir='models/cluster-toy', n_clusters=2)
    print(m)
    data = [
        'Student debt to cost Britain billions within decades',
        'Chinese education for TV experiment',
        'Abbott government spends $8 million on higher education',
        'Middle East and Asia boost investment in top level sports',
        'Summit Series look launches HBO Canada sports doc series: Mudhar'
    ]
    m.train(data)
    m.load_model()
    r = m.predict(['Abbott government spends $8 million on higher education media blitz',
                   'Middle East and Asia boost investment in top level sports'])
    print(r)

    ########### load chinese train data from 1w data file
    from sklearn.feature_extraction.text import TfidfVectorizer

    tcluster = TextCluster(output_dir='models/cluster', feature=TfidfVectorizer(ngram_range=(1, 2)), n_clusters=10)
    data = tcluster.load_file_data('thucnews_train_1w.txt', sep='\t', use_col=1)
    feature, labels = tcluster.train(data[:5000])
    tcluster.show_clusters(feature, labels, 'models/cluster/cluster_train_seg_samples.png')
    r = tcluster.predict(data[:30])
    print(r)

output:

TextCluster instance (MiniBatchKMeans(n_clusters=2, n_init=10), <pytextclassifier.utils.tokenizer.Tokenizer object at 0x7f80bd4682b0>, TfidfVectorizer(ngram_range=(1, 2)))
[1 1 1 1 1 1 1 1 1 1 1 8 1 1 1 1 1 1 1 1 1 1 9 1 1 8 1 1 9 1]

clustering plot image:

cluster_image

Contact

  • Issue(建议):GitHub issues
  • 邮件我:xuming: [email protected]
  • 微信我:加我微信号:xuming624, 进Python-NLP交流群,备注:姓名-公司名-NLP

Citation

如果你在研究中使用了pytextclassifier,请按如下格式引用:

APA:

Xu, M. Pytextclassifier: Text classifier toolkit for NLP (Version 1.2.0) [Computer software]. https://github.com/shibing624/pytextclassifier

BibTeX:

@misc{Pytextclassifier,
  title={Pytextclassifier: Text classifier toolkit for NLP},
  author={Xu Ming},
  year={2022},
  howpublished={\url{https://github.com/shibing624/pytextclassifier}},
}

License

授权协议为 The Apache License 2.0,可免费用做商业用途。请在产品说明中附加pytextclassifier的链接和授权协议。

Contribute

项目代码还很粗糙,如果大家对代码有所改进,欢迎提交回本项目,在提交之前,注意以下两点:

  • tests添加相应的单元测试
  • 使用python setup.py test来运行所有单元测试,确保所有单测都是通过的

之后即可提交PR。

More Repositories

1

pycorrector

pycorrector is a toolkit for text error correction. 文本纠错,实现了Kenlm,T5,MacBERT,ChatGLM3,LLaMA等模型应用在纠错场景,开箱即用。
Python
5,202
star
2

text2vec

text2vec, text to vector. 文本向量表征工具,把文本转化为向量矩阵,实现了Word2Vec、RankBM25、Sentence-BERT、CoSENT等文本表征、文本相似度计算模型,开箱即用。
Python
4,059
star
3

MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。
Python
2,564
star
4

python-tutorial

Python实用教程,包括:Python基础,Python高级特性,面向对象编程,多线程,数据库,数据科学,Flask,爬虫开发教程。
Jupyter Notebook
1,805
star
5

similarity

similarity: Text similarity calculation Toolkit for Java. 文本相似度计算工具包,java编写,可用于文本相似度计算、情感分析等任务,开箱即用。
Java
1,337
star
6

textgen

TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet and so on. 文本生成模型,实现了包括LLaMA,ChatGLM,BLOOM,GPT2,Seq2Seq,BART,T5,UDA等模型的训练和预测,开箱即用。
Python
893
star
7

similarities

Similarities: a toolkit for similarity calculation and semantic search. 相似度计算、匹配搜索工具包,支持亿级数据文搜文、文搜图、图搜图,python3开发,开箱即用。
Python
597
star
8

ChatPDF

RAG for Local LLM, chat with PDF/doc/txt files, ChatPDF
Python
435
star
9

parrots

Automatic Speech Recognition(ASR), Text-To-Speech(TTS) engine. 中英语音识别、多角色语音合成,支持多语言,准确率高
Python
423
star
10

nlp-tutorial

自然语言处理(NLP)教程,包括:词向量,词法分析,预训练语言模型,文本分类,文本语义匹配,信息抽取,翻译,对话。
Jupyter Notebook
353
star
11

dialogbot

dialogbot, provide search-based dialogue, task-based dialogue and generative dialogue model. 对话机器人,基于问答型对话、任务型对话、聊天型对话等模型实现,支持网络检索问答,领域知识问答,任务引导问答,闲聊问答,开箱即用。
Python
304
star
12

pke_zh

pke_zh, python keyphrase extraction for chinese(zh). 中文关键词或关键句提取工具,实现了KeyBert、PositionRank、TopicRank、TextRank等算法,开箱即用。
Python
155
star
13

lmft

ChatGLM-6B fine-tuning.
Python
137
star
14

nerpy

🌈 NERpy: Implementation of Named Entity Recognition using Python. 命名实体识别工具,支持BertSoftmax、BertSpan等模型,开箱即用。
Python
98
star
15

pysenti

Chinese Sentiment Classification Tool. 情感极性分类,基于知网、清华、BosonNLP情感词典,易扩展,基准方法,开箱即用。
Python
74
star
16

companynameparser

company name parser, extract company name brand. 中文公司名称分词工具,支持公司名称中的地名,品牌名(主词),行业词,公司名后缀提取。
Python
71
star
17

ChatPilot

ChatPilot: 实现AgentChat对话,支持Google搜索、文件网址对话(RAG)、代码解释器功能,复现了Kimi Chat(文件,拖进来;网址,发出来)。
Svelte
63
star
18

chatgpt-webui

ChatGPT WebUI using gradio. 给 LLM 对话和检索知识问答RAG提供一个简单好用的Web UI界面
Python
61
star
19

judger

自动作文评分工具,支持中文、英文作文智能评分,支持评分模型自训练,支持WEKA处理模型数据,支持自定义评分算法。java开发。
Roff
52
star
20

CodeAssist

CodeAssist is an advanced code completion tool that provides high-quality code completions for Python, Java, C++ and so on. CodeAssist 是一个高级代码补全工具,高质量为 Python、Java 和 C++ 补全代码。
Python
51
star
21

relext

RelExt: A Tool for Relation Extraction from Text. 文本实体关系抽取工具。
Python
44
star
22

text-feature

文本特征提取,适用于小说,论文,议论文等文本,提取词语、句子、依存关系等特征。python开发。
Python
37
star
23

rater

rater, recommender systems. 推荐模型,包括:DeepFM,Wide&Deep,DIN,DeepWalk,Node2Vec等模型实现,开箱即用。
Python
36
star
24

github-hot

Tracking the hot Github repos and update daily 每天自动追踪Github热门项目
Python
26
star
25

labelit

labelit, label tool with active learning, for classification task. 自动标注,基于主动学习,边标注边学习,减少人工标注量。
Python
26
star
26

title-generator

Automatic Text Summarization and Title Generation.
Python
24
star
27

pinyin-tokenizer

pinyintokenizer, 拼音分词器,将连续的拼音切分为单字拼音列表。
Python
19
star
28

case-analysis

NLP之病历分析:从病历文本之中提取关键信息,便于后续分析处理。
Java
19
star
29

EssaySocring

英文作文自动评分系统,支持评分模型自训练,支持WEKA处理模型数据,支持自定义评分算法。Java开发。
Roff
16
star
30

crf-seg

crf-seg:用于生产环境的中文分词处理工具,可自定义语料、可自定义模型、架构清晰,分词效果好。java编写。
Java
13
star
31

authorship-identification

【今日头条】文本作者身份识别比赛
Jupyter Notebook
9
star
32

text2vec-service

Service for Bert model to Vector. 高效的文本转向量(Text-To-Vector)服务,支持GPU多卡、多worker、多客户端调用,开箱即用。
Python
9
star
33

fake-news-detector

Fake News Detection Competition
Python
8
star
34

ChatGPT-API-server

build a python server for ChatGPT API.
Python
7
star
35

zh-normalization

Chinese(zh) sentence NSW(Non-Standard-Word) Normalization
Python
5
star
36

nlpcommon

NLP common tools.
Python
5
star
37

cpp-tutorial

C++开发实例教程,基础,开源库进阶,高级技巧。
C++
4
star
38

text2vec-encoder

**Text2vecEncoder** wraps the text2vec model with jina. It encodes text data into dense vectors.
Python
4
star
39

cvnet

have fun with image AI
Jupyter Notebook
4
star
40

BlogDemo

我的csdn博客中使用的代码,主要是算法。
Java
3
star
41

sbert

sbert, sentence bert.
Python
2
star
42

Diffusion-Tuning

Diffusion-Tuning: Training Your Own Diffusion model with custom dataset.
Python
2
star
43

actions

Actions: Agent pipelines.
2
star
44

tools

tools
JavaScript
2
star
45

pyweb

Web server use tornado.
Python
1
star
46

html5-demos

Use the html5 to show funny web demos
JavaScript
1
star
47

shibing624

1
star
48

phrase-search

短语搜索,支持公司名称、地址名称等短语的搜索,支持自定义排序、拼音处理,内置jetty提供web接口。java编写。
Java
1
star