lonePatient/NeZha_Chinese_PyTorch lonePatient/NeZha_Chinese_PyTorch

  • Stars
    star
    261
  • Rank 156,630 (Top 4 %)
  • Language
    Python
  • License
    MIT License
  • Created about 4 years ago
  • Updated over 3 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
lonePatient/NeZha_Chinese_PyTorch
github

lonePatient

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

NEZHA: Neural Contextualized Representation for Chinese Language Understanding

NeZha_Chinese_PyTorch

pytorch版NEZHA，适配transformers

论文下载地址: NEZHA: Neural Contextualized Representation for Chinese Language Understanding

运行脚本依赖模块

如果需要运行该案例脚本，需要安装以下模块：

  1. trainsformers>=4.1.1
  2. TorchBlocks

模型权重下载

官方提供的Tensorflow版本权重下载地址：huawei-noah

已经转化为PyTorch版本权重下载地址如下：

说明：若加载的模型权重是从下列百度网盘下载的PyTorch模型权重，则需要保证torch版本>=1.6.0

运行

执行命令:

sh scripts/run_task_text_classification_chnsenti.sh

长文本

长文本可以通过设置config.max_position_embeddings参数实现，默认值为512，如：

config.max_position_embeddings=args.train_max_seq_length

结果

NEZHA(base-wwm) chnsenti
tensorflow 94.75
pytorch 94.92

More Repositories

1

awesome-pretrained-chinese-nlp-models

Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合
Python
4,712
star
2

BERT-NER-Pytorch

Chinese NER(Named Entity Recognition) using BERT(Softmax, CRF, Span)
Python
2,062
star
3

Bert-Multi-Label-Text-Classification

This repo contains a PyTorch implementation of a pretrained BERT model for multi-label text classification.
Python
860
star
4

albert_pytorch

A Lite Bert For Self-Supervised Learning Language Representations
Python
708
star
5

lookahead_pytorch

pytorch implement of Lookahead Optimizer
Python
188
star
6

TorchBlocks

A PyTorch-based toolkit for natural language processing
Python
151
star
7

daguan_2019_rank9

datagrand 2019 information extraction competition rank9
Python
130
star
8

BiLSTM-CRF-NER-PyTorch

This repo contains a PyTorch implementation of a BiLSTM-CRF model for named entity recognition task.
Python
120
star
9

Deep_Learning_For_Computer_Vision_With_Python

Deep Learning For Computer Vision With Python
Python
118
star
10

BERT-chinese-text-classification-pytorch

This repo contains a PyTorch implementation of a pretrained BERT model for text classification.
Python
99
star
11

electra_pytorch

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
Python
91
star
12

CLUE_pytorch

CLUE baseline pytorch CLUE的pytorch版本基线
Python
73
star
13

MobileBert_PyTorch

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Python
61
star
14

BERT-Attribute-Value-Extract

A Pytorch implementation of "Scaling Up Open Tagging from Tens to Thousands: Comprehension Empowered Attribute Value Extraction from Product Title" (ACL 2019).
Python
59
star
15

multi-sample_dropout_pytorch

a simple pytorch implement of Multi-Sample Dropout
Python
56
star
16

BERT-SDA

A PyTorch implementation of "Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation"
Python
56
star
17

ERNIE-text-classification-pytorch

This repo contains a PyTorch implementation of a pretrained ERNIE model for text classification.
Python
54
star
18

chinese-word2vec-pytorch

word2vec implementation for skip-gram in pytorch
Python
53
star
19

bert-sentence-similarity-pytorch

This repo contains a PyTorch implementation of a pretrained BERT model for sentence similarity task.
Python
49
star
20

label_smoothing_pytorch

pytorch implement of Label Smoothing
Python
32
star
21

EvoNorms_PyTorch

Evolving Normalization-Activation Layers
Python
19
star
22

NovoGrad-pytorch

pytorch implement of NovoGrad Optimizer
Python
18
star
23

cw2vec-pytorch

cw2vec implementation in pytorch
Python
17
star
24

train-bert-pytorch

Python
15
star
25

knowledge-driven-dialogue-lic2019-rank5

2019语言与智能技术竞赛第5名方案
Python
14
star
26

2021-GAIIC-Track1-idea

全球人工智能技术创新大赛【赛道一】
10
star
27

pytorch_fashionMNIST_practice

使用pytorch进行图像训练的模板
Python
9
star
28

keras_learning

Jupyter Notebook
9
star
29

Contextual-Chinese-Strokes-Embeddings

Implementation of the language model for Contextual chinese strokes Embeddings with PyTorch
Python
8
star
30

lonePatient.github.io

HTML
6
star
31

kaggle-camera-model-identification

IEEE's Signal Processing Society - Camera Model Identification
Python
6
star
32

tensorflow-eager-examples

Examples of Eager Execution in tensorflow
Python
6
star
33

char-cnn-text-classification

This repo contains a PyTorch implementation of a char-level CNN model for text classification.
Python
3
star