• Stars
    star
    250
  • Rank 162,397 (Top 4 %)
  • Language
    Python
  • Created about 5 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Chinese EHR Bert Pretrained Model.

ChineseEHRBert

A Chinese Electric Health Record Bert Pretrained Model.

中文版

cleaner

The cleaner is responsible for cleaning txt files, which is used for training a Chinese bert model. The cleaner split lines in original lines into small lines. Each small line is a complete sentence with a punctuation. This is required for training next sentence prediction task.

usage

cd ./cleaner/
python parser.py [-h] [--input INPUT] [--output OUTPUT] [-s] [--log LOG]
  • --input: input directory
  • --output: output directory
  • -s: output is one single file
  • --log: log frequency

train

Pre-train a bert model with cleaned text. We should generate .tfrecord first, and pre-train with google's code. To notice, cleaner file may be too big to load in RAM. Our script splits these files and generate multiple .tfrecord.

usage

Split file and convert to .tfrecord

cd ./train/
python make_pretrain_bert.py [-h] [-f FILE_PATH] [-s SPLIT_LINE]
                             [-p SPLIT_PATH] [-o OUTPUT_PATH] [-l MAX_LENGTH]
                             [-b BERT_BASE_DIR]
  • -f: cleaned file path
  • -s: split line count, default=500000
  • -p: splited file save path
  • -o: .tfrecord save path
  • -l: max length
  • -b: bert base dir

One should change parameters for your specific requirement in pretrain128.sh and pretrain512.sh.

sh pretrain128.sh
sh pretrain512.sh

test

Test Chinese medical NLP tasks by BERT in one line! Two NER tasks, one QA task, one RE task and one sentence similarity task.

cd ./test/
sh run_test.sh

Tasks include CCKS2019NER, cMedQA2, Tianchi_NER, Tianchi_RE, ncov2019_sim.

Results

Results compared with original BERT and ChineseEHRBert. Results are preparing.

Citation

Author

More Repositories

1

awesome_Chinese_medical_NLP

中文医学NLP公开资源整理:术语集/语料库/词向量/预训练模型/知识图谱/命名实体识别/QA/信息抽取/模型/论文/etc
2,076
star
2

RRHF

[NIPS2023] RRHF & Wombat
Python
784
star
3

CODER

CODER: Knowledge infused cross-lingual medical term embedding for term normalization. [JBI, ACL-BioNLP 2022]
Python
71
star
4

KeBioLM

Improving Biomedical Pretrained Language Models with Knowledge [BioNLP 2021]
Python
64
star
5

Tenpai_prediction

用深层神经网络预测日本麻将立直听牌
Python
57
star
6

math401-llm

Source codes and datasets for How well do Large Language Models perform in Arithmetic tasks?
55
star
7

BioBART

BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model [ACL-BioNLP 2022]
Python
50
star
8

ICD-MSMN

Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding [ACL 2022]
Python
47
star
9

Triaffine-nested-ner

Fusing Heterogeneous Factors with Triaffine Mechanism for Nested Named Entity Recognition [ACL 2022 Findings]
Python
43
star
10

CHIP2020_term_normalization

CHIP2020 Task 3 术语标准化任务
Python
30
star
11

RAMM

Codes and Pre-trained models for RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training [ACM MM 2023]
Python
22
star
12

GTS

Code for Unsupervised multi-granular Chinese word segmentation and term discovery via graph partition [JBI]
Python
15
star
13

HMDB51_CNN

Use CNN to classify HMDB51.
Python
7
star
14

bios_re

Relation Extraction for BIOS
Python
7
star
15

pyserverchan

Server-chan for python. / Server酱python版。
Python
7
star
16

Deep-Learning-Playground

A playground for Deep Learning
Jupyter Notebook
6
star
17

Moleformer

Codes for Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling
4
star
18

stealer

Calculate the preflop 3bet steal range in NLHE.
Python
4
star
19

ACG_translator

Train a jp2zh NMT model with ACG parallel corpus.
Python
4
star
20

embedding_script

大文档计算word2vec,glove,ngram的脚本
Python
4
star
21

GanjinZero.github.io

HTML
3
star
22

Quora-Insincere-Questions-Classification

Detect toxic content to improve online conversations
Jupyter Notebook
3
star
23

wana

python, R, linux, tensorflow, pytorch, numpy, pandas, etc. 踩坑记录
2
star
24

dangdang_gpt2

Python
2
star
25

Ace-Blocker

Calculate MTT EV, variance, VAR, min buyin, ...
Python
2
star
26

GTO_study

Some notes on GTO result.
2
star
27

Haiku_generator

Generate Haiku by seq2seq.
Python
1
star