bojone/attention bojone/attention

  • Stars
    star
    1,407
  • Rank 33,429 (Top 0.7 %)
  • Language
    Python
  • Created almost 7 years ago
  • Updated over 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
bojone/attention
github

bojone

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

some attention implements

attention

some attention implements

《Attention is All You Need》中的Attention机制的实现

http://kexue.fm/archives/4765/

测试环境

python 2.7 + tensorflow 1.8+ + keras 2.2.4

注：本项目不再跟进新版本的tensorflow/keras，如果有新版本的需求，可以到bert4keras的layers.py处复制对应的函数。

交流

QQ交流群：67729435，微信群请加机器人微信号spaces_ac_cn

More Repositories

1

bert4keras

keras implement of transformers for humans
Python
5,071
star
2

vae

a simple vae and cvae from keras
Python
1,092
star
3

kg-2019

2019年百度的三元组抽取比赛，“科学空间队”源码
Python
765
star
4

bert_in_keras

在Keras下微调Bert的一些例子；some examples of bert in keras
Python
650
star
5

SimCSE

SimCSE在中文任务上的简单实验
Python
536
star
6

word-discovery

速度更快、效果更好的中文新词发现
Python
476
star
7

BERT-whitening

简单的向量白化改善句向量质量
Python
437
star
8

SPACES

端到端的长本文摘要模型（法研杯2020司法摘要赛道）
Python
365
star
9

Capsule

A Capsule Implement with Pure Keras
Python
350
star
10

seq2seq

keras example of seq2seq, auto title
Python
330
star
11

CoSENT

比Sentence-BERT更有效的句向量方案
Python
284
star
12

gan

some demo of GANs
Python
264
star
13

crf

keras implementation of conditional random field
Python
246
star
14

lic2020_baselines

some baselines for lic2020 (http://lic2020.cipsc.org.cn/)
Python
221
star
15

kg-2019-baseline

2019年百度的三元组抽取比赛，一个baseline
Python
211
star
16

GlobalPointer

全局指针统一处理嵌套与非嵌套NER
Python
209
star
17

flow

Keras implement of flow-based models
Python
206
star
18

NBCE

Naive Bayes-based Context Extension
Python
177
star
19

keras_lookahead

lookahead optimizer for keras
Python
172
star
20

dgcnn_for_reading_comprehension

keras implement of dgcnn for reading comprehension
Python
166
star
21

t5_in_bert4keras

整理一下在keras中使用T5模型的要点
Python
164
star
22

Pattern-Exploiting-Training

Pattern-Exploiting Training在中文上的简单实验
Python
163
star
23

on-lstm

Keras implement of ON-LSTM (Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks)
Python
154
star
24

Keras-DDPM

生成扩散模型的Keras实现
Python
149
star
25

infomax

extract features by maximizing mutual information
Python
140
star
26

nlp-zero

基于最小熵原理的NLP工具包
Python
136
star
27

oppo-text-match

小布助手对话短文本语义匹配的一个baseline
Python
136
star
28

CLUE-bert4keras

真 · “Deep Learning for Humans”
Python
133
star
29

o-gan

O-GAN: Extremely Concise Approach for Auto-Encoding Generative Adversarial Networks
Python
124
star
30

P-tuning

P-tuning方法在中文上的简单实验
Python
124
star
31

labse

Language-agnostic BERT Sentence Embedding (LaBSE)
Python
122
star
32

ee-2019-baseline

面向金融领域的事件主体抽取（ccks2019），一个baseline
Python
121
star
33

accum_optimizer_for_keras

wrapping a keras optimizer to implement gradient accumulation
Python
120
star
34

tf_word2vec

a tensorflow version of Word2Vec with a new loss
Python
118
star
35

el-2019-baseline

2019年百度的实体链指比赛（ccks2019），一个baseline
Python
117
star
36

GPLinker

基于GlobalPointer的实体/关系/事件抽取
Python
117
star
37

gan-qp

GAN-QP: A Novel GAN Framework without Gradient Vanishing and Lipschitz Constraint
Python
117
star
38

el-2019

2019年百度的实体链接比赛，“科学空间队”源码
Python
107
star
39

perturbed_masking

基于BERT的无监督分词和句法分析
Python
105
star
40

margin-softmax

keras sparse implement of margin-softmax
Python
103
star
41

chinese-gen

中文生成式预训练模型
98
star
42

nezha_gpt_dialog

Python
95
star
43

r-drop

R-Drop方法在中文任务上的简单实验
Python
86
star
44

n2n-ocr-for-qqcaptcha

an n2n ocr for qq captcha, 端到端的腾讯验证码识别
Python
85
star
45

ape210k_baseline

用bert4keras来解小学数学应用题
73
star
46

keras_radam

RAdam optimizer for keras
Python
70
star
47

KgCLUE-bert4keras

基于“Seq2Seq+前缀树”的知识图谱问答
Python
67
star
48

albert_zh

转换 https://github.com/brightmart/albert_zh 到google格式
Python
62
star
49

CCL_CMRC2017

第一届“讯飞杯”中文机器阅读理解评测参考模型
Python
60
star
50

simple-chinese-ocr

A Simple Chinese OCR from tipdm contest
Python
60
star
51

CPM_LM_bert4keras

在bert4keras下加载CPM_LM模型
Python
51
star
52

univae

基于Transformer的单模型、多尺度的VAE模型
Python
51
star
53

unsupervised-text-generation

无监督文本生成的一些方法
Python
48
star
54

sohu2021-baseline

2021搜狐校园文本匹配算法大赛baseline
Python
45
star
55

keras_adversarial_training

Adversarial Training for NLP in Keras
Python
44
star
56

bytepiece

更纯粹、更高压缩率的Tokenizer
Python
43
star
57

infomap

a beautiful method for cluster or community detection
Python
43
star
58

gpt_cchess

bert4keras实现gpt下中国象棋
38
star
59

rnn

一些RNN的实现
Python
38
star
60

CDial-GPT-tf

用bert4keras加载CDial-GPT
Python
38
star
61

tiger

A Tight-fisted Optimizer
Python
38
star
62

T-GANs

Training Generative Adversarial Networks Via Turing Test
Python
37
star
63

keras_recompute

saving memory by recomputing for keras
Python
37
star
64

bert-of-theseus

bert-of-theseus via bert4keras
Python
30
star
65

shuffle

Python下shuffle几百G文件
Python
29
star
66

text_compare

用python比较两个字符串差异，高亮差异部分
Python
26
star
67

mydog

监控文件改动，随时自动备份，彻底防止误删
Python
22
star
68

pytorch_bert_to_tf

pytorch版bert权重转tf
Python
22
star
69

nezha

精简版NEZHA模型权重
21
star
70

keras_lazyoptimizer

Keras implement of Lazy optimizer
Python
21
star
71

unsupervised-vocabulary-search

完整的新词发现&词库构建例子
Python
20
star
72

adafactor

adafactor optimizer for keras
Python
19
star
73

NNCWS

Neutral Network based Chinese Segment System
Python
18
star
74

exposure_bias

some strategies for exposure bias in seq2seq
Python
17
star
75

vib

Variational Information Bottleneck
Python
15
star
76

antiminer

简单的挖矿病毒查杀脚本
Python
15
star
77

lic2021_baselines

目前只有阅读理解赛道的
Python
14
star
78

LST-CLUE

Ladder Side-Tuning在CLUE上的简单尝试
Python
14
star
79

FSQ

Keras implement of Finite Scalar Quantization
Python
12
star
80

baidu_dog_classifier

Python
12
star
81

analytical-classification

逻辑回归和单层softmax的解析解
Python
11
star
82

baidu-ner-contest

a bilstm-seq2seq ner script from baidu-ner contest
Python
10
star
83

memm

Keras implementation of Maximum Entropy Markov Model
Python
9
star
84

python-snippets

some frequently-used snippets for python
Python
8
star
85

adax

AdaX optimizer for keras
Python
7
star
86

beiguo

根据进程pid找对应的container
Python
6
star