Beomi/InfiniTransformer Beomi/InfiniTransformer

  • Stars
    star
    318
  • Rank 131,872 (Top 3 %)
  • Language
    Python
  • License
    MIT License
  • Created 7 months ago
  • Updated 7 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
Beomi/InfiniTransformer
github

Beomi

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Unofficial PyTorch/๐Ÿค—Transformers(Gemma/Llama3) implementation of Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

More Repositories

1

KoAlpaca

KoAlpaca: ํ•œ๊ตญ์–ด ๋ช…๋ น์–ด๋ฅผ ์ดํ•ดํ•˜๋Š” ์˜คํ”ˆ์†Œ์Šค ์–ธ์–ด๋ชจ๋ธ
Jupyter Notebook
1,529
star
2

KcBERT

๐Ÿค— Pretrained BERT model & WordPiece tokenizer trained on Korean Comments ํ•œ๊ตญ์–ด ๋Œ“๊ธ€๋กœ ํ”„๋ฆฌํŠธ๋ ˆ์ด๋‹ํ•œ BERT ๋ชจ๋ธ๊ณผ ๋ฐ์ดํ„ฐ์…‹
469
star
3

BitNet-Transformers

0๏ธโƒฃ1๏ธโƒฃ๐Ÿค— BitNet-Transformers: Huggingface Transformers Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch with Llama(2) Architecture
Python
255
star
4

KcELECTRA

๐Ÿค— Korean Comments ELECTRA: ํ•œ๊ตญ์–ด ๋Œ“๊ธ€๋กœ ํ•™์Šตํ•œ ELECTRA ๋ชจ๋ธ
232
star
5

cite.gg

๐Ÿ” ์šฐ๋ฆฌ๊ฐ€ ์ฝ์„ ๋…ผ๋ฌธ์„ ์ฐพ์•„์„œ, Cite.GG
Vue
85
star
6

ko-lm-evaluation-harness

Forked repo from https://github.com/EleutherAI/lm-evaluation-harness/commit/1f66adc
Python
58
star
7

easy-lm-trainer

๐Ÿค— ์ตœ์†Œํ•œ์˜ ์„ธํŒ…์œผ๋กœ LM์„ ํ•™์Šตํ•˜๊ธฐ ์œ„ํ•œ ์ƒ˜ํ”Œ์ฝ”๋“œ
Python
57
star
8

simple_bank_korea

simple crawler for Korean banks with Transactions
Python
56
star
9

pypapago

๐Ÿง[Archived][Unofficial] Python wrapper for Papago translation service
Python
47
star
10

Gemma-EasyLM

Train GEMMA on TPU/GPU! (Codebase for training Gemma-Ko Series)
Python
40
star
11

KcBERT-Finetune

KcBERT/KcELECTRA Fine Tune Benchmarks code (forked from https://github.com/monologg/KoELECTRA/tree/master/finetune)
Python
39
star
12

aws-lambda-py3

Pre-compiled Python3(3.6+) packages for AWS Lambda layers
Dockerfile
29
star
13

transformers-language-modeling

Train ๐Ÿค—transformers with DeepSpeed: ZeRO-2, ZeRO-3
Python
20
star
14

irkshop

Django Shopping Mall WebSite for IRK / DEMO:
JavaScript
17
star
15

gptpy

GPTPy: Your kind Python guide, powered by AI to fix errors and explain code
Python
14
star
16

data_camp_wcr

ํŒŒ์ด์ฌ์„ ํ™œ์šฉํ•œ ์‹ค์ „ ์›นํฌ๋กค๋ง CAMP ๊ฐ•์˜ 1-2๊ธฐ ์†Œ์Šค์ฝ”๋“œ
Python
11
star
17

data_camp_wcr_3

ํŒŒ์ด์ฌ์„ ํ™œ์šฉํ•œ ์‹ค์ „ ์›นํฌ๋กค๋ง CAMP 3๊ธฐ ์†Œ์Šค์ฝ”๋“œ
Python
10
star
18

exbert-transformers

exBERT on Transformers๐Ÿค—
Python
10
star
19

django-tube

์žฅ๊ณ ๊ฑธ์Šค ์„œ์šธ ํŠœํ† ๋ฆฌ์–ผ ๋ฏธ๋‹ˆ์‹œ๋ฆฌ์ฆˆ: DjangoTube
9
star
20

where-is-my-customs

๋‚ด ํ†ต๊ด€์€ ์–ด๋””์ฏค? ์นด์นด์˜คํ†ก ๋ด‡
Python
8
star
21

naverNewsContentsCrawler

naverNewsContentsCrawler
Python
7
star
22

beomi.github.io_2019

Beomi's Tech Blog (Deprecated)
HTML
7
star
23

SlideShare_Character_Updater

Prevents [SlideShare Korean(or UTF-8 langs) Character "Blank"] phenomenon
JavaScript
7
star
24

megatronlm_dataset_autotokenizer

Megatron-LM/GPT-NeoX compatible Text Encoder with ๐Ÿค—Transformers AutoTokenizer.
Python
6
star
25

Wifibucks

KT Starbucks Wi-Fi Auto Connector
JavaScript
6
star
26

beomi.github.io

Beomi's Tech Blog (HTML Only)
HTML
5
star
27

s3-direct-uploader-demo

Direct S3 Upload with Lambda Demo
HTML
4
star
28

kb_transaction

KB ๊ตญ๋ฏผ์€ํ–‰ ๊ฑฐ๋ž˜๋‚ด์—ญ ์กฐํšŒ
Python
4
star
29

deepo-nlp

Dockerfile for NLP(Korean) based on Deepo
Dockerfile
4
star
30

typora-s3-uploader

CLI file/img S3 uploader for Typora
Python
2
star
31

lambda-requirements-builder

Simple Script Example to Build Lambda Layers (with Python3)
Shell
1
star
32

tf-keras-on-lambda

Example Repo for Tensorflow + Keras on AWS Lambda
Python
1
star
33

SmartSpamFilter-QnA

Smart Spam Filter QnA
1
star
34

heroku-helloworld

Test DRF Project for Heroku Deploy
Python
1
star
35

UpdateNoticeBOT

Web site update alert BOT with telegram, made with Python and Celery
Python
1
star
36

Evergreen-SBERT

๐ŸŒฒEvergreen-SBERT: Extensible Sentence BERT with Up-to-Date Corpora
Jupyter Notebook
1
star
37

transformers-pytorch-gpu

๐Ÿ’ก Docker image for Huggingface ๐Ÿค— Transformers + GPU + Jupyter notebook + OhMyZsh
Jupyter Notebook
1
star
38

PYCON2017

2017 ํŒŒ์ด์ฝ˜ ๋ฐœํ‘œ์ž๋ฃŒ: <์ฒ˜์Œ๋ถ€ํ„ฐ ์•Œ์•„๋ณด๋Š” ์›น ํฌ๋กค๋Ÿฌ>
Python
1
star
39

paper2code

paper2code with TF / PyTorch
Dockerfile
1
star
40

WebCrawlerExamples

WebCrawler ๊ฐ•์˜ ์˜ˆ์ œ
Python
1
star
41

NexonTalk20180612

๋„ฅ์Šจํ† ํฌ 20180612 : ๋”ฅ๋Ÿฌ๋‹ ์†๊ธ€์”จ ์ธ์‹ ์„œ๋น„์Šค - III, ์„œ๋ฒ„๋ฆฌ์Šค ์›น ํ”„๋ก ํŠธ ๋งŒ๋“ค๊ธฐ
JavaScript
1
star