Korean-PLM (Korean Pre-trained Language Models)
곡κ°λ νκ΅μ΄ μ¬μ νμ΅ λͺ¨λΈμ κΈ°λ‘ν©λλ€. ν¬κ² μλ 3κ°μ λͺ¨λΈ κ³μ΄λ‘ ꡬλΆνμΌλ©°, λͺ¨λΈ μ¬μ΄μ¦λ μ ννμ§ μμ μ μμ΅λλ€.
- Encoder Model (BERT κ³μ΄)
- Decoder Model (GPT κ³μ΄)
- Encoder-Decoder Model (Seq2seq κ³μ΄)
Encoder Model (BERT κ³μ΄)
| Model Name | Size | Link |
|---|---|---|
| LASSL BERT | Small (10M) | link |
| LASSL RoBERTa | Small (10M) | link |
| TUNiB ELECTRA (Ko) | Small (10M) | link |
| TUNiB ELECTRA (Ko-En) | Small (10M) | link |
| KoELECTRA | Small (10M) | link |
| KcELECTRA | Small (10M) | link |
| Soongsil-BERT | Small (10M) | link |
| Ko-CHAR-ELECTRA | Small (10M) | link |
| Distill-KoBERT | 30M | link |
| KoBERT | Base (110M) | link |
| KorBERT | Base (110M) | link |
| HanBERT | Base (110M) | link |
| KcBERT | Base (110M) | link |
| Soongsil-BERT | Base (110M) | link |
| KLUE-RoBERTa | Base (110M) | link |
| KoELECTRA | Base (110M) | link |
| KcELECTRA | Base (110M) | link |
| TUNiB-ELECTRA (Ko) | Base (110M) | link |
| TUNiB-ELECTRA (Ko-En) | Base (110M) | link |
| LMKor Albert | Base (110M) | link |
| LMKor Bert | Base (110M) | link |
| LMKor Funnel | Base (110M) | link |
| LMKor ELECTRA | Base (110M) | link |
| KalBERT | Base (110M) | link |
| LASSL BERT | Base (110M) | link |
| KoBigBird | Base (110M) | link |
| Ko-CHAR-BERT | Base (110M) | link |
| Ko-CHAR-ELECTRA | Base (110M) | link |
| BrainSBERT | Base (110M) | link |
| KPFBERT | Base (110M) | link |
| KcBERT | Large (340M) | link |
| KLUE-RoBERTa | Large (340M) | link |
Decoder Model (GPT κ³μ΄)
| Model Name | Size | Link |
|---|---|---|
| LASSL GPT2 | Small (20M) | link |
| LMKor KoGPT2 | Base (110M) | link |
| SKT KoGPT2 | Base (110M) | link |
| SKT KoGPT Trinity | 1.2B | link |
| EleutherAI Polyglot-Ko | 1.3B | link |
| EleutherAI Polyglot-Ko | 3.8B | link |
| EleutherAI Polyglot-Ko | 5.8B | link |
| KakaoBrain KoGPT | 6B | link |
Encoder-Decoder Model (Seq2seq κ³μ΄)
| Model Name | Size | Link |
|---|---|---|
| Cosmoquester BART | Mini (13M) | link |
| KE-T5 (Ko-En) | Small (60M) | link |
| KE-T5 (Ko) | Small (60M) | link |
| LMKor T5 | Small (60M) | link |
| Cosmoquester BART | Small (40M) | link |
| LMKor BERT Shared | Base (130M) | link |
| SKT KoBART | Base (130M) | link |
| Cosmoquester BART | Base (130M) | link |
| Kolang-T5 | Base (220M) | link |
| KcT5 | Base (220M) | link |
| KE-T5 (Ko-En) | Base (220M) | link |
| KE-T5 (Ko) | Base (220M) | link |
| Asian BART (Ko) | Base (360M) | link |
| pko-t5 (Ko) | Base (250M) | link |
| Wisenut KoT5 (Ko) | Base (220M) | link |
| mBART | Large (610M) | link |
| KE-T5 (Ko-En) | Large (770M) | link |
| KE-T5 (Ko) | Large (770M) | link |
| pko-t5 (Ko) | Large (800M) | link |