ictnlp/LLaMA-Omni ictnlp/LLaMA-Omni

  • Stars
    star
    1,889
  • Rank 24,550 (Top 0.5 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created 2 months ago
  • Updated 2 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
ictnlp/LLaMA-Omni
github

ictnlp

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

More Repositories

1

StreamSpeech

StreamSpeech is an “All in One” seamless model for offline and simultaneous speech recognition, speech translation and speech synthesis.
Python
882
star
2

BayLing

“百聆”是一个基于LLaMA的语言对齐增强的英语/中文大语言模型，具有优越的英语/中文能力，在多语言和通用任务等多项测试中取得ChatGPT 90%的性能。BayLing is an English/Chinese LLM equipped with advanced language alignment, showing superior capability in English/Chinese generation, instruction following and multi-turn interaction.
Python
294
star
3

TruthX

Code for ACL 2024 paper "TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space"
Python
105
star
4

awesome-transformer

A collection of transformer's guides, implementations and variants.
102
star
5

DialoFlow

Code for ACL 2021 main conference paper "Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances".
Python
93
star
6

NAST-S2x

A fast speech-to-any translation model that supports simultaneous decoding and offers 28× speedup.
Python
60
star
7

DASpeech

Code for NeurIPS 2023 paper "DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation".
Python
56
star
8

DSTC8-AVSD

We rank the 1st in DSTC8 Audio-Visual Scene-Aware Dialog competition. This is the source code for our IEEE/ACM TASLP (AAAI2020-DSTC8-AVSD) paper "Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog".
Python
55
star
9

OR-NMT

Source Code for ACL2019 paper <Bridging the Gap between Training and Inference for Neural Machine Translation>
Python
42
star
10

STEMM

Code for ACL 2022 main conference paper "STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation".
Python
35
star
11

DiSeg

Source code for ACL 2023 paper "End-to-End Simultaneous Speech Translation with Differentiable Segmentation"
Python
32
star
12

BoN-NAT

Python
23
star
13

Seq-NAT

Source code for <Sequence-Level Training for Non-Autoregressive Neural Machine Translation>.
Python
23
star
14

PLUVR

Code for ACL 2022 main conference paper "Neural Machine Translation with Phrase-Level Universal Visual Representations".
Python
21
star
15

HMT

Source code for ICLR 2023 spotlight paper "Hidden Markov Transformer for Simultaneous Machine Translation"
Python
21
star
16

ComSpeech

Code for ACL 2024 main conference paper "Can We Achieve High-quality Direct Speech-to-Speech Translation Without Parallel Speech Data?".
Python
21
star
17

TLAT-NMT

Source code for the EMNLP 2020 long paper <Token-level Adaptive Training for Neural Machine Translation>.
Python
20
star
18

NMLA-NAT

Code for NeurIPS 2022 Spotlight paper " Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation"
Python
20
star
19

AIH

Code for Findings of ACL 2021 paper "Addressing Inquiries about History: An Efficient and Practical Framework for Evaluating Open-domain Chatbot Consistency".
Python
19
star
20

RSI-NAT

Source code for "Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation"
Python
19
star
21

DiverseNMT

Source code for the AAAI 2020 long paper <Modeling Fluency and Faithfulness for Diverse Neural Machine Translation>.
Python
19
star
22

PTE-NMT

Source code for the NAACL 2021 paper: Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation
Python
15
star
23

CMOT

Code for ACL 2023 main conference paper "CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation"
Python
14
star
24

LNMT-CA

Code for EMNLP 2022 main conference paper "Low-resource Neural Machine Translation with Cross-modal Alignment".
Jupyter Notebook
14
star
25

CRESS

Code for ACL 2023 main conference paper "Understanding and Bridging the Modality Gap for Speech Translation".
Python
14
star
26

ITST

Code for EMNLP 2022 main conference paper "Information-Transport-based Policy for Simultaneous Translation"
Python
14
star
27

SiLLM

SiLLM is a Simultaneous Machine Translation (SiMT) Framework. It utilizes a Large Language model as the translation model and employs a traditional SiMT model for policy-decision to achieve SiMT through collaboration.
Python
14
star
28

TACS

Source code for Truth-Aware Context Selection: Mitigating the Hallucinations of Large Language Models Being Misled by Untruthful Contexts
Python
13
star
29

BT4ST

Code for ACL 2023 main conference paper "Back Translation for Speech-to-text Translation Without Transcripts".
Python
12
star
30

Dual-Path

Code for ACL 2022 main conference paper "Modeling Dual Read/Write Paths for Simultaneous Machine Translation"
Python
12
star
31

NA-MNMT

Source code for "Importance-based Neuron Allocation for Multilingual Neural Machine Translation"
Python
12
star
32

Convex-Learning

Code for NeurIPS 2023 paper "Beyond MLE: Convex Learning for Text Generation"
Python
12
star
33

GMA

Code for ACL 2022 findings paper "Gaussian Multi-head Attention for Simultaneous Machine Translation"
Python
11
star
34

PCFG-NAT

Code for NeurIPS 2023 paper "Non-autoregressive Machine Translation with Probabilistic Context-free Grammar".
Cuda
10
star
35

NAST

Official implementation for EMNLP 2023 paper "Non-autoregressive Streaming Transformer for Simultaneous Translation"
Python
10
star
36

COKD

Code for ACL 2022 main conference paper "Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation".
JavaScript
9
star
37

MoE-Waitk

Code for EMNLP 2021 oral paper "Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy"
Python
8
star
38

DDRS-NAT

Code for NAACL2022 main conference paper "One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation"
Python
8
star
39

SU4MT

Code for EMNLP 2023 paper "Enhancing Neural Machine Translation with Semantic Units"
Python
8
star
40

SeerForcingNMT

Source code for "Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation"
7
star
41

Zero-MNMT

Python
7
star
42

Multiscale-Contextualization

ACL2024 Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation
Python
7
star
43

Wait-info

Source code for our EMNLP 2022 paper "Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation"
Python
7
star
44

BS-SiMT

Source code for our ACL 2023 paper "Learning Optimal Policy for Simultaneous Machine Translation via Binary Search"
Python
7
star
45

CTC-S2UT

Code for ACL 2024 findings paper "CTC-based Non-autoregressive Textless Speech-to-Speech Translation"
7
star
46

GS4NMT

source code for "Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation"
Python
6
star
47

DST

DST is a Decoder-only simultaneous machine translation model, which can conduct policy decision and translation concurrently
Python
6
star
48

LFR-NMT

Source code for the EMNLP 2022 paper "Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions"
Python
5
star
49

CPDecoder

optimize the decoder of the neural machine translation model by the cube pruning algorithm
Python
5
star
50

nar-tutorial

Slides for NAR tutorial
4
star
51

CAPT

Code for EMNLP 2022 main conference paper "Counterfactual Data Augmentation via Perspective Transition for Open-Domain Dialogues".
4
star
52

PED-SiMT

Code for "Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation"
Python
4
star
53

Glance-SiMT

Python
3
star
54

SAMMT

Code for EMNLP 2023 paper "Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation"
Python
3
star
55

RN4NMT

source code for "Refining Source Representations with Relation Networks for Neural Machine Translation".
PLSQL
2
star
56

Seg2Seg

2
star
57

Tailored-Ref

Code for EMNLP 2023 paper "Simultaneous Machine Translation with Tailored Reference"
2
star
58

SemLing-MNMT

Code for ACL 2024 paper "Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features".
Python
2
star
59

ComSpeech-Site

JavaScript
1
star
60

corpus_NKD

data repo for "Knowledge Diffusion for Neural Dialogue Generation"
1
star
61

TruthX-site

HTML
1
star
62

StreamSpeech-site

JavaScript
1
star
63

Rephraser-NAT

Code for AAAI 2023 paper "Rephrasing the Reference for Non-Autoregressive Machine Translation"
1
star
64

Auto-RAG

Python
1
star