declare-lab/RelationPrompt declare-lab/RelationPrompt

  • Stars
    star
    123
  • Rank 290,145 (Top 6 %)
  • Language
    Python
  • License
    MIT License
  • Created about 3 years ago
  • Updated over 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
declare-lab/RelationPrompt
github

declare-lab

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

This repository implements our ACL Findings 2022 research paper RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction. The goal of Zero-Shot Relation Triplet Extraction (ZeroRTE) is to extract relation triplets of the format (head entity, tail entity, relation), despite not having annotated data for the test relation labels.

RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction

PWC Colab Jupyter

This repository implements our ACL Findings 2022 research paper RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction. The goal of Zero-Shot Relation Triplet Extraction (ZeroRTE) is to extract relation triplets of the format (head entity, tail entity, relation), despite not having annotated data for the test relation labels.

diagram

Installation

  • Python 3.7
  • If your GPU uses CUDA 11, first install the specific PyTorch: pip install torch==1.10.0 --extra-index-url https://download.pytorch.org/whl/cu113
  • Install requirements: pip install -r requirements.txt or conda env create --file environment.yml
  • Download and extract the datasets here to outputs/data/splits/zero_rte
  • FewRel Pretrained Model (unseen=10, seed=0)
  • Wiki-ZSL Pretrained Model (unseen=10, seed=0)

Data Exploration | Colab

from wrapper import Dataset

data = Dataset.load(path)
for s in data.sents:
    print(s.tokens)
    for t in s.triplets:
        print(t.head, t.tail, t.label)

Generate with Pretrained Model | Colab

from wrapper import Generator

model = Generator(load_dir="gpt2", save_dir="outputs/wrapper/fewrel/unseen_10_seed_0/generator")
model.generate(labels=["location", "religion"], path_out="synthetic.jsonl")

Extract with Pretrained Model | Colab

from wrapper import Extractor

model = Extractor(load_dir="facebook/bart-base", save_dir="outputs/wrapper/fewrel/unseen_10_seed_0/extractor_final")
model.predict(path_in=path_test, path_out="pred.jsonl")

Model Training | Colab

Train the Generator and Extractor models:

from pathlib import Path
from wrapper import Generator, Extractor

generator = Generator(
    load_dir="gpt2",
    save_dir=str(Path(save_dir) / "generator"),
)
extractor = Extractor(
    load_dir="facebook/bart-base",
    save_dir=str(Path(save_dir) / "extractor"),
)
generator.fit(path_train, path_dev)
extractor.fit(path_train, path_dev)

Generate synthetic data with relation triplets for test labels:

generator.generate(labels_test, path_out=path_synthetic)

Train the final Extractor model using the synthetic data and predict on test sentences:

extractor_final = Extractor(
    load_dir=str(Path(save_dir) / "extractor" / "model"),
    save_dir=str(Path(save_dir) / "extractor_final"),
)
extractor_final.fit(path_synthetic, path_dev)
extractor_final.predict(path_in=path_test, path_out=path_pred)

Experiment Scripts

Run training in wrapper.py (You can change "fewrel" to "wiki" or unseen to 5/10/15 or seed to 0/1/2/3/4):

python wrapper.py main \
--path_train outputs/data/splits/zero_rte/fewrel/unseen_10_seed_0/train.jsonl \                                       
--path_dev outputs/data/splits/zero_rte/fewrel/unseen_10_seed_0/dev.jsonl \                                           
--path_test outputs/data/splits/zero_rte/fewrel/unseen_10_seed_0/test.jsonl \                                         
--save_dir outputs/wrapper/fewrel/unseen_10_seed_0

Run evaluation (Single-triplet setting)

python wrapper.py run_eval \                                                                                               
--path_model outputs/wrapper/fewrel/unseen_10_seed_0/extractor_final \                                                  
--path_test outputs/data/splits/zero_rte/fewrel/unseen_10_seed_0/test.jsonl \
--mode single

Run evaluation (Multi-triplet setting)

python wrapper.py run_eval \                                                                                               
--path_model outputs/wrapper/fewrel/unseen_10_seed_0/extractor_final \                                                  
--path_test outputs/data/splits/zero_rte/fewrel/unseen_10_seed_0/test.jsonl \
--mode multi

Research Citation

If the code is useful for your research project, we appreciate if you cite the following paper:

@inproceedings{chia-etal-2022-relationprompt,
    title = "{R}elation{P}rompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction",
    author = "Chia, Yew Ken  and
      Bing, Lidong  and
      Poria, Soujanya  and
      Si, Luo",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2022",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.findings-acl.5",
    doi = "10.18653/v1/2022.findings-acl.5",
    pages = "45--57",
    abstract = "Despite the importance of relation extraction in building and representing knowledge, less research is focused on generalizing to unseen relations types. We introduce the task setting of Zero-Shot Relation Triplet Extraction (ZeroRTE) to encourage further research in low-resource relation extraction methods. Given an input sentence, each extracted triplet consists of the head entity, relation label, and tail entity where the relation label is not seen at the training stage. To solve ZeroRTE, we propose to synthesize relation examples by prompting language models to generate structured texts. Concretely, we unify language model prompts and structured text approaches to design a structured prompt template for generating synthetic relation samples when conditioning on relation label prompts (RelationPrompt). To overcome the limitation for extracting multiple relation triplets in a sentence, we design a novel Triplet Search Decoding method. Experiments on FewRel and Wiki-ZSL datasets show the efficacy of RelationPrompt for the ZeroRTE task and zero-shot relation classification. Our code and data are available at github.com/declare-lab/RelationPrompt.",
}

More Repositories

1

conv-emotion

This repo contains implementation of different architectures for emotion recognition in conversations.
Python
1,336
star
2

MELD

MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversation
Python
793
star
3

tango

Codes and Model of the paper "Text-to-Audio Generation using Instruction Tuned LLM and Latent Diffusion Model"
Python
754
star
4

multimodal-deep-learning

This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.
OpenEdge ABL
730
star
5

awesome-sentiment-analysis

Reading list for Awesome Sentiment Analysis papers
517
star
6

instruct-eval

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.
Python
500
star
7

flan-alpaca

This repository contains code for extending the Stanford Alpaca synthetic instruction tuning to existing instruction-tuned models such as Flan-T5.
Python
343
star
8

awesome-emotion-recognition-in-conversations

A comprehensive reading list for Emotion Recognition in Conversations
252
star
9

MISA

MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Python
192
star
10

RECCON

This repository contains the dataset and the PyTorch implementations of the models from the paper Recognizing Emotion Cause in Conversations.
Python
169
star
11

Multimodal-Infomax

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.
Python
154
star
12

dialogue-understanding

This repository contains PyTorch implementation for the baseline models from the paper Utterance-level Dialogue Understanding: An Empirical Study
Python
123
star
13

contextual-utterance-level-multimodal-sentiment-analysis

Context-Dependent Sentiment Analysis in User-Generated Videos
Python
123
star
14

flacuna

Flacuna was developed by fine-tuning Vicuna on Flan-mini, a comprehensive instruction collection encompassing various tasks. Vicuna is already an excellent writing assistant, and the intention behind Flacuna was to enhance Vicuna's problem-solving capabilities. To achieve this, we curated a dedicated instruction dataset called Flan-mini.
Python
106
star
15

CASCADE

This repo contains code to detect sarcasm from text in discussion forum using deep learning
Python
86
star
16

red-instruct

Codes and datasets of the paper Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
Python
75
star
17

BBFN

This repository contains the implementation of the paper -- Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis
Python
62
star
18

CICERO

The purpose of this repository is to introduce new dialogue-level commonsense inference datasets and tasks. We chose dialogues as the data source because dialogues are known to be complex and rich in commonsense.
Python
60
star
19

dialog-HGAT

Dialogue Relation Extraction with Document-level Heterogeneous Graph Attention Networks
Python
57
star
20

kingdom

Domain Adaptation using External Knowledge for Sentiment Analysis
Python
55
star
21

hfusion

Multimodal sentiment analysis using hierarchical fusion with context modeling
Python
44
star
22

MIME

This repository contains PyTorch implementations of the models from the paper An Empirical Study MIME: MIMicking Emotions for Empathetic Response Generation.
Python
43
star
23

speech-adapters

Codes and datasets for our ICASSP2023 paper, Evaluating parameter-efficient transfer learning approaches on SURE benchmark for speech understanding
Python
40
star
24

LLM-PuzzleTest

This repository is maintained to release dataset and models for multimodal puzzle reasoning.
Python
33
star
25

HyperTTS

Python
33
star
26

MSA-Robustness

NAACL 2022 paper on Analyzing Modality Robustness in Multimodal Sentiment Analysis
Python
31
star
27

CIDER

This repository contains the dataset and the pytorch implementations of the models from the paper CIDER: Commonsense Inference for Dialogue Explanation and Reasoning. CIDER has been accepted to appear at SIGDIAL 2021.
Python
28
star
28

sentence-ordering

This repository contains the PyTorch implementation of the paper STaCK: Sentence Ordering with Temporal Commonsense Knowledge appearing at EMNLP 2021.
Python
28
star
29

resta

Restore safety in fine-tuned language models through task arithmetic
Python
25
star
30

HyperRED

This repository implements our EMNLP 2022 research paper A Dataset for Hyper-Relational Extraction and a Cube-Filling Approach.
Python
25
star
31

MM-InstructEval

This repository contains code to evaluate various multimodal large language models using different instructions across multiple multimodal content comprehension tasks.
Python
24
star
32

MM-Align

[EMNLP 2022] This repository contains the official implementation of the paper "MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences"
Python
24
star
33

identifiable-transformers

Python
22
star
34

exemplary-empathy

This repository contains the source codes of the paper -- Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication
Python
22
star
35

della

DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
Python
22
star
36

TEAM

Our EMNLP 2022 paper on MCQA
Python
21
star
37

DoubleMix

Code for the COLING 2022 paper "DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification"
Python
20
star
38

adapter-mix

Python
15
star
39

M2H2-dataset

This repository contains the dataset and baselines explained in the paper: M2H2: A Multimodal Multiparty Hindi Dataset For HumorRecognition in Conversations
Python
15
star
40

ASTE-RL

This repository contains the source codes for the paper: "Aspect Sentiment Triplet Extraction using Reinforcement Learning" published at CIKM 2021.
Python
14
star
41

SAT

Code for the EMNLP 2022 Findings short paper "SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training"
Jupyter Notebook
13
star
42

KNOT

This repository contains the implementation of the paper -- KNOT: Knowledge Distillation using Optimal Transport for Solving NLP Tasks
Python
13
star
43

WikiDes

A Wikipedia-based summarization dataset
Python
13
star
44

Sealing

[NAACL 2024] Official Implementation of paper "Self-Adaptive Sampling for Efficient Video Question Answering on Image--Text Models"
Python
9
star
45

VIP

Our EMNLP 2022 paper on VIP-Based Prompting for Parameter-Efficient Learning
Python
8
star
46

domadapter

Code for EACL'23 paper "Udapter: Efficient Domain Adaptation Using Adapters"
Python
8
star
47

RobustMIFT

[Arxiv 2024] Official Implementation of the paper: "Towards Robust Instruction Tuning on Multimodal Large Language Models"
Jupyter Notebook
8
star
48

ferret

Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Technique
Python
8
star
49

SANCL

[COLING 2022] This repository contains the code of the paper SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learning.
Python
7
star
50

segue

Codes and Checkpoints of the Interspeech paper "Sentence Embedder Guided Utterance Encoder (SEGUE) for Spoken Language Understanding"
Python
6
star
51

llm_robustness

Python
3
star
52

NLP-OT

1
star