INK-USC/KagNet INK-USC/KagNet

  • This repository has been archived on 05/Jul/2020
  • Stars
    star
    271
  • Rank 151,717 (Top 3 %)
  • Language
    Python
  • License
    MIT License
  • Created about 5 years ago
  • Updated over 4 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
INK-USC/KagNet
github

INK-USC

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Knowledge-Aware Graph Networks for Commonsense Reasoning (EMNLP-IJCNLP 19)

KagNet: Knowledge-Aware Graph Networks

News: We released a more general-purpose LM-GNN reasoning framework, MHGRN, which includes more options for text/graph encoders. It also matches the current state-of-the-art performance (76.5% acc) on the offical CommonsenseQA test set. We won't maintain this repo, so please follow the new repo.

Introduction

This codebase is an implementation of the proposed KagNet model for commonsense reasoning (EMNLP-IJCNLP 2019).

  • Overall Workflow
  • GCN + LSTM-based Path Encoder + Hierarchical Path Attention

Install Dependencies

sudo apt-get install graphviz libgraphviz-dev pkg-config
conda create -n kagnet_test python==3.6.3
conda activate kagnet_test
# which python
# which pip
pip install torch torchvision 
pip install tensorflow-gpu==1.10.0
conda install faiss-gpu cudatoolkit=10.0 -c pytorch -n kagnet_test 
pip install nltk
conda install -c conda-forge spacy -n kagnet_test
python -m spacy download en
pip install jsbeautifier
pip install networkx
pip install dgl
pip install pygraphviz
pip install allennlp

Datasets downloading


cd datasets
mkdir csqa_new

wget -P csqa_new https://s3.amazonaws.com/commensenseqa/train_rand_split.jsonl
wget -P csqa_new https://s3.amazonaws.com/commensenseqa/dev_rand_split.jsonl 
wget -P csqa_new https://s3.amazonaws.com/commensenseqa/test_rand_split_no_answers.jsonl


python convert_csqa.py csqa_new/train_rand_split.jsonl csqa_new/train_rand_split.jsonl.statements
python convert_csqa.py csqa_new/dev_rand_split.jsonl csqa_new/dev_rand_split.jsonl.statements
python convert_csqa.py csqa_new/test_rand_split_no_answers.jsonl csqa_new/test_rand_split_no_answers.jsonl.statements

Preprocess ConceptNet and embedding files

cd ../conceptnet
wget https://s3.amazonaws.com/conceptnet/downloads/2018/edges/conceptnet-assertions-5.6.0.csv.gz
gzip -d conceptnet-assertions-5.6.0.csv.gz
python extract_cpnet.py


cd ../triple_string
python triple_string_generation.py

# get concept and relation embeddings with frequency and vocab files
cd ../embeddings/
cd glove/
wget http://nlp.stanford.edu/data/glove.6B.zip
unzip glove.6B.zip
rm glove.*.zip
cd ../
python glove_to_npy.py  
python create_embeddings_glove.py

Concept Grounding

# concept grounding: core concept recognition (find mentioned concepts)
 
cd ../grounding/
python batched_grounding.py generate_bash "../datasets/csqa_new/train_rand_split.jsonl.statements"
bash cmd.sh
python batched_grounding.py combine "../datasets/csqa_new/train_rand_split.jsonl.statements"
python prune_qc.py ../datasets/csqa_new/train_rand_split.jsonl.statements.mcp

python batched_grounding.py generate_bash "../datasets/csqa_new/dev_rand_split.jsonl.statements"
bash cmd.sh
python batched_grounding.py combine "../datasets/csqa_new/dev_rand_split.jsonl.statements"
python prune_qc.py ../datasets/csqa_new/dev_rand_split.jsonl.statements.mcp

# python batched_grounding.py generate_bash "../datasets/csqa_new/test_rand_split.jsonl.statements"
# bash cmd.sh
# python batched_grounding.py combine "../datasets/csqa_new/test_rand_split.jsonl.statements"

Schema Graph Construction

cd ../pathfinder/
python graph_construction.py

python batched_pathfinding.py generate_bash "../datasets/csqa_new/train_rand_split.jsonl.statements.mcp"
bash cmd.sh
python batched_pathfinding.py combine "../datasets/csqa_new/train_rand_split.jsonl.statements.mcp"

python batched_pathfinding.py generate_bash "../datasets/csqa_new/dev_rand_split.jsonl.statements.mcp"
bash cmd.sh
python batched_pathfinding.py combine "../datasets/csqa_new/dev_rand_split.jsonl.statements.mcp"

# Pruning 

python path_scoring.py train
python path_scoring.py dev

python path_pruning.py train
python path_pruning.py dev

cd ../graph_generation
python graph_gen.py train
python graph_gen.py test

Train KagNet based on extracted BERT embeddings

cd ../baselines/

bash train_csqa_bert.sh
python extract_csqa_bert.py --bert_model bert-large-uncased --do_eval --do_lower_case --data_dir ../datasets/csqa_new --eval_batch_size 60 --learning_rate 1e-4  --max_seq_length 70 --mlp_hidden_dim 16 --output_dir ./models/ --save_model_name bert_large_b60g4lr1e-4wd0.01wp0.1_1337 --epoch_id 1 --data_split_to_extract train_rand_split.jsonl --output_sentvec_file ../datasets/csqa_new/train_rand_split.jsonl.statements.finetuned.large --layer_id -1 
python extract_csqa_bert.py --bert_model bert-large-uncased --do_eval --do_lower_case --data_dir ../datasets/csqa_new --eval_batch_size 60 --learning_rate 1e-4  --max_seq_length 70 --mlp_hidden_dim 16 --output_dir ./models/ --save_model_name bert_large_b60g4lr1e-4wd0.01wp0.1_1337 --epoch_id 1 --data_split_to_extract dev_rand_split.jsonl --output_sentvec_file ../datasets/csqa_new/dev_rand_split.jsonl.statements.finetuned.large --layer_id -1

cd ../models/
python main.py

Citation

@inproceedings{kagnet-emnlp19,
  author    = {Bill Yuchen Lin and Xinyue Chen and Jamin Chen and Xiang Ren},
  title     = {KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning.},
  booktitle = {Proceedings of EMNLP-IJCNLP},
  year      = {2019},
}

Remarks

Feel free to email yuchen[dot]lin[at]usc[dot]edu if you have any questions and need help.

More Repositories

1

RE-Net

Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs (EMNLP 2020)
Python
436
star
2

USC-DS-RelationExtraction

Distantly Supervised Relation Extraction
C++
419
star
3

MHGRN

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (EMNLP 2020)
Python
246
star
4

TriggerNER

TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition (ACL 2020)
Python
173
star
5

CommonGen

A Constrained Text Generation Challenge Towards Generative Commonsense Reasoning
Python
139
star
6

AlpacaTag

AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (ACL 2019 Demo)
HTML
137
star
7

CrossFit

Code for paper "CrossFit 🏋️: A Few-shot Learning Challenge for Cross-task Generalization in NLP" (https://arxiv.org/abs/2104.08835)
Python
102
star
8

ClusType

Automatic Entity Recognition and Typing for Domain-Specific Corpora (KDD'15)
Python
99
star
9

temporal-gcn-lstm

Code for Characterizing and Forecasting User Engagement with In-App Action Graphs: A Case Study of Snapchat
Python
77
star
10

AFET

AFET: Automatic Fine-Grained Entity Typing (EMNLP'16)
Python
57
star
11

CPL

Collaborative Policy Learning for Open Knowledge Graph Reasoning (EMNLP 2019)
Python
56
star
12

PLE

Label Noise Reduction in Entity Typing (KDD'16)
C++
53
star
13

NERO

Source Code for paper "NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction", WWW 2020
Python
47
star
14

fewNER

Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER (ACL 2022)
Python
43
star
15

StructMineDataPipeline

Performs entity detection, distant supervision, candidate generation, and produces JSON files for typing systems (PLE, AFET, CoType)
C++
43
star
16

shifted-label-distribution

Source code for paper "Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction" (EMNLP 2019)
C++
39
star
17

DualRE

Source code for paper: "Learning Dual Retrieval Module for Semi-supervised Relation Extraction"
Python
36
star
18

hierarchical-explanation-neural-sequence-models

Source code for "Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models", ICLR 2020.
Python
30
star
19

CALM

Source code for ICLR 2021 paper : Pre-training Text-to-Text Transformers for Concept-Centric Common Sense
Python
27
star
20

ReQuest

Indirect Supervision for Relation Extraction Using Question-Answer Pairs (WSDM'18)
C++
24
star
21

DIG

Discretized Integrated Gradients for Explaining Language Models (EMNLP 2021)
Python
24
star
22

LEAN-LIFE

Label Efficient Learning From Explanations
Python
23
star
23

XCSR

Code Repo for the ACL21 paper "Common Sense Beyond English: Evaluating and Improving Multilingual LMs for Commonsense Reasoning"
Python
22
star
24

ReCross

ReCross: Unsupervised Cross-Task Generalization via Retrieval Augmentation
Python
22
star
25

VisCOLL

Code and data for the project "Visually grounded continual learning of compositional semantics"
Python
21
star
26

DArtNet

Temporal Attribute Prediction via Joint Modeling of Multi-Relational Structure Evolution
Python
19
star
27

NumerSense

The data and code for NumerSense (EMNLP2020)
Python
19
star
28

NExT

Source Code for paper "Learning from Explanations with Neural Execution Tree", ICLR 2020
Python
18
star
29

GMED

Source code for "Gradient Based Memory Editing for Task-Free Continual Learning", 4th Lifelong ML Workshop@ICML 2020
Python
17
star
30

HGN

Learning Contextualized Knowledge Structures for Commonsense Reasoning
Python
17
star
31

SalKG

This is the official PyTorch implementation of our NeurIPS 2021 paper: "SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning"
Python
14
star
32

FaiRR

FaiRR: Faithful and Robust Deductive Reasoning over Natural Language (ACL 2022)
Python
14
star
33

hypter

Zero-shot Learning by Generating Task-specific Adapters
Python
14
star
34

FiD-ICL

"FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning" (ACL 2023)
Python
13
star
35

IsoBN

IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
Python
13
star
36

sparse-distillation

Code for "Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models"
Python
12
star
37

expl-refinement

Code for the paper "Refining Language Model with Compositional Explanation" (NeurIPS 2021)
Python
12
star
38

RiddleSense

RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge
Python
12
star
39

ConNet

Python
12
star
40

entity-robustness

Code and data for paper "On the Robustness of Reading Comprehension Models to Entity Renaming" (NAACL'22)
Python
11
star
41

mrc-explanation

Source Code for "Teaching Machine Comprehension with Compositional Explanations" (Findings of EMNLP 2020)
Python
11
star
42

Reflect

Data and Code for Paper "Reflect Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality" (EMNLP 2022)
Python
11
star
43

rockner

Python
10
star
44

BITE

Code and data for paper "BITE: Textual Backdoor Attacks with Iterative Trigger Injection"
Python
9
star
45

CLIF

Code for Findings at EMNLP 2021 paper: "Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning"
Python
8
star
46

G-PlanET

Python
8
star
47

procedural-extraction

Code for paper Eliciting Knowledge from Experts: Automatic Transcript Parsing for Cognitive Task Analysis, in proceedings of ACL 2019
Python
8
star
48

XMD

XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models
Vue
7
star
49

RobustLR

A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners
Python
7
star
50

RationaleMultiRewardDistillation

Code and Dataset for preprint titled "Tailoring Self-Rationalizers with Multi-Reward Distillation"
Python
6
star
51

LINK

Code for paper "In Search of the Long-Tail: Systematic Generation of Long-Tail Knowledge via Logical Rule Guided Search"
Python
6
star
52

Upstream-Bias-Mitigation

Code and data for NAACL 2021 paper "On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning"
Python
5
star
53

RationaleHumanUtility

Codebase for Human Utility of FTRs at ACL 2023
Python
5
star
54

Lifelong-ICL

Code for paper "Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack"
Jupyter Notebook
4
star
55

PE2

Code for paper "Prompt Engineering a Prompt Engineer" (https://arxiv.org/abs/2311.05661)
Python
4
star
56

deceive-KG-models

An implementation of the experiments on KG robustness
Python
4
star
57

ER-Test

Code for ER-Test, accepted to the Findings of EMNLP 2022
Python
3
star
58

get-started-on-dl-experiments

2
star
59

ink-usc.github.io

INK Research Lab Website
JavaScript
2
star
60

CrossTaskMoE

Code for paper "Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts" (Findings of EMNLP 2022)
Python
2
star
61

predicting-big-bench

Code for paper "How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench"
Python
2
star
62

bias-mitigation-via-transfer-learning

Source code for Arxiv paper: Efficiently Mitigating Classification Bias via Transfer Learning
2
star
63

Controllable-AV-Explanations

Python
1
star
64

lm-forgetting-prediction-code

Python
1
star
65

MACROSCORE

MACROSCORE - Scoring Scientific Research
Jupyter Notebook
1
star