amazon-science/auto-cot

Stars
1,218
Rank 38,490 (Top 0.8 %)
Language
Jupyter Notebook
License
Apache License 2.0
Created about 2 years ago
Updated 8 months ago

amazon-science/auto-cot

amazon-science

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Official implementation for "Automatic Chain of Thought Prompting in Large Language Models" (stay tuned & more will be updated)

Auto-CoT: Automatic Chain of Thought Prompting in Large Language Models (ICLR 2023)

Cheer AI up with the "let's think step by step" prompt? More plz. Let’s think not just step by step, but also one by one.

Auto-CoT uses more cheers & diversity to SAVE huge manual efforts in chain of thought prompt design, matching or even exceeding performance of manual design on GPT-3.

Check out our 25-page paper for more information.

Requirements

Python>=3.8

pip install torch==1.8.2+cu111 torchtext==0.9.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
pip install -r requirements.txt

Datasets

Download the datasets from the following:

https://github.com/kojima-takeshi188/zero_shot_cot/tree/main/dataset
https://github.com/kojima-takeshi188/zero_shot_cot/tree/main/log

Quick Start

See try_cot.ipynb

Instructions

Construct Demos:

python run_demo.py \
--task multiarith \
--pred_file log/multiarith_zero_shot_cot.log \
--demo_save_dir demos/multiarith

Run inference:

python run_inference.py \
--dataset multiarith \
--demo_path demos/multiarith \
--output_dir experiment/multiarith

Citing Auto-CoT

@inproceedings{zhang2023automatic,
  title={Automatic Chain of Thought Prompting in Large Language Models},
  author={Zhang, Zhuosheng and Zhang, Aston and Li, Mu and Smola, Alex},
  booktitle={The Eleventh International Conference on Learning Representations (ICLR 2023)},
  year={2023}
}

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

mm-cot

Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tuned and more will be updated)

chronos-forecasting

Chronos: Pretrained (Language) Models for Probabilistic Time Series Forecasting

patchcore-inspection

siam-mot

SiamMOT: Siamese Multi-Object Tracking

alexa-teacher-models

bigdetection

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

earth-forecasting-transformer

Official implementation of Earthformer

Jupyter Notebook

sccl

Pytorch implementation of Supporting Clustering with Contrastive Learning, NAACL 2021

prompt-pretraining

Official implementation for the paper "Prompt Pre-Training with Over Twenty-Thousand Classes for Open-Vocabulary Visual Recognition"

RefChecker

RefChecker provides automatic checking pipeline and benchmark dataset for detecting fine-grained hallucinations generated by Large Language Models.

esci-data

Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search

video-contrastive-learning

Video Contrastive Learning with Global Context, ICCVW 2021

tgl

gan-control

This package provides a pythorch implementation of "GAN-Control: Explicitly Controllable GANs", ICCV 2021.

Jupyter Notebook

polygon-transformer

ReFinED

ReFinED is an efficient and accurate entity linking (EL) system.

tanl

Structured Prediction as Translation between Augmented Natural Languages

unconditional-time-series-diffusion

Official PyTorch implementation of TSDiff models presented in the NeurIPS 2023 paper "Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting"

crossnorm-selfnorm

CrossNorm and SelfNorm for Generalization under Distribution Shifts, ICCV 2021

cceval

CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)

wqa_tanda

This repo provides code and data used in our TANDA paper.

spot-diff

Project for <SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation> (ECCV 2022)

mintaka

Dataset from the paper "Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering" (COLING 2022)

mix-generation

MixGen: A New Multi-Modal Data Augmentation

long-short-term-transformer

[NeurIPS 2021 Spotlight] Official implementation of Long Short-Term Transformer for Online Action Detection

alexa-arena

fraud-dataset-benchmark

Repository for Fraud Dataset Benchmark

Jupyter Notebook

glass-text-spotting

Official implementation for "GLASS: Global to Local Attention for Scene-Text Spotting" (ECCV'22)

meta-q-learning

Code for the paper "Meta-Q-Learning"( ICLR 2020)

exponential-moving-average-normalization

PyTorch implementation of EMAN for self-supervised and semi-supervised learning: https://arxiv.org/abs/2101.08482

co-with-gnns-example

datatuner

Code related to "Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity" paper

mxeval

sentence-representations

CodeSage

CodeSage: Code Representation Learning At Scale (ICLR 2024)

semimtr-text-recognition

Multimodal Semi-Supervised Learning for Text Recognition (SemiMTR)

fact-check-summarization

instruct-video-to-video

tabsyn

Official Implementations of "Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space""

object-centric-learning-framework

omni-detr

PyTorch implementation of Omni-DETR for omni-supervised object detection: https://arxiv.org/abs/2203.16089

progressive-coordinate-transforms

Progressive Coordinate Transforms for Monocular 3D Object Detection, NeurIPS 2021

FeatGraph

small-baseline-camera-tracking

A dataset to facilitate the research of Structure-from-Motion (SfM) for movie and TV shows.

tubelet-transformer

This is an official implementation of TubeR: Tubelet Transformer for Video Action Detection

embert

Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

RAGChecker

RAGChecker: A Fine-grained Framework For Diagnosing RAG

probconserv

Datasets and code for results presented in the ProbConserv paper

semi-vit

PyTorch implementation of Semi-supervised Vision Transformers

qa-dataset-converter

Code from the paper "What do Models Learn from Question Answering Datasets?" (EMNLP 2020)

masked-diffusion-lm

Official implementation for the paper "A Cheaper and Better Diffusion Language Model with Soft-Masked Noise"

transformer-gan

transformers-data-augmentation

Code associated with the "Data Augmentation using Pre-trained Transformer Models" paper

gluonmm

A library of transformer models for computer vision and multi-modality research

crossmodal-contrastive-learning

CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representations, ICCV 2021

recode

Releasing code for "ReCode: Robustness Evaluation of Code Generation Models"

tracking-dataset

dstc11-track2-intent-induction

DSTC 11 Track 2: Intent Induction from Conversations for Task-Oriented Dialogue

dse

dq-bart

DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization (ACL 2022)

gnn-tail-generalization

auto-rag-eval

Code repo for the ICML 2024 paper "Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation"

boon

Datasets and code for results presented in the BOON paper

Jupyter Notebook

proteno

This repository contains data used in the NAACL 2021 Paper - Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems (https://arxiv.org/abs/2104.07777)

fact-graph

Implementation of the paper "FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations (NAACL 2022)"

c2f-seg

Official Implementation for ICCV'23 paper Coarse-to-Fine Amodal Segmentation with Shape Prior (C2F-Seg).

amazon-multilingual-counterfactual-dataset

QA-ViT

indoor-scene-generation-eai

Jupyter Notebook

long-tailed-ood-detection

Official implementation for "Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition" (ICML'22 Long Presentation)

efficient-longdoc-classification

object-centric-multiple-object-tracking

hyperbolic-embeddings

Code for hyperboloid embeddings for knowledge graph entities

domain-knowledge-injection

azcausal

Causal Inference in Python

Repoformer

Repoformer: Selective Retrieval for Repository-Level Code Completion (ICML 2024)

ContraCLM

[ACL 2023] Code for ContraCLM: Contrastive Learning For Causal Language Model

unified-ept

A Unified Efficient Pyramid Transformer for Semantic Segmentation, ICCVW 2021

robust-tableqa

Two approaches for robust TableQA: 1) ITR is a general-purpose retrieval-based approach for handling long tables in TableQA transformer models. 2) LI-RAGE is a robust framework for open-domain TableQA which addresses several limitations. (ACL 2023)

bold

Dataset associated with "BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation" paper

replay-based-recurrent-rl

Code for "Task-Agnostic Continual RL: In Praise of a Simple Baseline"

controlling-llm-memorization

carbon-assessment-with-ml

CaML: Carbon Footprinting of Household Products with Zero-Shot Semantic Text Similarity

Jupyter Notebook

peft-design-spaces

Official implementation for "Parameter-Efficient Fine-Tuning Design Spaces"

llm-interpret

Code for the ACL 2023 paper: "Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale"

creating-and-correcting-novel-ml-model-errors

Jupyter Notebook

BartGraphSumm

Implementation of the paper "Efficiently Summarizing Text and Graph Encodings of Multi-Document Clusters (NAACL 2021)"

tofueval

wqa-cascade-transformers

textadain-robust-recognition

TextAdaIN: Paying Attention to Shortcut Learning in Text Recognizers

multiatis

Data and code for the paper "End-to-End Slot Alignment and Recognition for Cross-Lingual NLU" (Accepted to EMNLP 2020)

iwslt-autodub-task

street-reasoning

STREET: a multi-task and multi-step reasoning dataset

contrastive-controlled-mt

Code and data for the IWSLT 2022 shared task on Formality Control for SLT

pizza-semantic-parsing-dataset

The PIZZA dataset continues the exploration of task-oriented parsing by introducing a new dataset for parsing pizza and drink orders, whose semantics cannot be captured by flat slots and intents.

redset

Redset is a dataset containing three months worth of user query metadata that ran on a selected sample of instances in the Amazon Redshift fleet. We provide query metadata for 200 provisioned and serverless instances each.

fast-rl-with-slow-updates

Jupyter Notebook

few-shot-baseline

doc-mt-metrics

normalizer-free-robust-training

Official implementation of "Removing Batch Normalization Boosts Adversarial Training" (ICML'22)