Helsinki-NLP/nlu-dataset-diagnostics

Stars
1
Language
Python
Created over 3 years ago
Updated almost 3 years ago

Helsinki-NLP/nlu-dataset-diagnostics

Helsinki-NLP

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

This repository contains data and scripts to reproduce the results from our paper: How Does Data Corruption Affect Natural Language Understanding Models? A Study on GLUE datasets.

Tatoeba-Challenge

Opus-MT

Open neural machine translation models and web services

OPUS-MT-train

Training open neural machine translation models

prosody

Helsinki Prosody Corpus and A System for Predicting Prosodic Prominence from Text

OpusFilter

OpusFilter - Parallel corpus processing toolkit

HBMP

Sentence Embeddings in NLI with Iterative Refinement Encoders

OPUS-CAT

OPUS-CAT is a collection of software which make it possible to OPUS-MT neural machine translation models in professional translation. OPUS-CAT includes a local offline MT engine and a collection of CAT tool plugins.

OpusTools

XED

XED multilingual emotion datasets

Jupyter Notebook

OPUS

The Open Parallel Corpus

UkrainianLT

A collection of links to Ukrainian language tools

OPUS-translator

Translation demonstrator

mammoth

MAMMOTH: MAssively Multilingual Modular Open Translation @ Helsinki

MuCoW

Automatically harvested multilingual contrastive word sense disambiguation test sets for machine translation

subalign

sentimentator

Tool for sentiment analysis annotation

OPUS-MT-testsets

benchmarks for evaluating MT models

OpusTools-perl

neural-search-tutorials

Additional Notebooks for the Building NLP Applications course

Jupyter Notebook

OPUS-interface

OPUS repository interface

OPUS-ingest

LanguageCodes

shroom

nli-data-sanity-check

Data and scripts for a diagnostics test suite which allows to assess whether an NLU dataset constitutes a good testbed for evaluating the models' meaning understanding capabilities.

Jupyter Notebook

OPUS-repository

doclevel-MT-benchmark

Document-level Machine Translation Benchmark

Uplug

americasnlp2021-st

AmericasNLP 2021 shared task

Geometry

shared-info

LSDC

Low-Saxon Dialect Classification

pdf2xml

Syntactic_Debiasing

OpusTranslationService

Translation service based on LibreTranslate

murre24

Manually annotated dataset of Finnish varieties in the Suomi24, the largest Finnish internet forum, the id's of automatically annotated dialectal messages and the scripts used for classification and evaluation.

OPUS-index

Index of resources in OPUS

OpusFilter-hub

A hub of OpusFilter configurations

NLU-Course-2020

SELF-FEIL

Emotion Lexicons for Finnish

ndc-aligned

Word-aligned version of the Norwegian Dialect Corpus

OPUS-MT-dashboard

External-MT-leaderboard

Leaderboards for external MT models

en-fi-testsuite

WMT18 Testsuite for Finnish morphology

finlandsvensk-AI

OPUS-website

OPUS website files

OPUS-MT-leaderboard-recipes

Makefile recipes shared between all leaderboard repos

OPUS-MT-leaderboard

murreviikko

Dialectologically annotated and normalized dataset of dialectal Finnish tweets

Sami-MT

machine translation for Sámi languages

lm-vs-mt

Two Stacks Are Better Than One: A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives

OPUS-API

API for searching corpora from OPUS

dialect-topic-model

Scripts and metadata for the paper "Corpus-based dialectometry with topic models"