cisnlp/GlotWeb

Stars
5
Rank 2,861,937 (Top 57 %)
Language
Python
License
Creative Commons ...
Created about 1 year ago
Updated 3 months ago

cisnlp/GlotWeb

cisnlp

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

GlotWeb: Web Indexing for Low-Resource Languages -- under construction.

simalign

Obtain Word Alignments using Pretrained Language Models (e.g., mBERT)

Glot500

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023

GlotLID

GlotLID: Language Identification with Support for More Than 2000 Labels -- EMNLP 2023

semi-markov-crf

Code for paper "Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging"

GlotScript

GlotScript: A Resource and Tool for Low Resource Writing System Identification -- LREC 2024

parcoure

ParCourE - Parallel Corpus Explorer

ofa

A Framework aims to wisely initialize unseen subword embeddings in PLMs for efficient large-scale continued pretraining

GlotCC

GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages -- under review

Jupyter Notebook

bias-in-nlp

Literature overview: gender bias in natural language processing

mPLM-Sim

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models

graph-align

code for EMNLP graph align paper

Taxi1500

TransMI

TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data

TransliCo

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models

GlotStoryBook

Children StoryBooks for 180 langauges.

Jupyter Notebook

ColexificationNet

Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs

Jupyter Notebook

cisnlp.github.io

Homepage of cisnlp

MaskLID

MaskLID: Code-Switching Language Identification through Iterative Masking -- ACL 2024

Transliteration-PPA

Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment

lohoravens-webpage

XAMPLER

XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples

Spatial_Schemas