polm/ja-tokenizer-benchmark

Stars
7
Rank 2,294,772 (Top 46 %)
Language
Python
Created about 5 years ago
Updated almost 3 years ago

polm/ja-tokenizer-benchmark

polm

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Compare the speed of various Japanese tokenizers in Python.

fugashi

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

cutlet

Japanese to romaji converter in Python

posuto

🏣📮〠 Japanese postal code data.

unidic-py

Unidic packaged for installation via pip.

ndl-crop

Script for cropping photos from the NDL.

unidic-lite

A small version of UniDic for easy pip installs.

showmemore

SHOW ME MORE OF [-----]

ipadic-py

IPAdic packaged for easy use from Python.

awesome-digital-collections

Publicly accessible digital collections.

palladian-facades

🏛️ Palladian Facade Generator for ProcJam2015

multilang-filter

Script for preprocessing multilingual Markdown.

deltos

A magic notepad. δ

gamefaces

Public domain headshots

dupdupdraw

Forthish drawing system with random program generation.

node-migemo

Japanese search regex generator

chargen

Random generator taking literature as input.

philtre

Search objects with a familiar syntax.

jp-ner

[abandoned] Work on generating an NER dataset for Japanese

jumandic-py

JumanDic packaged for use with PyPI.

shesha

Random generator toolkit

awesome-gamedev-jp

ゲーム開発に役立つリンク集

bontan.ls

Bontan is a simple scraper primarily intended for articles.

lua-mecab

Lua wrapper for Mecab Japanese morphological analyzer.

fugashi-streamlit-demo

Streamlit demo for fugashi

gutenjuice

Top books from Project Gutenberg, in raw form and extracted.

bookoff-redirect

Deal with BookOff query parameter nonsense.

fugashi-sagemaker-demo

A basic introduction to using fugashi for Japanese tokenization.

Jupyter Notebook

github-tasks.vim

Github task plugin for vim

mecab-packed

[broken/wip] Bundled mecab & unidic for installing via pip.

language-disruptor

Randomly replace words in Japanese sentences.

poine-tool

POINE関連のツール

bontan

Get embed code for a link, using OEmbed as appropriate.

yuzulabo.works

Yuzu Labo web site

mecab-manylinux1-wheel-builder

Build manylinux1 wheels with MeCab installed.

deltos.vim

A vim plugin for use with Deltos.

kanji

Kanji data package for Python

visidata-conll

CoNLL-U data loader for Visidata.

everybayes

Document classification for everyone.

jfmt.lua

Tool for wrapping Japanese text to natural width

searchy

[discontinued] Simple interactive search for Node