LD-Net

Check Our New NER Toolkit🚀🚀🚀

• Inference:
  • LightNER: inference w. models pre-trained / trained w. any following tools, efficiently.
• Training:
  • LD-Net: train NER models w. efficient contextualized representations.
  • VanillaNER: train vanilla NER models w. pre-trained embedding.
• Distant Training:
  • AutoNER: train NER models w.o. line-by-line annotations and get competitive performance.

LD-Net provides sequence labeling models featuring:

• Efficiency: constructing efficient contextualized representations without retraining language models.
• Portability: well-organized, easy-to-modify and well-documented.

Remarkablely, our pre-trained NER model achieved:

• 92.08 test F1 on the CoNLL03 NER task.
• 160K words/sec decoding speed (6X speedup compared to its original model).

Details about LD-Net can be accessed at: https://arxiv.org/abs/1804.07827.

• Model notes
• Benchmarks
• Pretrained model
  • Language models
  • Named Entity Recognition
  • Chunking
• Training
  • Dependency
  • Data
  • Model
  • Command
• Inference
• Citation

Model Notes

Benchmarks

Pretrained Models

Here we provide both pre-trained language models and pre-trained sequence labeling models.

Language Models

Our pretrained language model contains word embedding, 10-layer densely-connected LSTM and adative softmax, and achieve an average PPL of 50.06 on the one billion benchmark dataset.

Named Entity Recognition

The original pre-trained named entity tagger achieves 91.95 F1, the pruned tagged achieved 92.08 F1.

Chunking

The original pre-trained named entity tagger achieves 96.13 F1, the pruned tagged achieved 95.79 F1.

Training

Demo Scripts

To pruning the original LD-Net for the CoNLL03 NER, please run:

bash ldnet_ner_prune.sh

To pruning the original LD-Net for the CoNLL00 Chunking, please run:

bash ldnet_np_prune.sh

Dependency

Our package is based on Python 3.6 and the following packages:

numpy
tqdm
torch-scope
torch==0.4.1

Data

Pre-process scripts are available in pre_seq and pre_word_ada, while pre-processed data has been stored in:

Model

Our implementations are available in model_seq and model_word_ada, and the documentations are hosted in ReadTheDoc

Inference

For model inference, please check our LightNER package

Citation

If you find the implementation useful, please cite the following paper: Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling

@inproceedings{liu2018efficient,
  title = "{Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling}", 
  author = {Liu, Liyuan and Ren, Xiang and Shang, Jingbo and Peng, Jian and Han, Jiawei}, 
  booktitle = {EMNLP}, 
  year = 2018, 
}