• Stars
    star
    28,700
  • Rank 600 (Top 0.02 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 10 years ago
  • Updated 9 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

๐Ÿ’ซ Industrial-strength Natural Language Processing (NLP) in Python

spaCy: Industrial-strength NLP

spaCy is a library for advanced Natural Language Processing in Python and Cython. It's built on the very latest research, and was designed from day one to be used in real products.

spaCy comes with pretrained pipelines and currently supports tokenization and training for 70+ languages. It features state-of-the-art speed and neural network models for tagging, parsing, named entity recognition, text classification and more, multi-task learning with pretrained transformers like BERT, as well as a production-ready training system and easy model packaging, deployment and workflow management. spaCy is commercial open-source software, released under the MIT license.

๐Ÿ’ซ Version 3.7 out now! Check out the release notes here.

tests Current Release Version pypi Version conda Version Python wheels Code style: black
PyPi downloads Conda downloads spaCy on Twitter

๐Ÿ“– Documentation

Documentation
โญ๏ธ spaCy 101 New to spaCy? Here's everything you need to know!
๐Ÿ“š Usage Guides How to use spaCy and its features.
๐Ÿš€ New in v3.0 New features, backwards incompatibilities and migration guide.
๐Ÿช Project Templates End-to-end workflows you can clone, modify and run.
๐ŸŽ› API Reference The detailed reference for spaCy's API.
โฉ GPU Processing Use spaCy with CUDA-compatible GPU processing.
๐Ÿ“ฆ Models Download trained pipelines for spaCy.
๐Ÿฆ™ Large Language Models Integrate LLMs into spaCy pipelines.
๐ŸŒŒ Universe Plugins, extensions, demos and books from the spaCy ecosystem.
โš™๏ธ spaCy VS Code Extension Additional tooling and features for working with spaCy's config files.
๐Ÿ‘ฉโ€๐Ÿซ Online Course Learn spaCy in this free and interactive online course.
๐Ÿ“ฐ Blog Read about current spaCy and Prodigy development, releases, talks and more from Explosion.
๐Ÿ“บ Videos Our YouTube channel with video tutorials, talks and more.
๐Ÿ›  Changelog Changes and version history.
๐Ÿ’ Contribute How to contribute to the spaCy project and code base.
๐Ÿ‘• Swag Support us and our work with unique, custom-designed swag!
Tailored Solutions Custom NLP consulting, implementation and strategic advice by spaCyโ€™s core development team. Streamlined, production-ready, predictable and maintainable. Send us an email or take our 5-minute questionnaire, and well'be in touch! Learn more โ†’

๐Ÿ’ฌ Where to ask questions

The spaCy project is maintained by the spaCy team. Please understand that we won't be able to provide individual support via email. We also believe that help is much more valuable if it's shared publicly, so that more people can benefit from it.

Type Platforms
๐Ÿšจ Bug Reports GitHub Issue Tracker
๐ŸŽ Feature Requests & Ideas GitHub Discussions
๐Ÿ‘ฉโ€๐Ÿ’ป Usage Questions GitHub Discussions ยท Stack Overflow
๐Ÿ—ฏ General Discussion GitHub Discussions

Features

  • Support for 70+ languages
  • Trained pipelines for different languages and tasks
  • Multi-task learning with pretrained transformers like BERT
  • Support for pretrained word vectors and embeddings
  • State-of-the-art speed
  • Production-ready training system
  • Linguistically-motivated tokenization
  • Components for named entity recognition, part-of-speech-tagging, dependency parsing, sentence segmentation, text classification, lemmatization, morphological analysis, entity linking and more
  • Easily extensible with custom components and attributes
  • Support for custom models in PyTorch, TensorFlow and other frameworks
  • Built in visualizers for syntax and NER
  • Easy model packaging, deployment and workflow management
  • Robust, rigorously evaluated accuracy

๐Ÿ“– For more details, see the facts, figures and benchmarks.

โณ Install spaCy

For detailed installation instructions, see the documentation.

  • Operating system: macOS / OS X ยท Linux ยท Windows (Cygwin, MinGW, Visual Studio)
  • Python version: Python 3.7+ (only 64 bit)
  • Package managers: pip ยท conda (via conda-forge)

pip

Using pip, spaCy releases are available as source packages and binary wheels. Before you install spaCy and its dependencies, make sure that your pip, setuptools and wheel are up to date.

pip install -U pip setuptools wheel
pip install spacy

To install additional data tables for lemmatization and normalization you can run pip install spacy[lookups] or install spacy-lookups-data separately. The lookups package is needed to create blank models with lemmatization data, and to lemmatize in languages that don't yet come with pretrained models and aren't powered by third-party libraries.

When using pip it is generally recommended to install packages in a virtual environment to avoid modifying system state:

python -m venv .env
source .env/bin/activate
pip install -U pip setuptools wheel
pip install spacy

conda

You can also install spaCy from conda via the conda-forge channel. For the feedstock including the build recipe and configuration, check out this repository.

conda install -c conda-forge spacy

Updating spaCy

Some updates to spaCy may require downloading new statistical models. If you're running spaCy v2.0 or higher, you can use the validate command to check if your installed models are compatible and if not, print details on how to update them:

pip install -U spacy
python -m spacy validate

If you've trained your own models, keep in mind that your training and runtime inputs must match. After updating spaCy, we recommend retraining your models with the new version.

๐Ÿ“– For details on upgrading from spaCy 2.x to spaCy 3.x, see the migration guide.

๐Ÿ“ฆ Download model packages

Trained pipelines for spaCy can be installed as Python packages. This means that they're a component of your application, just like any other module. Models can be installed using spaCy's download command, or manually by pointing pip to a path or URL.

Documentation
Available Pipelines Detailed pipeline descriptions, accuracy figures and benchmarks.
Models Documentation Detailed usage and installation instructions.
Training How to train your own pipelines on your data.
# Download best-matching version of specific model for your spaCy installation
python -m spacy download en_core_web_sm

# pip install .tar.gz archive or .whl from path or URL
pip install /Users/you/en_core_web_sm-3.0.0.tar.gz
pip install /Users/you/en_core_web_sm-3.0.0-py3-none-any.whl
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.0.0/en_core_web_sm-3.0.0.tar.gz

Loading and using models

To load a model, use spacy.load() with the model name or a path to the model data directory.

import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("This is a sentence.")

You can also import a model directly via its full name and then call its load() method with no arguments.

import spacy
import en_core_web_sm

nlp = en_core_web_sm.load()
doc = nlp("This is a sentence.")

๐Ÿ“– For more info and examples, check out the models documentation.

โš’ Compile from source

The other way to install spaCy is to clone its GitHub repository and build it from source. That is the common way if you want to make changes to the code base. You'll need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, virtualenv and git installed. The compiler part is the trickiest. How to do that depends on your system.

Platform
Ubuntu Install system-level dependencies via apt-get: sudo apt-get install build-essential python-dev git .
Mac Install a recent version of XCode, including the so-called "Command Line Tools". macOS and OS X ship with Python and git preinstalled.
Windows Install a version of the Visual C++ Build Tools or Visual Studio Express that matches the version that was used to compile your Python interpreter.

For more details and instructions, see the documentation on compiling spaCy from source and the quickstart widget to get the right commands for your platform and Python version.

git clone https://github.com/explosion/spaCy
cd spaCy

python -m venv .env
source .env/bin/activate

# make sure you are using the latest pip
python -m pip install -U pip setuptools wheel

pip install -r requirements.txt
pip install --no-build-isolation --editable .

To install with extras:

pip install --no-build-isolation --editable .[lookups,cuda102]

๐Ÿšฆ Run tests

spaCy comes with an extensive test suite. In order to run the tests, you'll usually want to clone the repository and build spaCy from source. This will also install the required development dependencies and test utilities defined in the requirements.txt.

Alternatively, you can run pytest on the tests from within the installed spacy package. Don't forget to also install the test utilities via spaCy's requirements.txt:

pip install -r requirements.txt
python -m pytest --pyargs spacy

More Repositories

1

thinc

๐Ÿ”ฎ A refreshing functional take on deep learning, compatible with your favorite libraries
Python
2,777
star
2

spacy-course

๐Ÿ‘ฉโ€๐Ÿซ Advanced NLP with spaCy: A free online course
Python
2,254
star
3

sense2vec

๐Ÿฆ† Contextually-keyed word vectors
Python
1,585
star
4

spacy-models

๐Ÿ’ซ Models for the spaCy Natural Language Processing (NLP) library
Python
1,486
star
5

spacy-transformers

๐Ÿ›ธ Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy
Python
1,310
star
6

projects

๐Ÿช End-to-end NLP workflows from prototype to production
Python
1,226
star
7

spacy-llm

๐Ÿฆ™ Integrating LLMs into structured NLP pipelines
Python
870
star
8

curated-transformers

๐Ÿค– A PyTorch library of curated Transformer models and their composable components
Python
820
star
9

spacy-streamlit

๐Ÿ‘‘ spaCy building blocks and visualizers for Streamlit apps
Python
757
star
10

spacy-stanza

๐Ÿ’ฅ Use the latest Stanza (StanfordNLP) research models directly in spaCy
Python
707
star
11

prodigy-recipes

๐Ÿณ Recipes for the Prodigy, our fully scriptable annotation tool
Jupyter Notebook
458
star
12

wasabi

๐Ÿฃ A lightweight console printing and formatting toolkit
Python
434
star
13

cymem

๐Ÿ’ฅ Cython memory pool for RAII-style memory management
Cython
433
star
14

srsly

๐Ÿฆ‰ Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)
Python
407
star
15

displacy

๐Ÿ’ฅ displaCy.js: An open-source NLP visualiser for the modern web
JavaScript
344
star
16

lightnet

๐ŸŒ“ Bringing pjreddie's DarkNet out of the shadows #yolo
C
319
star
17

prodigy-openai-recipes

โœจ Bootstrap annotation with zero- & few-shot learning via OpenAI GPT-3
Python
315
star
18

spacy-notebooks

๐Ÿ’ซ Jupyter notebooks for spaCy examples and tutorials
Jupyter Notebook
284
star
19

spacy-services

๐Ÿ’ซ REST microservices for various spaCy-related tasks
Python
239
star
20

cython-blis

๐Ÿ’ฅ Fast matrix-multiplication as a self-contained Python library โ€“ no system dependencies!
C
208
star
21

displacy-ent

๐Ÿ’ฅ displaCy-ent.js: An open-source named entity visualiser for the modern web
CSS
196
star
22

jupyterlab-prodigy

๐Ÿงฌ A JupyterLab extension for annotating data with Prodigy
TypeScript
186
star
23

tokenizations

Robust and Fast tokenizations alignment library for Rust and Python https://tamuhey.github.io/tokenizations/
Rust
178
star
24

spacymoji

๐Ÿ’™ Emoji handling and meta data for spaCy with custom extension attributes
Python
177
star
25

wheelwright

๐ŸŽก Automated build repo for Python wheels and source packages
Python
173
star
26

catalogue

Super lightweight function registries for your library
Python
169
star
27

confection

๐Ÿฌ Confection: the sweetest config system for Python
Python
164
star
28

spacy-dev-resources

๐Ÿ’ซ Scripts, tools and resources for developing spaCy
Python
125
star
29

radicli

๐Ÿ•Š๏ธ Radically lightweight command-line interfaces
Python
96
star
30

spacy-experimental

๐Ÿงช Cutting-edge experimental spaCy components and features
Python
93
star
31

spacy-lookups-data

๐Ÿ“‚ Additional lookup tables and data resources for spaCy
Python
91
star
32

talks

๐Ÿ’ฅ Browser-based slides or PDFs of our talks and presentations
JavaScript
90
star
33

thinc-apple-ops

๐Ÿ Make Thinc faster on macOS by calling into Apple's native Accelerate library
Cython
87
star
34

healthsea

Healthsea is a spaCy pipeline for analyzing user reviews of supplementary products for their effects on health.
Python
84
star
35

preshed

๐Ÿ’ฅ Cython hash tables that assume keys are pre-hashed
Cython
78
star
36

spacy-huggingface-pipelines

๐Ÿ’ฅ Use Hugging Face text and token classification pipelines directly in spaCy
Python
54
star
37

spacy-ray

โ˜„๏ธ Parallel and distributed training with spaCy and Ray
Python
53
star
38

ml-datasets

๐ŸŒŠ Machine learning dataset loaders for testing and example scripts
Python
45
star
39

assets

๐Ÿ’ฅ Explosion Assets
43
star
40

weasel

๐Ÿฆฆ weasel: A small and easy workflow system
Python
41
star
41

murmurhash

๐Ÿ’ฅ Cython bindings for MurmurHash2
C++
40
star
42

spacy-huggingface-hub

๐Ÿค— Push your spaCy pipelines to the Hugging Face Hub
Python
39
star
43

vscode-prodigy

๐Ÿงฌ A VS Code extension for annotating data with Prodigy
TypeScript
29
star
44

spacy-alignments

๐Ÿ’ซ A spaCy package for Yohei Tamura's Rust tokenizations library
Python
26
star
45

wikid

Generate a SQLite database from Wikipedia & Wikidata dumps.
Python
23
star
46

spacy-benchmarks

๐Ÿ’ซ Runtime performance comparison of spaCy against other NLP libraries
Python
20
star
47

spacy-vscode

spaCy extension for Visual Studio Code
Python
20
star
48

spacy-curated-transformers

spaCy entry points for Curated Transformers
Python
16
star
49

spacy-vectors-builder

๐ŸŒธ Train floret vectors
Python
15
star
50

prodigy-hf

Train huggingface models on top of Prodigy annotations
Python
15
star
51

os-signpost

Wrapper for the macOS signpost API
Cython
11
star
52

spacy-loggers

๐Ÿ“Ÿ Logging utilities for spaCy
Python
11
star
53

prodigy-evaluate

๐Ÿ”Ž A Prodigy plugin for evaluating spaCy pipelines
Python
11
star
54

prodigy-segment

Select pixels in Prodigy via Facebook's Segment-Anything model.
Python
10
star
55

curated-tokenizers

Lightweight piece tokenization library
Cython
10
star
56

conll-2012

A slightly cleaned up version of the scripts & data for the CoNLL 2012 Coreference task.
Python
10
star
57

thinc_gpu_ops

๐Ÿ”ฎ GPU kernels for Thinc
C++
9
star
58

prodigy-pdf

A Prodigy plugin for PDF annotation
Python
8
star
59

princetondh

Code for our presentation in Princeton DH 2023 April.
Jupyter Notebook
4
star
60

spacy-legacy

๐Ÿ•ธ๏ธ Legacy architectures and other registered spaCy v3.x functions for backwards-compatibility
Python
4
star
61

prodigy-ann

A Prodigy pluging for ANN techniques
Python
3
star
62

prodigy-whisper

Audio transcription with OpenAI's whisper model in the loop.
Python
3
star
63

ec2buildwheel

Python
2
star
64

aiGrunn-2023

Materials for the aiGrunn 2023 talk on spaCy Transformer pipelines
Python
1
star
65

spacy-io-binder

๐Ÿ“’ Repository used to build Binder images for the interactive spaCy code examples
Jupyter Notebook
1
star
66

prodigy-lunr

A Prodigy plugin for document search via LUNR
Python
1
star
67

.github

:octocat: GitHub settings
1
star
68

span-labeling-datasets

Loaders for various span labeling datasets
Python
1
star