pandas-stubs: Public type stubs for pandas

What is it?

These are public type stubs for pandas, following the convention of providing stubs in a separate package, as specified in PEP 561. The stubs cover the most typical use cases of pandas. In general, these stubs are narrower than what is possibly allowed by pandas, but follow a convention of suggesting best recommended practices for using pandas.

The stubs are likely incomplete in terms of covering the published API of pandas. NOTE: The current 2.0.x releases of pandas-stubs do not support all of the new features of pandas 2.0. See this tracker to understand the current compatibility with version 2.0.

The stubs are tested with mypy and pyright and are currently shipped with the Visual Studio Code extension pylance.

Usage

Let’s take this example piece of code in file round.py

import pandas as pd

decimals = pd.DataFrame({'TSLA': 2, 'AMZN': 1})
prices = pd.DataFrame(data={'date': ['2021-08-13', '2021-08-07', '2021-08-21'],
                            'TSLA': [720.13, 716.22, 731.22], 'AMZN': [3316.50, 3200.50, 3100.23]})
rounded_prices = prices.round(decimals=decimals)

Mypy won't see any issues with that, but after installing pandas-stubs and running it again:

mypy round.py

we get the following error message:

round.py:6: error: Argument "decimals" to "round" of "DataFrame" has incompatible type "DataFrame"; expected "Union[int, Dict[Any, Any], Series[Any]]"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

And, if you use pyright:

pyright round.py

you get the following error message:

 round.py:6:40 - error: Argument of type "DataFrame" cannot be assigned to parameter "decimals" of type "int | Dict[Unknown, Unknown] | Series[Unknown]" in function "round"
    Type "DataFrame" cannot be assigned to type "int | Dict[Unknown, Unknown] | Series[Unknown]"
      "DataFrame" is incompatible with "int"
      "DataFrame" is incompatible with "Dict[Unknown, Unknown]"
      "DataFrame" is incompatible with "Series[Unknown]" (reportGeneralTypeIssues)

And after confirming with the docs we can fix the code:

decimals = pd.Series({'TSLA': 2, 'AMZN': 1})

Version Numbering Convention

The version number x.y.z.yymmdd corresponds to a test done with pandas version x.y.z, with the stubs released on the date mm/yy/dd. It is anticipated that the stubs will be released more frequently than pandas as the stubs are expected to evolve due to more public visibility.

Where to get it

The source code is currently hosted on GitHub at: https://github.com/pandas-dev/pandas-stubs

Binary installers for the latest released version are available at the Python Package Index (PyPI) and on conda-forge.

# conda
conda install pandas-stubs

# or PyPI
pip install pandas-stubs

Dependencies

• pandas: powerful Python data analysis toolkit
• typing-extensions >= 4.2.0 - supporting the latest typing extensions

Installation from sources

• Make sure you have python >= 3.8 installed.
• Install poetry

# conda
conda install poetry

# or PyPI
pip install poetry

• Install the project dependencies

poetry update -vvv

• Build and install the distribution

poetry run poe build_dist
poetry run poe install_dist

License

BSD 3

Documentation

Documentation is a work-in-progress.

Background

These stubs are the result of a strategic effort led by the core pandas team to integrate Microsoft type stub repository with the VirtusLabs pandas_stubs repository.

These stubs were initially forked from the Microsoft project at https://github.com/microsoft/python-type-stubs as of this commit.

We are indebted to Microsoft and that project for providing the initial set of public type stubs. We are also grateful for the original pandas-stubs project at https://github.com/VirtusLab/pandas-stubs, which created the framework for testing the stubs.

Differences between type declarations in pandas and pandas-stubs

The https://github.com/pandas-dev/pandas/ project has type declarations for some parts of pandas, both for the internal and public API's. Those type declarations are used to make sure that the pandas code is internally consistent.

The https://github.com/pandas-dev/pandas-stubs/ project provides type declarations for the pandas public API. The philosophy of these stubs can be found at https://github.com/pandas-dev/pandas-stubs/blob/main/docs/philosophy.md/. While it would be ideal if the pyi files in this project would be part of the pandas distribution, this would require consistency between the internal type declarations and the public declarations, and the scope of a project to create that consistency is quite large. That is a long term goal. Finally, another goal is to do more frequent releases of the pandas-stubs than is done for pandas, in order to make the stubs more useful.

If issues are found with the public stubs, pull requests to correct those issues are welcome. In addition, pull requests on the pandas repository to fix the same issue are welcome there as well. However, since the goals of typing in the two projects are different (internal consistency vs. public usage), it may be a challenge to create consistent type declarations across both projects. See https://pandas.pydata.org/docs/development/contributing_codebase.html#type-hints for a discussion of typing standards used within the pandas code.

Getting help

Ask questions and report issues on the pandas-stubs repository.

Discussion and Development

Most development discussions take place on GitHub in the pandas-stubs repository. Further, the pandas-dev mailing list can also be used for specialized discussions or design issues, and a Gitter channel is available for quick development related questions.

Contributing to pandas-stubs

All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome. See https://github.com/pandas-dev/pandas-stubs/tree/main/docs/ for instructions.