• Stars
    star
    105
  • Rank 320,663 (Top 7 %)
  • Language
    Python
  • License
    MIT License
  • Created over 7 years ago
  • Updated 12 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Python library to load structured table data from files/strings/URL with various data format: CSV / Excel / Google-Sheets / HTML / JSON / LDJSON / LTSV / Markdown / SQLite / TSV.

Summary

pytablereader is a Python library to load structured table data from files/strings/URL with various data format: CSV / Excel / Google-Sheets / HTML / JSON / LDJSON / LTSV / Markdown / SQLite / TSV.

PyPI package version Supported Python versions Supported Python implementations CI status of Linux/macOS/Windows Test coverage CodeQL

Features

  • Extract structured tabular data from various data format:
  • Supported data sources are:
    • Files on a local file system
    • Accessible URLs
    • str instances
  • Loaded table data can be used as:

Examples

Load a CSV table

Sample Code:
import pytablereader as ptr
import pytablewriter as ptw


# prepare data ---
file_path = "sample_data.csv"
csv_text = "\n".join([
    '"attr_a","attr_b","attr_c"',
    '1,4,"a"',
    '2,2.1,"bb"',
    '3,120.9,"ccc"',
])

with open(file_path, "w") as f:
    f.write(csv_text)

# load from a csv file ---
loader = ptr.CsvTableFileLoader(file_path)
for table_data in loader.load():
    print("\n".join([
        "load from file",
        "==============",
        "{:s}".format(ptw.dumps_tabledata(table_data)),
    ]))

# load from a csv text ---
loader = ptr.CsvTableTextLoader(csv_text)
for table_data in loader.load():
    print("\n".join([
        "load from text",
        "==============",
        "{:s}".format(ptw.dumps_tabledata(table_data)),
    ]))
Output:
load from file
==============
.. table:: sample_data

    ======  ======  ======
    attr_a  attr_b  attr_c
    ======  ======  ======
         1     4.0  a
         2     2.1  bb
         3   120.9  ccc
    ======  ======  ======

load from text
==============
.. table:: csv2

    ======  ======  ======
    attr_a  attr_b  attr_c
    ======  ======  ======
         1     4.0  a
         2     2.1  bb
         3   120.9  ccc
    ======  ======  ======

Get loaded table data as pandas.DataFrame instance

Sample Code:
import pytablereader as ptr

loader = ptr.CsvTableTextLoader(
    "\n".join([
        "a,b",
        "1,2",
        "3.3,4.4",
    ]))
for table_data in loader.load():
    print(table_data.as_dataframe())
Output:
     a    b
0    1    2
1  3.3  4.4

For more information

More examples are available at https://pytablereader.rtfd.io/en/latest/pages/examples/index.html

Installation

Install from PyPI

pip install pytablereader

Some of the formats require additional dependency packages, you can install the dependency packages as follows:

  • Excel
    • pip install pytablereader[excel]
  • Google Sheets
    • pip install pytablereader[gs]
  • Markdown
    • pip install pytablereader[md]
  • Mediawiki
    • pip install pytablereader[mediawiki]
  • SQLite
    • pip install pytablereader[sqlite]
  • Load from URLs
    • pip install pytablereader[url]
  • All of the extra dependencies
    • pip install pytablereader[all]

Install from PPA (for Ubuntu)

sudo add-apt-repository ppa:thombashi/ppa
sudo apt update
sudo apt install python3-pytablereader

Dependencies

Optional Python packages

Optional packages (other than Python packages)

  • libxml2 (faster HTML conversion)
  • pandoc (required when loading MediaWiki file)

Documentation

https://pytablereader.rtfd.io/

Related Project

  • pytablewriter
    • Tabular data loaded by pytablereader can be written another tabular data format with pytablewriter.

Sponsors

Charles Becker (chasbecker) onetime: Arturi0 onetime: Dmitry Belyaev (b4tman)

Become a sponsor

More Repositories

1

sqlitebiter

A CLI tool to convert CSV / Excel / HTML / JSON / Jupyter Notebook / LDJSON / LTSV / Markdown / SQLite / SSV / TSV / Google-Sheets to a SQLite database file.
Python
831
star
2

tcconfig

A tc command wrapper. Make it easy to set up traffic control of network bandwidth/latency/packet-loss/packet-corruption/etc. to a network-interface/Docker-container(veth).
Python
767
star
3

pytablewriter

pytablewriter is a Python library to write a table in various formats: AsciiDoc / CSV / Elasticsearch / HTML / JavaScript / JSON / LaTeX / LDJSON / LTSV / Markdown / MediaWiki / NumPy / Excel / Pandas / Python / reStructuredText / SQLite / TOML / TSV.
Python
594
star
4

allpairspy

A python library for test combinations generator. The generator allows one to create a set of tests using "pairwise combinations" method, reducing a number of combinations of variables into a lesser set that covers most situations.
Python
251
star
5

pathvalidate

A Python library to sanitize/validate a string such as filenames/file-paths/etc.
Python
207
star
6

SimpleSQLite

SimpleSQLite is a Python library to simplify SQLite database operations: table creation, data insertion and get data as other data formats. Simple ORM functionality for SQLite.
Python
128
star
7

DateTimeRange

DateTimeRange is a Python library to handle a time range. e.g. check whether a time is within the time range, get the intersection of time ranges, truncate a time range, iterate through a time range, and so forth.
Python
100
star
8

pingparsing

pingparsing is a CLI-tool/Python-library parser and transmitter for ping command ↪️
Python
75
star
9

pytest-md-report

A pytest plugin to generate test outcomes reports with markdown table format.
Python
28
star
10

subprocrunner

A Python wrapper library for subprocess module.
Python
21
star
11

typepy

A Python library for variable type checker/validator/converter at a run time.
Python
16
star
12

DataProperty

A Python library for extract property from data.
Python
15
star
13

humanreadable

humanreadable is a Python library to convert human-readable values to other units.
Python
15
star
14

cleanpy

cleanpy is a CLI tool to remove caches and temporary files related to Python.
Python
11
star
15

sqliteschema

sqliteschema is a Python library to dump table schema of a SQLite database file.
Python
10
star
16

elasticsearch-faker

elasticsearch-faker is a CLI tool to generate fake data for Elasticsearch.
Python
10
star
17

tcolorpy

tcolopy is a Python library to apply true color for terminal text.
Python
10
star
18

excelrd

excelrd is a modified version of xlrd to work for the latest Python versions.
Python
9
star
19

cmakew

cmakew is a CMake wrapper CLI tool.
Python
9
star
20

ghscard

A JavaScript widget to generate interactive GitHub user/repository/organization cards for static web pages (like GitHub pages).
JavaScript
9
star
21

tabledata

tabledata is a Python library to represent tabular data.
Python
7
star
22

thank-you-stars

thank-you-stars is a CLI tool to stars to a PyPI package and its dependencies hosted on GitHub ⭐
Python
7
star
23

mbstrdecoder

Python library for multi-byte character string decoder.
Python
6
star
24

pytest-discord

A pytest plugin to notify test results to a Discord channel.
Python
5
star
25

tblfaker

tblfaker is a Python library to generate fake tabular data.
Python
5
star
26

CriterionSample

📔 Examples of Criterion (https://github.com/Snaipe/Criterion)
C
5
star
27

python-lib-project-template

A project template for a Python library with CI configurations
Python
5
star
28

envinfopy

envinfopy is a Python Library to get execution environment information.
Python
4
star
29

NFStest

NFStest mirror with command help wiki.
Python
4
star
30

readmemaker

A Python utility library to help make a README file from document files.
Python
4
star
31

retryrequests

A Python library that make HTTP requests with exponential back-off retry by using requests package.
Python
4
star
32

gtest-project-template

C++ test project template using Google Test.
C++
4
star
33

appconfigpy

A Python library to create/load an application configuration file.
Python
3
star
34

msgfy

msgfy is a Python library for convert Exception instance to a human-readable error message.
Python
3
star
35

python-cli-project-template

A project template for a Python CLI tool with CI configurations
Python
2
star
36

releasecmd

releasecmd is a release subcommand for setup.py (setuptools.setup). The subcommand create a git tag and push, and upload packages to PyPI.
Python
2
star
37

homebrew-sqlitebiter

Homebrew Formula for sqlitebiter
Ruby
2
star
38

docker-opengrok

Docker image for OpenGrok
Dockerfile
2
star
39

vscode-snippets

User defined snippets for Visual Studio Code
2
star
40

pytablewriter-altrow-theme

pytablewriter-altrow-theme is a pytablewriter plugin to provide a theme that colored rows alternatively.
Python
1
star
41

thutils

Personal python utility library (may delete in the future)
Python
1
star
42

install-gh

Simple one-liner installer of gh (cli/cli) from the release assets.
Shell
1
star
43

Connectathon_README

Connectathon test suite README
HTML
1
star
44

docker-alias

Docker aliases and functions 🐳
Shell
1
star
45

universal-ctags-installer

Build and installation script for Universal Ctags
Shell
1
star
46

df-diskcache

df-diskcache is a Python library for caching pandas.DataFrame objects to local disk.
Python
1
star
47

pathvalidate-cli

pathvalidate-cli is a command line interface for pathvalidate library. The tool can do sanitize/validate a string such as file-names/file-paths.
Python
1
star
48

gh-upgrade

gh-upgrade is a gh extension that updates the gh and its extensions to the latest version.
Shell
1
star
49

PythonExamples

📔 Learning Python libraries by examples.
Jupyter Notebook
1
star
50

dotfiles

♻️ dotfiles
Python
1
star