pdfplumber

Plumb a PDF for detailed information about each text character, rectangle, and line. Plus: Table extraction and visual debugging.

Works best on machine-generated, rather than scanned, PDFs. Built on pdfminer.six.

Currently tested on Python 3.7, 3.8, 3.9, 3.10.

Translations of this document are available in: Chinese (by @hbh112233abc).

To report a bug or request a feature, please file an issue. To ask a question or request assistance with a specific PDF, please use the discussions forum.

👋 This repository’s maintainers are available to hire for PDF data-extraction consulting projects. To get a cost estimate, contact Jeremy (for projects of any size or complexity) and/or Samkit (specifically for table extraction).

Table of Contents

• Installation
• Command line interface
• Python library
• Visual debugging
• Extracting text
• Extracting tables
• Extracting form values
• Demonstrations
• Comparison to other libraries
• Acknowledgments / Contributors
• Contributing

Installation

pip install pdfplumber

Command line interface

Basic example

curl "https://raw.githubusercontent.com/jsvine/pdfplumber/stable/examples/pdfs/background-checks.pdf" > background-checks.pdf
pdfplumber < background-checks.pdf > background-checks.csv

The output will be a CSV containing info about every character, line, and rectangle in the PDF.

Options

Python library

Basic example

import pdfplumber

with pdfplumber.open("path/to/file.pdf") as pdf:
    first_page = pdf.pages[0]
    print(first_page.chars[0])

Loading a PDF

To start working with a PDF, call pdfplumber.open(x), where x can be a:

• path to your PDF file
• file object, loaded as bytes
• file-like object, loaded as bytes

The open method returns an instance of the pdfplumber.PDF class.

To load a password-protected PDF, pass the password keyword argument, e.g., pdfplumber.open("file.pdf", password = "test").

To set layout analysis parameters to pdfminer.six's layout engine, pass the laparams keyword argument, e.g., pdfplumber.open("file.pdf", laparams = { "line_overlap": 0.7 }).

Invalid metadata values are treated as a warning by default. If that is not intended, pass strict_metadata=True to the open method and pdfplumber.open will raise an exception if it is unable to parse the metadata.

The pdfplumber.PDF class

The top-level pdfplumber.PDF class represents a single PDF and has two main properties:

... and also has the following method:

The pdfplumber.Page class

The pdfplumber.Page class is at the core of pdfplumber. Most things you'll do with pdfplumber will revolve around this class. It has these main properties:

... and these main methods:

Additional methods are described in the sections below:

• Visual debugging
• Extracting text
• Extracting tables

Objects

Each instance of pdfplumber.PDF and pdfplumber.Page provides access to several types of PDF objects, all derived from pdfminer.six PDF parsing. The following properties each return a Python list of the matching objects:

• .chars, each representing a single text character.
• .lines, each representing a single 1-dimensional line.
• .rects, each representing a single 2-dimensional rectangle.
• .curves, each representing any series of connected points that pdfminer.six does not recognize as a line or rectangle.
• .images, each representing an image.
• .annots, each representing a single PDF annotation (cf. Section 8.4 of the official PDF specification for details)
• .hyperlinks, each representing a single PDF annotation of the subtype Link and having an URI action attribute

Each object is represented as a simple Python dict, with the following properties:

char properties

Note: A character’s matrix property represents the “current transformation matrix,” as described in Section 4.2.2 of the PDF Reference (6th Ed.). The matrix controls the character’s scale, skew, and positional translation. Rotation is a combination of scale and skew, but in most cases can be considered equal to the x-axis skew. The pdfplumber.ctm submodule defines a class, CTM, that assists with these calculations. For instance:

from pdfplumber.ctm import CTM
my_char = pdf.pages[0].chars[3]
my_char_ctm = CTM(*my_char["matrix"])
my_char_rotation = my_char_ctm.skew_x

line properties

rect properties

curve properties

Derived properties

Additionally, both pdfplumber.PDF and pdfplumber.Page provide access to several derived lists of objects: .rect_edges (which decomposes each rectangle into its four lines), .curve_edges (which does the same for curve objects), and .edges (which combines .rect_edges, .curve_edges, and .lines).

image properties

[To be completed.]

Obtaining higher-level layout objects via pdfminer.six

If you pass the pdfminer.six-handling laparams parameter to pdfplumber.open(...), then each page's .objects dictionary will also contain pdfminer.six's higher-level layout objects, such as "textboxhorizontal".

Visual debugging

pdfplumber's visual debugging tools can be helpful in understanding the structure of a PDF and the objects that have been extracted from it.

Note: To use this feature, you'll also need to have two additional pieces of software installed on your computer:

• ImageMagick. Installation instructions here.
• ghostscript. Installation instructions here, or simply apt install ghostscript (Ubuntu) / brew install ghostscript (Mac).

Creating a PageImage with .to_image()

To turn any page (including cropped pages) into an PageImage object, call my_page.to_image(). You can optionally pass one of the following keyword arguments:

• resolution: The desired number pixels per inch. Defaults to 72. See note below.
• width: The desired image width in pixels.
• height: The desired image width in pixels.

For instance:

im = my_pdf.pages[0].to_image(resolution=150)

From a script or REPL, im.show() will open the image in your local image viewer. But PageImage objects also play nicely with IPython/Jupyter notebooks; they automatically render as cell outputs. For example:

Note: pdfplumber passes the resolution parameter to Wand, the Python library we use for image conversion. Wand will create the image with the desired number of total pixels of height/width, but does not fully respect the resolution in the strict sense of that word: Although PNGs are capable of storing an image's resolution density as metadata, Wand's PNGs do not.

Note: .to_image(...) works as expected with Page.crop(...)/CroppedPage instances, but is unable to incorporate changes made via Page.filter(...)/FilteredPage instances.

Basic PageImage methods

Drawing methods

You can pass explicit coordinates or any pdfplumber PDF object (e.g., char, line, rect) to these methods.

Note: The methods above are built on Pillow's ImageDraw methods, but the parameters have been tweaked for consistency with SVG's fill/stroke/stroke_width nomenclature.

Troubleshooting ImageMagick on Debian-based systems

If you're using pdfplumber on a Debian-based system and encounter a PolicyError, you may be able to fix it by changing the following line in /etc/ImageMagick-6/policy.xml from this:

<policy domain="coder" rights="none" pattern="PDF" />

... to this:

<policy domain="coder" rights="read|write" pattern="PDF" />

(More details about policy.xml available here.)

Extracting text

pdfplumber can extract text from any given page (including cropped and derived pages). It can also attempt to preserve the layout of that text, as well as to identify the coordinates of words and search queries. Page objects can call the following text-extraction methods:

Extracting tables

pdfplumber's approach to table detection borrows heavily from Anssi Nurminen's master's thesis, and is inspired by Tabula. It works like this:

1. For any given PDF page, find the lines that are (a) explicitly defined and/or (b) implied by the alignment of words on the page.
2. Merge overlapping, or nearly-overlapping, lines.
3. Find the intersections of all those lines.
4. Find the most granular set of rectangles (i.e., cells) that use these intersections as their vertices.
5. Group contiguous cells into tables.

Table-extraction methods

pdfplumber.Page objects can call the following table methods:

For example:

pdf = pdfplumber.open("path/to/my.pdf")
page = pdf.pages[0]
page.extract_table()

Click here for a more detailed example.

Table-extraction settings

By default, extract_tables uses the page's vertical and horizontal lines (or rectangle edges) as cell-separators. But the method is highly customizable via the table_settings argument. The possible settings, and their defaults:

{
    "vertical_strategy": "lines", 
    "horizontal_strategy": "lines",
    "explicit_vertical_lines": [],
    "explicit_horizontal_lines": [],
    "snap_tolerance": 3,
    "snap_x_tolerance": 3,
    "snap_y_tolerance": 3,
    "join_tolerance": 3,
    "join_x_tolerance": 3,
    "join_y_tolerance": 3,
    "edge_min_length": 3,
    "min_words_vertical": 3,
    "min_words_horizontal": 1,
    "keep_blank_chars": False,
    "text_tolerance": 3,
    "text_x_tolerance": 3,
    "text_y_tolerance": 3,
    "intersection_tolerance": 3,
    "intersection_x_tolerance": 3,
    "intersection_y_tolerance": 3,
}

Table-extraction strategies

Both vertical_strategy and horizontal_strategy accept the following options:

Notes

• Often it's helpful to crop a page — Page.crop(bounding_box) — before trying to extract the table.

• Table extraction for pdfplumber was radically redesigned for v0.5.0, and introduced breaking changes.

Extracting form values

Sometimes PDF files can contain forms that include inputs that people can fill out and save. While values in form fields appear like other text in a PDF file, form data is handled differently. If you want the gory details, see page 671 of this specification.

pdfplumber doesn't have an interface for working with form data, but you can access it using pdfplumber's wrappers around pdfminer.

For example, this snippet will retrieve form field names and values and store them in a dictionary. You may have to modify this script to handle cases like nested fields (see page 676 of the specification).

pdf = pdfplumber.open("document_with_form.pdf")

fields = pdf.doc.catalog["AcroForm"].resolve()["Fields"]

form_data = {}

for field in fields:
    field_name = field.resolve()["T"]
    field_value = field.resolve()["V"]
    form_data[field_name] = field_value

Demonstrations

• Using extract_table on a California Worker Adjustment and Retraining Notification (WARN) report. Demonstrates basic visual debugging and table extraction.
• Using extract_table on the FBI's National Instant Criminal Background Check System PDFs. Demonstrates how to use visual debugging to find optimal table extraction settings. Also demonstrates Page.crop(...) and Page.extract_text(...).
• Inspecting and visualizing curve objects.
• Extracting fixed-width data from a San Jose PD firearm search report, an example of using Page.extract_text(...).

Comparison to other libraries

Several other Python libraries help users to extract information from PDFs. As a broad overview, pdfplumber distinguishes itself from other PDF processing libraries by combining these features:

• Easy access to detailed information about each PDF object
• Higher-level, customizable methods for extracting text and tables
• Tightly integrated visual debugging
• Other useful utility functions, such as filtering objects via a crop-box

It's also helpful to know what features pdfplumber does not provide:

• PDF generation
• PDF modification
• Optical character recognition (OCR)
• Strong support for extracting tables from OCR'ed documents

Specific comparisons

• pdfminer.six provides the foundation for pdfplumber. It primarily focuses on parsing PDFs, analyzing PDF layouts and object positioning, and extracting text. It does not provide tools for table extraction or visual debugging.

• PyPDF2 is a pure-Python library "capable of splitting, merging, cropping, and transforming the pages of PDF files. It can also add custom data, viewing options, and passwords to PDF files." It can extract page text, but does not provide easy access to shape objects (rectangles, lines, etc.), table-extraction, or visually debugging tools.

• pymupdf is substantially faster than pdfminer.six (and thus also pdfplumber) and can generate and modify PDFs, but the library requires installation of non-Python software (MuPDF). It also does not enable easy access to shape objects (rectangles, lines, etc.), and does not provide table-extraction or visual debugging tools.

• camelot, tabula-py, and pdftables all focus primarily on extracting tables. In some cases, they may be better suited to the particular tables you are trying to extract.

Acknowledgments / Contributors

Many thanks to the following users who've contributed ideas, features, and fixes:

• Jacob Fenton
• Dan Nguyen
• Jeff Barrera
• Bob Lannon
• Dustin Tindall
• @yevgnen
• @meldonization
• Oisín Moran
• Samkit Jain
• Francisco Aranda
• Kwok-kuen Cheung
• Marco
• Idan David
• @xv44586
• Alexander Regueiro
• Daniel Peña
• @bobluda
• @ramcdona
• @johnhuge
• Jhonatan Lopes
• Ethan Corey
• Shannon Shen
• Matsumoto Toshi
• John West

Contributing

Pull requests are welcome, but please submit a proposal issue first, as the library is in active development.

Current maintainers:

• Jeremy Singer-Vine
• Samkit Jain