BobLd/DocumentLayoutAnalysis BobLd/DocumentLayoutAnalysis

  • Stars
    star
    572
  • Rank 77,995 (Top 2 %)
  • Language
    C#
  • Created about 5 years ago
  • Updated about 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
BobLd/DocumentLayoutAnalysis
github

BobLd

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Document Layout Analysis resources repos for development with PdfPig.

Document Layout Analysis repos for development with PdfPig.

From wikipedia: Document layout analysis is the process of identifying and categorizing the regions of interest in the scanned image of a text document. A reading system requires the segmentation of text zones from non-textual ones and the arrangement in their correct reading order. Detection and labeling of the different zones (or blocks) as text body, illustrations, math symbols, and tables embedded in a document is called geometric layout analysis. But text zones play different logical roles inside the document (titles, captions, footnotes, etc.) and this kind of semantic labeling is the scope of the logical layout analysis.

Related projects

Cited by

Resources

Text extraction

Word segmentation

example

Page segmentation

Recursive XY Cut code PdfPig

The X-Y cut segmentation algorithm, also referred to as recursive X-Y cuts (RXYC) algorithm, is a tree-based top-down algorithm. The root of the tree represents the entire document page. All the leaf nodes together represent the final segmentation. The RXYC algorithm recursively splits the document into two or more smaller rectangular blocks which represent the nodes of the tree. At each step of the recursion, the horizontal and vertical projection profiles of each node are computed. Then, the valleys along the horizontal and vertical directions, VX and VY, are compared to corresponding predefined thresholds TX and TY. If the valley is larger than the threshold, the node is split at the mid-point of the wider of VX and VY into two children nodes. The process continues until no leaf node can be split further. Then, noise regions are removed using noise removal thresholds TnX and TnY. source example

Docstrum code PdfPig code PdfPig

The Docstrum algorithm by Gorman is a bottom-up approach based on nearest-neighborhood clustering of connected components extracted from the document image. After noise removal, the connected components are separated into two groups, one with dominant characters and another one with characters in titles and section heading, using a character size ratio factor fd. Then, K nearest neighbors are found for each connected component. Then, text-lines are found by computing the transitive closure on within-line nearest neighbor pairings using a threshold ft. Finally, text-lines are merged to form text blocks using a parallel distance threshold fpa and a perpendicular distance threshold fpe. source example or example

Voronoi

The Voronoi-diagram based segmentation algorithm by Kise et al. is also a bottom-up algorithm. In the first step, it extracts sample points from the boundaries of the connected components using a sampling rate sr. Then, noise removal is done using a maximum noise zone size threshold nm, in addition to width, height, and aspect ratio thresholds. After that the Voronoi diagram is generated using sample points obtained from the borders of the connected components. Superfluous Voronoi edges are deleted using a criterion involving the area ratio threshold ta, and the inter-line spacing margin control factor fr. Since we evaluate all algorithms on document pages with Manhattan layouts, a modified version of the algorithm is used to generate rectangular zones.source

Constrained text-line detection code PdfPig

The layout analysis approach by Breuel finds text-lines as a two step process:

  1. Find tall whitespace rectangles and evaluate them as candidates for gutters, column separators, etc. The algorithm for finding maximal empty whitespace is described in Breuel. The whitespace rectangles are returned in order of decreasing quality and are allowed a maximum overlap of Om.
  2. The whitespace rectangles representing the columns are used as obstacles in a robust least square, globally optimal text-line detection algorithm. Then, the bounding box of all the characters making the text-line is computed. The method was merely intended by its author as a demonstration of the application of two geometric algorithms, and not as a complete layout analysis system; nevertheless, we included it in the comparison because it has already proven useful in some applications. It is also nearly parameter free and resolution independent.source

PDF/A standard

PDF/A-1a compliant document make the following information available:

  1. Language specification
  2. Hierarchical document structure
  3. Tagged text spans and descriptive text for images and symbols
  4. Character mappings to Unicode

Zone classification/extraction & Reading order

Reading order

Table

Systems

Sparse line

Chart and diagram

Mathematical expression

Margins recognition

NLP & ML

Pre-trained models

Workshops

Related topics

Bounding boxes

Images

Shape detection

Character Recognition

Layout Similarity

Dehyphenation

Data structure

Datasets

Output file format

Validate and transform between OCR file formats (hOCR, ALTO, PAGE, FineReader)

Pdf page to image converter

A Pdf page to image converter is available to help in the research proces. It relies on the mupdf library, available in the sumatra pdf reader.

Pdf layout analysis viewer

A Pdf layout analysis viewer is available, also relies on the mupdf library.

viewer

More Repositories

1

tabula-sharp

Extract tables from PDF files (port of tabula-java)
C#
154
star
2

YOLOv4MLNet

Use the YOLO v4 and v5 (ONNX) models for object detection in C# using ML.Net
C#
79
star
3

camelot-sharp

A C# library to extract tabular data from PDFs (port of camelot Python version using PdfPig).
C#
31
star
4

PdfPigMLNetBlockClassifier

Proof of concept of training a simple Region Classifier using PdfPig and ML.NET (LightGBM). The objective is to classify each text block in a pdf document page as either title, text, list, table and image.
C#
23
star
5

lean-monitor-2

Windows/Linux/MacOS Desktop App to browse QuantConnect Lean engine's backtest and monitor live performances. Original project https://github.com/mirthestam/lean-monitor
C#
23
star
6

Caly

Cross-platform pdf reader application
C#
23
star
7

simple-docstrum

A step-by-step C# implementation of the Docstrum algorithm
Jupyter Notebook
22
star
8

YOLOv3MLNet

Use the YOLO v3 (ONNX) model for object detection in C# using ML.Net
C#
20
star
9

Nzy3d

A .Net API for 3d charts (based on nzy3d-api)
C#
19
star
10

PublayNet-maskrcnn-mlnet

Using a MaskRCNN model trained on the PublayNet dataset with ML.Net in C# / .Net for Document layout analysis and page segmmentation task.
C#
16
star
11

youtube-transcript-api-sharp

This is a C# API which allows you to get the transcript/subtitles for a given YouTube video. It also works for automatically generated subtitles and it does not require a headless browser, like other selenium based solutions do! Ported from https://github.com/jdepoix/youtube-transcript-api
C#
15
star
12

PdfPig.Rendering.Skia

Cross-platform library to render pdf documents as images with PdfPig using SkiaSharp
C#
13
star
13

RamerDouglasPeuckerNetV2

An algorithm that decimates a curve composed of line segments to a similar curve with fewer points.
C#
12
star
14

RamerDouglasPeuckerNet

Ramer-Douglas-Peucker algorithm for 2D data in C#
C#
10
star
15

PdfPigSvmRegionClassifier

Proof of concept of a simple SVM Region Classifier using PdfPig and Accord.Net. The objective is to classify each text block in a pdf document page as either title, text, list, table and image.
C#
7
star
16

SnipsNlu

Snips NLU C# wrapper library to extract meaning from text
C#
5
star
17

RapidOcrNet

Cross-platform OCR processing using PaddleOCR ONNX models. Based on RapidAI's RapidOCR
C#
3
star
18

PublayNetSharp

Extract and convert PubLayNet data to PageXml format
C#
2
star
19

GenerateRandomScatter

Tool to generate random scatter plots with their ground truth in C# using OxyPlot.
C#
1
star
20

VirtualizingStackPanelDuplicatesIssue

Minimum app for issue with VirtualizingStackPanel
C#
1
star
21

OxyPlot.Dark.Wpf

OxyPlot modification with a dark theme
C#
1
star
22

IccProfile

ICC Profile reader
C#
1
star