• Stars
    star
    377
  • Rank 113,535 (Top 3 %)
  • Language
    Python
  • License
    BSD 3-Clause "New...
  • Created over 12 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

An attempt to port David Beazley's PLY to RPython, and give it a cooler API.

RPLY

https://secure.travis-ci.org/alex/rply.png

Welcome to RPLY! A pure Python parser generator, that also works with RPython. It is a more-or-less direct port of David Beazley's awesome PLY, with a new public API, and RPython support.

You can find the documentation online.

Basic API:

from rply import ParserGenerator, LexerGenerator
from rply.token import BaseBox

lg = LexerGenerator()
# Add takes a rule name, and a regular expression that defines the rule.
lg.add("PLUS", r"\+")
lg.add("MINUS", r"-")
lg.add("NUMBER", r"\d+")

lg.ignore(r"\s+")

# This is a list of the token names. precedence is an optional list of
# tuples which specifies order of operation for avoiding ambiguity.
# precedence must be one of "left", "right", "nonassoc".
# cache_id is an optional string which specifies an ID to use for
# caching. It should *always* be safe to use caching,
# RPly will automatically detect when your grammar is
# changed and refresh the cache for you.
pg = ParserGenerator(["NUMBER", "PLUS", "MINUS"],
        precedence=[("left", ['PLUS', 'MINUS'])], cache_id="myparser")

@pg.production("main : expr")
def main(p):
    # p is a list, of each of the pieces on the right hand side of the
    # grammar rule
    return p[0]

@pg.production("expr : expr PLUS expr")
@pg.production("expr : expr MINUS expr")
def expr_op(p):
    lhs = p[0].getint()
    rhs = p[2].getint()
    if p[1].gettokentype() == "PLUS":
        return BoxInt(lhs + rhs)
    elif p[1].gettokentype() == "MINUS":
        return BoxInt(lhs - rhs)
    else:
        raise AssertionError("This is impossible, abort the time machine!")

@pg.production("expr : NUMBER")
def expr_num(p):
    return BoxInt(int(p[0].getstr()))

lexer = lg.build()
parser = pg.build()

class BoxInt(BaseBox):
    def __init__(self, value):
        self.value = value

    def getint(self):
        return self.value

Then you can do:

parser.parse(lexer.lex("1 + 3 - 2+12-32"))

You can also substitute your own lexer. A lexer is an object with a next() method that returns either the next token in sequence, or None if the token stream has been exhausted.

Why do we have the boxes?

In RPython, like other statically typed languages, a variable must have a specific type, we take advantage of polymorphism to keep values in a box so that everything is statically typed. You can write whatever boxes you need for your project.

If you don't intend to use your parser from RPython, and just want a cool pure Python parser you can ignore all the box stuff and just return whatever you like from each production method.

Error handling

By default, when a parsing error is encountered, an rply.ParsingError is raised, it has a method getsourcepos(), which returns an rply.token.SourcePosition object.

You may also provide an error handler, which, at the moment, must raise an exception. It receives the Token object that the parser errored on.

pg = ParserGenerator(...)

@pg.error
def error_handler(token):
    raise ValueError("Ran into a %s where it wasn't expected" % token.gettokentype())

Python compatibility

RPly is tested and known to work under Python 2.7, 3.4+, and PyPy. It is also valid RPython for PyPy checkouts from 6c642ae7a0ea onwards.

Links

More Repositories

1

what-happens-when

An attempt to answer the age old interview question "What happens when you type google.com into your browser and press enter?"
39,628
star
2

nyt-2020-election-scraper

HTML
1,761
star
3

letsencrypt-aws

Python
729
star
4

pretend

A library for stubbing in python
Python
290
star
5

django-ajax-validation

A reusable application to preform ajax validation on django forms.
Python
252
star
6

django-templatetag-sugar

This project exists to make defining template tags in Django kickass
Python
213
star
7

ecs-terraform

ECS + Terraform = Crazy delicious
HCL
203
star
8

csv-sql

Query your CSV files with SQL
Rust
192
star
9

django-fixture-generator

django-fixture-generator is a reusable django application to make writing fixtures not suck.
Python
138
star
10

zero_buffer

zero_buffer is a high-performance, zero-copy, implementation of a byte-buffer for Python.
Python
136
star
11

pyvcs

A pure python abstraction layer of multiple VCS, very lightweight.
Python
128
star
12

django-admin-histograms

A library for simple histograms in Django's admin.
Python
106
star
13

alchimia

Python
103
star
14

rust-asn1

A Rust ASN.1 (DER) serializer.
Rust
99
star
15

django-vcs

A django application for working with a VCS, analagous to some of Trac's features.
Python
86
star
16

just-use

Just Use /dev/urandom -- now with more safety at early boot
Rust
60
star
17

django-wsgi

A library for better integration between django and the WSGI world.
Python
49
star
18

python-decompiler

A decompiler for CPython bytecode
Python
48
star
19

piano-man

Python
44
star
20

django-project-skeleton

A skeleton for Django 1.4's new project template support.
Python
41
star
21

django-resume-builder

Python
36
star
22

line-counter

Like `wc -l`, but in Rust and maybe faster
Rust
29
star
23

django-plugins

A generic plugin system for django.
Python
28
star
24

otp-cop

Tooling for verifying that everyone in your orgs has 2fa enabled.
Rust
28
star
25

nba-gamethread

A game thread generator for r/nba
Python
26
star
26

vcs-translator

Python
25
star
27

bagel

Bagels are delicious
Python
22
star
28

Shore

A statically typed programming language with inspiration from C++, Python, and others.
Python
19
star
29

ct-tools

Some Rust code for submitting a cert to all of Chrome's trusted CT logs and getting back the SCTs
Rust
17
star
30

x509-validator

A pure-Python x509 validation library based on pyca/cryptography
Python
16
star
31

httpfuzz

Python
13
star
32

tracebin

The ultimate performance command and control center for PyPy.
Python
12
star
33

django-object-cache

A reusable django object cache.
12
star
34

tls-stats

A webExtension to collect TLS stats
JavaScript
11
star
35

pyelection

A python application for following the US primaries
Python
10
star
36

recipes

My Recipes
Python
10
star
37

ex-post-progress

Rust
9
star
38

evolves

A Python implementation of a genetic algorithm for generating images from polygons
Python
9
star
39

doc-extractor

Go
8
star
40

tls-cal

Python
8
star
41

election-sim

An application for following and simulating the 2008 US Presidential Election.
Python
7
star
42

alex-s-language

Inteded to be a compiled Python, by translating source files to C++
Python
7
star
43

optimizer-model

A work in progress model for a new optimizer for PyPy's JIT.
Python
7
star
44

http-client-bench

Python
7
star
45

revocation-tracker

Python
7
star
46

flickr-avatars

a small webservice to make getting flickr users' avatars
Python
6
star
47

yaffi

Yet Another FFI
Python
6
star
48

json_writer

Python
5
star
49

rdio-export

JavaScript
4
star
50

space-hogs

A game made in python with pyglet.
Python
4
star
51

csv-to-parquet

Go
3
star
52

independent-study-tracebin

Documentation for my independent study on tracebin
Python
3
star
53

montgomery-ladder-go

An implementation of modular exponentiation using the montgomery ladder in Go.
Go
2
star
54

api-serializer

Random hack, don't use.
Python
2
star
55

azure-pipelines-docker-test

2
star
56

win32k-stuff

JavaScript
2
star
57

client-beta

Go
2
star
58

temp-rust-coverage

Rust
1
star
59

doc8

Python
1
star
60

temp-gha-macos-python

1
star
61

botan-fuzzers

Fuzzer instrumentation for botan
C++
1
star