• Stars
    star
    507
  • Rank 87,068 (Top 2 %)
  • Language
    Rust
  • License
    Other
  • Created over 6 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Rust grammar tool libraries and binaries

Grammar and parsing libraries for Rust

Bors enabled lrpar on crates.io lrlex on crates.io lrtable on crates.io cfgrammar on crates.io

grmtools is a suite of Rust libraries and binaries for parsing text, both at compile-time, and run-time. Most users will probably be interested in the compile-time Yacc feature, which allows traditional .y files to be used (mostly) unchanged in Rust.

Quickstart

A minimal example using this library consists of two files (in addition to the grammar and lexing definitions). First we need to create a file build.rs in the root of our project with the following content:

use cfgrammar::yacc::YaccKind;
use lrlex::CTLexerBuilder;

fn main() {
    CTLexerBuilder::new()
        .lrpar_config(|ctp| {
            ctp.yacckind(YaccKind::Grmtools)
                .grammar_in_src_dir("calc.y")
                .unwrap()
        })
        .lexer_in_src_dir("calc.l")
        .unwrap()
        .build()
        .unwrap();
    Ok(())
}

This will generate and compile a parser and lexer, where the definitions for the lexer can be found in src/calc.l:

%%
[0-9]+ "INT"
\+ "+"
\* "*"
\( "("
\) ")"
[\t ]+ ;

and where the definitions for the parser can be found in src/calc.y:

%start Expr
%avoid_insert "INT"
%%
Expr -> Result<u64, ()>:
      Expr '+' Term { Ok($1? + $3?) }
    | Term { $1 }
    ;

Term -> Result<u64, ()>:
      Term '*' Factor { Ok($1? * $3?) }
    | Factor { $1 }
    ;

Factor -> Result<u64, ()>:
      '(' Expr ')' { $2 }
    | 'INT'
      {
          let v = $1.map_err(|_| ())?;
          parse_int($lexer.span_str(v.span()))
      }
    ;
%%
// Any functions here are in scope for all the grammar actions above.

fn parse_int(s: &str) -> Result<u64, ()> {
    match s.parse::<u64>() {
        Ok(val) => Ok(val),
        Err(_) => {
            eprintln!("{} cannot be represented as a u64", s);
            Err(())
        }
    }
}

We can then use the generated lexer and parser within our src/main.rs file as follows:

use std::env;

use lrlex::lrlex_mod;
use lrpar::lrpar_mod;

// Using `lrlex_mod!` brings the lexer for `calc.l` into scope. By default the
// module name will be `calc_l` (i.e. the file name, minus any extensions,
// with a suffix of `_l`).
lrlex_mod!("calc.l");
// Using `lrpar_mod!` brings the parser for `calc.y` into scope. By default the
// module name will be `calc_y` (i.e. the file name, minus any extensions,
// with a suffix of `_y`).
lrpar_mod!("calc.y");

fn main() {
    // Get the `LexerDef` for the `calc` language.
    let lexerdef = calc_l::lexerdef();
    let args: Vec<String> = env::args().collect();
    // Now we create a lexer with the `lexer` method with which we can lex an
    // input.
    let lexer = lexerdef.lexer(&args[1]);
    // Pass the lexer to the parser and lex and parse the input.
    let (res, errs) = calc_y::parse(&lexer);
    for e in errs {
        println!("{}", e.pp(&lexer, &calc_y::token_epp));
    }
    match res {
        Some(r) => println!("Result: {:?}", r),
        _ => eprintln!("Unable to evaluate expression.")
    }
}

For more information on how to use this library please refer to the grmtools book, which also includes a more detailed quickstart guide.

Examples

lrpar contains several examples on how to use the lrpar/lrlex libraries, showing how to generate parse trees and ASTs, use start conditions/states or execute code while parsing.

Documentation

Latest release master
grmtools book grmtools book
cfgrammar cfgrammar
lrpar lrpar
lrlex lrlex
lrtable lrtable

Documentation for all past and present releases

More Repositories

1

krun

High fidelity benchmark runner
Python
84
star
2

mattermost-github-integration

GitHub integration for Mattermost
Python
78
star
3

snare

snare: GitHub webhooks daemon
Rust
60
star
4

eco

Editor for language composition
Python
56
star
5

lang_tester

Rust testing framework for compilers and VMs
Rust
49
star
6

lua_benchmarking

Lua benchmark suite
Lua
21
star
7

libkalibera

libkalibera
Python
17
star
8

depub

Reduce the visibility of elements in a Rust code base
Rust
17
star
9

vob

Vector of Bits
Rust
15
star
10

libgc

A library for garbage collection in Rust.
Rust
12
star
11

warmup_stats

A VM performance data analysis system
Python
11
star
12

cactus

Immutable cactus stack
Rust
10
star
13

user_rdpmc

Enable RDPMC from Linux userspace.
C
9
star
14

warmup_experiment

Experiment designed to investigate JIT warmup times.
Python
8
star
15

yksom

Rust
8
star
16

sparsevec

Compress sparse tables using row displacement
Rust
6
star
17

ykrustc

Yorick Meta-tracer
Rust
6
star
18

llrm

The Low Level RPython Machine: A RPython interpreter for LLVM bytecode.
Python
5
star
19

packedvec

Store vectors of integers efficiently
Rust
5
star
20

grammars

A collection of language grammars for `lrpar`
Yacc
5
star
21

bcvm

LLVM bitcode interpreter.
Rust
4
star
22

phdrs

A Rust wrapper around `dl_iterate_phdr(3)`
Rust
4
star
23

softdevbib

soft-dev.org BibTeX
TeX
2
star
24

lrtable

Rust LR parser table generator
Rust
2
star
25

fm

Non-backtracking fuzzy text matching
Rust
1
star
26

sd_audit

Audit our public repos for security vulnerabilities
Python
1
star
27

diffract

A Rust implementation of the GumTree differ
Rust
1
star
28

vtable_bench

Fat pointers vs. inner vtables experiment.
Rust
1
star
29

ykstackmaps

Stackmap section parser
LLVM
1
star
30

warmup_paper

Warmup paper
TeX
1
star
31

pavel.yaiwr

Yet Another Interpreter Written In Rust
Rust
1
star
32

grass

Rust
1
star
33

gcmalloc

A Rust allocator with support for conservative garbage collection.
Rust
1
star
34

lrpar

Rust LR parser
Rust
1
star
35

new_benchmarking_experiment

The new benchmarking experiment
Python
1
star
36

v8_handles_paper

TeX
1
star