• This repository has been archived on 20/Sep/2021
  • Stars
    star
    453
  • Rank 96,573 (Top 2 %)
  • Language
    PHP
  • Created about 12 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

The Hoa\Compiler library.

Hoa


Build status Code coverage Packagist License

Hoa is a modular, extensible and structured set of PHP libraries.
Moreover, Hoa aims at being a bridge between industrial and research worlds.

Hoa\Compiler

Help on IRC Help on Gitter Documentation Board

This library allows to manipulate LL(1) and LL(k) compiler compilers. A dedicated grammar description language is provided for the last one: the PP language.

Learn more.

Installation

With Composer, to include this library into your dependencies, you need to require hoa/compiler:

$ composer require hoa/compiler '~3.0'

For more installation procedures, please read the Source page.

Testing

Before running the test suites, the development dependencies must be installed:

$ composer install

Then, to run all the test suites:

$ vendor/bin/hoa test:run

For more information, please read the contributor guide.

Quick usage

As a quick overview, we will look at the PP language and the LL(k) compiler compiler.

The PP language

A grammar is constituted by tokens (the units of a word) and rules (please, see the documentation for an introduction to the language theory). The PP language declares tokens with the following construction:

%token [source_namespace:]name value [-> destination_namespace]

The default namespace is default. The value of a token is represented by a PCRE. We can skip tokens with the %skip construction.

As an example, we will take the simplified grammar of the JSON language. The complete grammar is in the hoa://Library/Json/Grammar.pp file. Thus:

%skip   space          \s
// Scalars.
%token  true           true
%token  false          false
%token  null           null
// Strings.
%token  quote_         "        -> string
%token  string:string  [^"]+
%token  string:_quote  "        -> default
// Objects.
%token  brace_         {
%token _brace          }
// Arrays.
%token  bracket_       \[
%token _bracket        \]
// Rest.
%token  colon          :
%token  comma          ,
%token  number         \d+

value:
    <true> | <false> | <null> | string() | object() | array() | number()

string:
    ::quote_:: <string> ::_quote::

number:
    <number>

#object:
    ::brace_:: pair() ( ::comma:: pair() )* ::_brace::

#pair:
    string() ::colon:: value()

#array:
    ::bracket_:: value() ( ::comma:: value() )* ::_bracket::

We can see the PP constructions:

  • rule() to call a rule;
  • <token> and ::token:: to declare a token;
  • | for a disjunction;
  • (…) to group multiple declarations;
  • e? to say that e is optional;
  • e+ to say that e can appear at least 1 time;
  • e* to say that e can appear 0 or many times;
  • e{x,y} to say that e can appear between x and y times;
  • #node to create a node the AST (resulting tree);
  • token[i] to unify tokens value between them.

Unification is very useful. For example, if we have a token that expresses a quote (simple or double), we could have:

%token  quote   "|'
%token  handle  \w+

string:
    ::quote:: <handle> ::quote::

So, the data "foo" and 'foo' will be valid, but also "foo' and 'foo"! To avoid this, we can add a new constraint on token value by unifying them, thus:

string:
    ::quote[0]:: <handle> ::quote[0]::

All quote[0] for the rule instance must have the same value. Another example is the unification of XML tags name.

LL(k) compiler compiler

The Hoa\Compiler\Llk\Llk class provide helpers to manipulate (load or save) a compiler. The following code will use the previous grammar to create a compiler, and we will parse a JSON string. If the parsing succeed, it will produce an AST (stands for Abstract Syntax Tree) we can visit, for example to dump the AST:

// 1. Load grammar.
$compiler = Hoa\Compiler\Llk\Llk::load(new Hoa\File\Read('Json.pp'));

// 2. Parse a data.
$ast = $compiler->parse('{"foo": true, "bar": [null, 42]}');

// 3. Dump the AST.
$dump = new Hoa\Compiler\Visitor\Dump();
echo $dump->visit($ast);

/**
 * Will output:
 *     >  #object
 *     >  >  #pair
 *     >  >  >  token(string, foo)
 *     >  >  >  token(true, true)
 *     >  >  #pair
 *     >  >  >  token(string, bar)
 *     >  >  >  #array
 *     >  >  >  >  token(null, null)
 *     >  >  >  >  token(number, 42)
 */

Pretty simple.

Compiler in CLI

This library proposes a script to parse and apply a visitor on a data with a specific grammar. Very useful. Moreover, we can use pipe (because Hoa\File\Read —please, see the Hoa\File library— supports 0 as stdin), thus:

$ echo '[1, [1, [2, 3], 5], 8]' | hoa compiler:pp Json.pp 0 --visitor dump
>  #array
>  >  token(number, 1)
>  >  #array
>  >  >  token(number, 1)
>  >  >  #array
>  >  >  >  token(number, 2)
>  >  >  >  token(number, 3)
>  >  >  token(number, 5)
>  >  token(number, 8)

You can apply any visitor classes.

Errors

Errors are well-presented:

$ echo '{"foo" true}' | hoa compiler:pp Json.pp 0 --visitor dump
Uncaught exception (Hoa\Compiler\Exception\UnexpectedToken):
Hoa\Compiler\Llk\Parser::parse(): (0) Unexpected token "true" (true) at line 1
and column 8:
{"foo" true}
       ↑
in hoa://Library/Compiler/Llk/Parser.php at line 1

Samplers

Some algorithms are available to generate data based on a grammar. We will give only one example with the coverage-based generation algorithm that will activate all branches and tokens in the grammar:

$sampler = new Hoa\Compiler\Llk\Sampler\Coverage(
    // Grammar.
    Hoa\Compiler\Llk\Llk::load(new Hoa\File\Read('Json.pp')),
    // Token sampler.
    new Hoa\Regex\Visitor\Isotropic(new Hoa\Math\Sampler\Random())
);

foreach ($sampler as $i => $data) {
    echo $i, ' => ', $data, "\n";
}

/**
 * Will output:
 *     0 => true
 *     1 => {" )o?bz " : null , " %3W) " : [false, 130    , " 6"   ]  }
 *     2 => [{" ny  " : true } ]
 *     3 => {" Ne;[3 " :[ true , true ] , " th: " : true," C[8} " :   true }
 */

Research papers

Documentation

The hack book of Hoa\Compiler contains detailed information about how to use this library and how it works.

To generate the documentation locally, execute the following commands:

$ composer require --dev hoa/devtools
$ vendor/bin/hoa devtools:documentation --open

More documentation can be found on the project's website: hoa-project.net.

Getting help

There are mainly two ways to get help:

Contribution

Do you want to contribute? Thanks! A detailed contributor guide explains everything you need to know.

License

Hoa is under the New BSD License (BSD-3-Clause). Please, see LICENSE for details.

More Repositories

1

Ruler

The Hoa\Ruler library.
PHP
625
star
2

Websocket

The Hoa\Websocket library.
PHP
422
star
3

Ustring

The Hoa\Ustring library.
PHP
402
star
4

Console

The Hoa\Console library.
PHP
366
star
5

Math

The Hoa\Math library.
PHP
366
star
6

Iterator

The Hoa\Iterator library.
PHP
333
star
7

File

The Hoa\File library.
PHP
323
star
8

Event

The Hoa\Event library
PHP
323
star
9

Consistency

The Hoa\Consistency library.
PHP
319
star
10

Exception

The Hoa\Exception library.
PHP
316
star
11

Stream

The Hoa\Stream library.
PHP
315
star
12

Regex

The Hoa\Regex library.
PHP
310
star
13

Protocol

The Hoa\Protocol library.
PHP
308
star
14

Visitor

The Hoa\Visitor library.
PHP
263
star
15

Zformat

The Hoa\Zformat library.
PHP
259
star
16

Eventsource

The Hoa\Eventsource library.
PHP
110
star
17

Central

Hoa is a modular, extensible, and structured set of PHP libraries.
PHP
105
star
18

Mime

The Hoa\Mime library.
PHP
101
star
19

Kitab

Kitab is the ideal companion for Documentation-Driven Quality: Render and Test your documentation.
PHP
79
star
20

Socket

The Hoa\Socket library.
PHP
64
star
21

Fastcgi

The Hoa\Fastcgi library.
PHP
60
star
22

Bench

The Hoa\Bench library.
PHP
56
star
23

Praspel

The Hoa\Praspel library.
PHP
40
star
24

Core

The Hoa\Core library.
PHP
35
star
25

Acl

The Hoa\Acl library.
PHP
28
star
26

Router

The Hoa\Router library.
PHP
28
star
27

Zombie

The Hoa\Zombie library.
PHP
28
star
28

Worker

The Hoa\Worker library.
PHP
26
star
29

Irc

The Hoa\Irc library.
PHP
25
star
30

Mail

The Hoa\Mail library.
PHP
24
star
31

Dns

The Hoa\Dns library.
PHP
23
star
32

Contributions-Symfony-ConsoleBridge

Hoa\Console to Symfony bundle bridge.
PHP
20
star
33

Contributions-Symfony-RulerBundle

The Hoa\Ruler Symfony2 bundle.
PHP
15
star
34

Graph

The Hoa\Graph library.
PHP
15
star
35

Database

The Hoa\Database library.
PHP
15
star
36

Locale

The Hoa\Locale library.
PHP
13
star
37

Json

The Hoa\Json library.
PHP
13
star
38

Dispatcher

The Hoa\Dispatcher library.
PHP
12
star
39

Registry

The Hoa\Registry library.
PHP
12
star
40

Cli

The Hoa\Cli library.
PHP
12
star
41

Test

The Hoa\Test library.
PHP
11
star
42

Http

The Hoa\Http library.
PHP
10
star
43

Session

The Hoa\Session library.
PHP
9
star
44

View

The Hoa\View library.
PHP
9
star
45

Option

The Hoa\Option library.
PHP
9
star
46

Cache

The Hoa\Cache library.
PHP
8
star
47

Memory

The Hoa\Memory library.
PHP
7
star
48

String

The Hoa\String library (deprecated by Hoa\Ustring).
PHP
7
star
49

-

The Hoa\  library.
PHP
7
star
50

Tree

The Hoa\Tree library.
PHP
7
star
51

Xml

The Hoa\Xml library.
PHP
7
star
52

Realdom

The Hoa\Realdom library.
PHP
7
star
53

Xyl

The Hoa\Xyl library.
PHP
7
star
54

Devtools

The Hoa\Devtools library.
PHP
6
star
55

Heap

The Hoa\Heap library.
PHP
5
star
56

Promise

The Hoa\Promise library.
PHP
5
star
57

Serialize

The Hoa\Serialize library.
PHP
5
star
58

Notification

The Hoa\Notification library.
PHP
4
star
59

Stringbuffer

The Hoa\Stringbuffer library.
PHP
4
star
60

Contributions-Atoum-PraspelExtension

Include Praspel inside atoum.
PHP
4
star
61

Translate

The Hoa\Translate library.
PHP
4
star
62

Embryo

The embryo repository helps to bootstrap an application, it is a skeleton.
PHP
4
star
63

Infrastructure

Everything related to the infrastructure of Hoa.
Shell
4
star
64

Log

The Hoa\Log library.
PHP
4
star
65

Xmlrpc

The Hoa\XmlRpc library.
PHP
4
star
66

W3

The W3 repository contains the website of Hoa.
PHP
4
star
67

Sandbox

Sandbox contains real examples.
PHP
3
star
68

Contributions-Symfony-ConsoleBundle

The Hoa\Console Symfony2 bundle.
PHP
3
star
69

Blog

Blog of the Hoa project
CSS
3
star
70

Model

The Hoa\Model library.
PHP
3
star
71

Prototype

The Hoa\Prototype library.
PHP
3
star
72

Contributions-Atom-Pp

PP —the grammar description language from Hoa\Compiler— support in Atom.
CoffeeScript
3
star
73

Literature

The literature repository contains all documentations, manuals & co.
Python
2
star
74

Keynote

Keynotes: presentations, conferences, notes, etc.
HTML
2
star
75

ActionBoard

Roadmap, actions, milestones… everything related to the schedule of Hoa is here
2
star
76

Contributions-Symfony-BenchBundle

The Hoa\Bench Symfony2 bundle.
PHP
2
star
77

Contributions-Provisioning

Provisioning scripts related to Hoa
2
star
78

Shop

Hoa's project shop
HTML
1
star
79

Contributions-Zsh-Hoa

Hoa\Cli support (autocompletion & co.) in Zsh.
1
star
80

Contributions-Vim-Pp

PP —the grammar description language from Hoa\Compiler— support in Vim.
Vim Script
1
star
81

Contributions-Atoum-Option-Extension

Add option asserter for atoum
PHP
1
star