• Stars
    star
    70
  • Rank 410,932 (Top 9 %)
  • Language
    Lua
  • Created almost 13 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Lexing & Syntax Highlighting in Lua (using LPeg)

LXSH: Lexing & Syntax Highlighting in Lua

LXSH is a collection of [lexers] lexing and [syntax highlighters] highlighting written in Lua lua using the excellent pattern-matching library LPeg lpeg. Several syntaxes are currently supported: Lua, C, BibTeX and shell script. The syntax highlighters support three output formats: HTML html designed to be easily embedded in web pages, LaTeX latex which can be used to generate high quality PDF files and RTF rtf which can be used in graphical text editors like Microsoft Word and LibreOffice (formerly OpenOffice). Three predefined color schemes are included. Here are some examples of the supported input languages, output formats and color schemes:

Earendel Slate Wiki
Lua: HTML PDF RTF HTML PDF RTF HTML PDF RTF
Lua (interactive prompt): HTML PDF RTF HTML PDF RTF HTML PDF RTF
C: HTML PDF RTF HTML PDF RTF HTML PDF RTF
BibTeX: HTML PDF RTF HTML PDF RTF HTML PDF RTF
Shell script: HTML PDF RTF HTML PDF RTF HTML PDF RTF

As you may have noticed in the above examples, the syntax highlighters replace standard library identifiers (and then some) with hyperlinks to the relevant documentation. You can also try switching between style sheets while staying on the same web page by using your web browser's View 鈫 Page styles menu (this works using so-called "alternate style sheets").

Installation

The easiest way to download and install LXSH is using LuaRocks luarocks:

$ luarocks install lxsh

If you don't have LuaRocks installed you can [download the latest release] zipball directly from GitHub as a ZIP archive. To install create an lxsh directory in your [$LUA_PATH] lua_path and copy the contents of the src directory from the ZIP archive to the lxsh directory so that you end up with the following structure:

  • $LUA_PATH/lxsh/init.lua
  • $LUA_PATH/lxsh/lexers/*.lua
  • $LUA_PATH/lxsh/highlighters/*.lua
  • $LUA_PATH/lxsh/formatters/*.lua
  • $LUA_PATH/lxsh/colors/*.lua
  • $LUA_PATH/lxsh/docs/*.lua

Usage

If you want to call a lexer or access an LPeg pattern defined by a lexer you can do so as follows (this example demonstrates the Lua lexer but the other lexers work the same way):

> -- Load the LXSH module.
> require 'lxsh'

> -- Run the lexer on a string of source code.
> for kind, text, lnum, cnum in lxsh.lexers.lua.gmatch 'i = i + 1\n-- example' do
>>  print(string.format('%s: %q (%i:%i)', kind, text, lnum, cnum))
>> end
identifier: "i" (1:1)
whitespace: " " (1:2)
operator:   "=" (1:3)
whitespace: " " (1:4)
identifier: "i" (1:5)
whitespace: " " (1:6)
operator:   "+" (1:7)
whitespace: " " (1:8)
number:     "1" (1:9)
whitespace: "\n" (1:10)
comment:    "-- example" (2:1)

> -- Use one of the patterns defined by the lexer.
> lxsh.lexers.lua.patterns.comment:match '--[=[ this is a long comment ]=]'

Note that you only need to load the main LXSH module with require(), the lexer and highlighter submodules are automatically loaded as they're first used. Lexers define the following functions:

  • lexer.find(subject [, init [, options ]]) takes a string and optional starting position, matches a single token (anchored) and returns two values: the token kind and the last matched character
  • lexer.match(subject [, init [, options ]]) takes a string and optional starting position, matches a single token (anchored) and returns two values: the token kind and the matched text
  • lexer.gmatch(subject [, options]) returns an iterator that produces four values on each iteration: the kind of token (see below), the matched text, the starting line number and the starting column number (line and column numbers start at 1)

When options is given it should be a table of options that can be used to configure lexers. Currently only one option is defined: When you pass join_identifiers=true to the Lua lexer, expressions like io.write will be matched as a single identifier instead of the sequence (identifier io, operator ., identifier write).

The syntax highlighters can be used as follows:

> print(lxsh.highlighters.lua("require 'lpeg'", { formatter = lxsh.formatters.html, external = true }))
<pre class="sourcecode lua">
<a href="http://www.lua.org/manual/5.1/manual.html#pdf-require" class="library">require</a>
<span class="constant">'lpeg'</span>
</pre>

You can customize the output of the highlighters by passing a table with one or more of the following options:

  • encodews: Instruct the HTML highlighter to replace newlines with <br> elements and ordinary spaces with non-breaking spaces so that whitespace is preserved when the highlighted code isn't embedded in a <pre> block
  • external: By default the HTML highlighter generates inline CSS which makes it easier to use the output directly but it also bloats the size significantly. If you want to reduce the size and don't mind including an external style sheet you can set this option to true. You'll need to make sure the required styles are loaded, e.g. by embedding the output of lxsh.highlighters.html.preamble(preferred, includeswitcher) in the <head> of your HTML document (the preferred argument indicates the default style sheet and if you pass includeswitcher as true then an interactive style sheet switcher using JavaScript is included)
  • colors: The color scheme to use, one of the following:
    • lxsh.colors.earendel based on the [Vim color scheme Earendel] earendel by Georg Dahn (this is the default)
    • lxsh.colors.slate based on the [Vim color scheme Slate] slate by Ralph Amissah
    • lxsh.colors.wiki based on the style of the [lua-users wiki] lua_wiki

Tokens produced by the lexers

The Lua lexer produces the following tokens:

  • comment
  • constant (true, false and nil)
  • error (invalid input)
  • identifier
  • keyword
  • number
  • operator
  • string
  • whitespace

The C lexer produces the following tokens:

  • comment
  • character (literals like 'C')
  • string (literals like "Lua")
  • error (invalid input)
  • identifier
  • keyword
  • number
  • operator
  • preprocessor
  • whitespace

The BibTeX lexer produces the following tokens:

  • entry (e.g. @Book)
  • field (e.g. author)
  • identifier
  • string
  • number
  • operator
  • delimiter
  • whitespace
  • error (invalid input)

The shell script lexer produces the following tokens:

  • comment
  • number
  • string
  • variable
  • operator
  • keyword
  • command
  • error (invalid input)

Contact

If you have questions, bug reports, suggestions, etc. the author can be contacted at [email protected]. The latest version is available at http://peterodding.com/code/lua/lxsh/ and http://github.com/xolox/lua-lxsh.

License

This software is licensed under the [MIT license] mit.
漏 2011 Peter Odding <[email protected]>.

More Repositories

1

vim-notes

Easy note taking in Vim
Vim Script
1,581
star
2

vim-easytags

Automated tag file generation and syntax highlighting of tags in Vim
Vim Script
1,020
star
3

vim-session

Extended session management for Vim (:mksession on steroids)
Vim Script
960
star
4

python-coloredlogs

Colored terminal output for Python's logging module
Python
514
star
5

vim-misc

Miscellaneous auto-load Vim scripts
Vim Script
364
star
6

python-humanfriendly

Human friendly input/output for text interfaces using Python
Python
298
star
7

vim-lua-ftplugin

Lua file type plug-in for the Vim text editor
Vim Script
185
star
8

vim-shell

Improved integration between Vim and its environment (fullscreen, open URL, background command execution)
Vim Script
169
star
9

python-rotate-backups

Simple command line interface for backup rotation
Python
157
star
10

dedupfs

A Python FUSE file system that features transparent deduplication and compression which make it ideal for archiving backups.
Python
118
star
11

vim-colorscheme-switcher

Makes it easy to quickly switch between color schemes in Vim
Vim Script
114
star
12

python-executor

Programmer friendly subprocess wrapper
Python
97
star
13

vim-lua-inspect

Semantic highlighting for Lua in Vim
Lua
94
star
14

vim-reload

Automatic reloading of Vim scripts ((file-type) plug-ins, auto-load/syntax/indent scripts, color schemes)
Vim Script
80
star
15

lua-apr

Apache Portable Runtime binding for Lua
C
57
star
16

python-rsync-system-backup

Linux system backups powered by rsync
Python
47
star
17

python-negotiator

Scriptable KVM/QEMU guest agent implemented in Python
Python
41
star
18

vim-tools

Python scripts that make it easier (for me) to publish Vim plug-ins
Python
41
star
19

python-deb-pkg-tools

Debian packaging tools
Python
39
star
20

python-apt-mirror-updater

Automated, robust apt-get mirror selection for Debian and Ubuntu
Python
38
star
21

python-verboselogs

Verbose logging for Python's logging module
Python
33
star
22

vim-pyref

A plug-in for the Vim text editor that provides context-sensitive documentation for Python source code.
Vim Script
31
star
23

python-capturer

Easily capture stdout/stderr of the current process and subprocesses
Python
29
star
24

python-proc

Simple interface to Linux process information
Python
22
star
25

python-redock

Human friendly wrapper around Docker
Python
16
star
26

sync-dotfiles

Quickly push your dotfiles from your workstation to your servers.
16
star
27

python-auto-adjust-display-brightness

Automatically adjust Linux display brightness
Python
15
star
28

python-chat-archive

Easy to use offline chat archive
Python
15
star
29

vim-publish

A Vim plug-in that helps you publish hyperlinked, syntax highlighted source code
Vim Script
14
star
30

python-property-manager

Useful property variants for Python programming
Python
13
star
31

python-qpass

Frontend for pass (the standard unix password manager)
Python
12
star
32

mopidy-simple-webclient

Simple and minimalistic Mopidy HTTP client, touch friendly, works in most (mobile) web browsers
JavaScript
12
star
33

python-naturalsort

Simple natural order sorting API for Python that just works
Python
12
star
34

python-vcs-repo-mgr

Version control repository manager
Python
11
star
35

mpd-myfm

A client for Music Player Daemon that fills your playlist based on similar artists from Last.fm
Python
9
star
36

lua-buildbot

A build bot for popular Lua projects (Lua 5.1, LuaJIT 1 & LuaJIT 2)
Lua
8
star
37

python-preview-markup

Live preview Markdown and reStructuredText files as HTML in a web browser
Python
8
star
38

python-debuntu-tools

Debian and Ubuntu system administration tools
Python
7
star
39

python-linux-utils

Linux system administration tools for Python
Python
7
star
40

python-apache-manager

Monitor and control Apache web server workers from Python
Python
6
star
41

python-update-dotdee

Generic modular configuration file manager
Python
6
star
42

python-npm-accel

Accelerator for npm, the Node.js package manager
Python
6
star
43

vim-tlv-mode

Transaction-Level Verilog support for Vim
Vim Script
5
star
44

python-pdiffcopy

Fast large file synchronization inspired by rsync
Python
5
star
45

python-crypto-drive-manager

Unlock all your encrypted drives with one pass phrase
Python
5
star
46

python-gentag

Simple and powerful tagging for Python objects
Python
4
star
47

python-dwim

Location aware application launcher
Python
3
star