• Stars
    star
    192
  • Rank 201,236 (Top 4 %)
  • Language
    C
  • License
    MIT License
  • Created about 15 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Lua library for conversion between markup formats

Lunamark

Lunamark is a lua library and command-line program for conversion of markdown to other textual formats. Currently HTML, dzslides (HTML5 slides), Docbook, ConTeXt, LaTeX, and Groff man are the supported output formats, but it is easy to add new writers or modify existing ones. The markdown parser is written using a PEG grammar and can also be modified by the user.

The library is as portable as lua and has very good performance. It is roughly as fast as the author's own C library peg-markdown, two orders of magnitude faster than Markdown.pl, and three orders of magnitude faster than markdown.lua.

Links

Extensions

Lunamark's markdown parser currently supports a number of extensions (which can be turned on or off individually), including:

  • Smart typography (fancy quotes, dashes, ellipses)
  • Significant start numbers in ordered lists
  • Footnotes (both regular and inline)
  • Definition lists
  • Pandoc-style title blocks
  • Pandoc-style citations
  • Fenced code blocks
  • Flexible metadata using lua declarations

See the lunamark(1) man page for a complete list.

It is very easy to extend the library by modifying the writers, adding new writers, and even modifying the markdown parser. Some simple examples are given in the API documentation.

Benchmarks

Generated with

PROG=$program make bench

This converts the input files from the original markdown test suite concatenated together 25 times.

     0.04s   sundown
     0.15s   discount
->   0.56s   lunamark + luajit
     0.80s   peg-markdown
->   0.97s   lunamark
     4.05s   PHP Markdown
     6.11s   pandoc
   113.13s   Markdown.pl
  2322.33s   markdown.lua

Installing

If you want a standalone version of lunamark that doesn't depend on lua or other lua modules being installed on your system, just do

make standalone

Your executable will be created in the standalone directory.

If you are a lua user, you will probably prefer to install lunamark using luarocks. You can install the latest development version this way:

git clone http://github.com/jgm/lunamark.git
cd lunamark
luarocks make

Released versions will be uploaded to the luarocks repository, so you should be able to install them using:

luarocks install lunamark

There may be a short delay between the release and the luarocks upload.

Using the library

Simple usage example:

local lunamark = require("lunamark")
local opts = { }
local writer = lunamark.writer.html.new(opts)
local parse = lunamark.reader.markdown.new(writer, opts)
print(parse("Here's my *text*"))

For more examples, see API documentation.

lunamark

The lunamark executable allows easy markdown conversion from the command line. For usage instructions, see the lunamark(1) man page.

lunadoc

Lunamark comes with a simple lua library documentation tool, lunadoc. For usage instructions, see the lunadoc(1) man page. lunadoc reads source files and parses specially marked markdown comment blocks. Here is an example of the result.

Tests

The source directory contains a large test suite in tests. This includes existing Markdown and PHP Markdown tests, plus more tests for lunamark-specific features and additional corner cases.

To run the tests, use bin/shtest.

bin/shtest --help            # get usage
bin/shtest                   # run all tests
bin/shtest indent            # run all tests matching "indent"
bin/shtest -p Markdown.pl -t # run all tests using Markdown.pl, and normalize using 'tidy'

Lunamark currently fails four of the PHP Markdown tests:

  • tests/PHP_Markdown/Quotes in attributes.test: The HTML is semantically equivalent; using the -t/--tidy option to bin/shtest makes the test pass.

  • tests/PHP_Markdown/Email auto links.test: The HTML is semantically equivalent. PHP markdown does entity obfuscation, and lunamark does not. This feature could be added easily enough, but the test would still fail, because the obfuscation involves randomness. Again, using the -t/--tidy option makes the test pass.

  • tests/PHP_Markdown/Ins & del.test: PHP markdown puts extra <p> tags around <ins>hello</ins>, while lunamark does not. It's hard to tell from the markdown spec which behavior is correct.

  • tests/PHP_Markdown/Emphasis.test: A bunch of corner cases with nested strong and emphasized text. These corner cases are left undecided by the markdown spec, so in my view the PHP test suite is not normative here; I think lunamark's behavior is perfectly reasonable, and I see no reason to change.

The make test target only runs the Markdown and lunamark tests, skipping the PHP Markdown tests.

Authors

lunamark is released under the MIT license.

Most of the library is written by John MacFarlane. Hans Hagen made some major performance improvements. Khaled Hosny added the original ConTeXt writer.

The dzslides HTML, CSS, and javascript code is by Paul Rouget, released under the DWTFYWT Public License.

More Repositories

1

pandoc

Universal markup converter
Haskell
33,780
star
2

gitit

A wiki using HAppS, pandoc, and git
Haskell
2,126
star
3

djot

A light markup language
HTML
1,557
star
4

peg-markdown

An implementation of markdown in C, using a PEG grammar
C
686
star
5

pandocfilters

A python module for writing pandoc filters, with a collection of examples
Python
507
star
6

pandoc-templates

Templates for pandoc, tagged to release
HTML
418
star
7

yst

create static websites from YAML data and string templates
Haskell
373
star
8

texmath

A Haskell library for converting LaTeX math to MathML.
Haskell
291
star
9

pandoc-citeproc

Library and executable for using citeproc with pandoc
Haskell
288
star
10

skylighting

A Haskell syntax highlighting library with tokenizers derived from KDE syntax highlighting descriptions
Haskell
185
star
11

citeproc

CSL citation processing library in Haskell
Haskell
138
star
12

commonmark-hs

Pure Haskell commonmark parsing library, designed to be flexible and extensible
Haskell
130
star
13

djot.js

JavaScript implementation of djot
TypeScript
120
star
14

highlighting-kate

A syntax highlighting library in Haskell, based on Kate syntax definitions
HTML
109
star
15

cheapskate

Experimental markdown processor in Haskell
HTML
105
star
16

pandoc-types

types for representing structured documents
Haskell
105
star
17

gitit2

A reimplementation of gitit in Yesod
Haskell
94
star
18

lcmark

Flexible CommonMark converter
Lua
54
star
19

doctemplates

Pandoc-compatible templating system
Haskell
49
star
20

zip-archive

Native Haskell library for working with zip archives
Haskell
44
star
21

cmark-hs

Haskell bindings to libcmark commonmark parser
C
43
star
22

djot.lua

Lua parser for the djot light markup language
Lua
39
star
23

typst-hs

Haskell library for parsing and evaluating typst
Haskell
32
star
24

dotvim

My vim configuration
Vim Script
30
star
25

scripts

A collection of small scripts to do various things
Shell
28
star
26

filestore

A versioning file store backed by git, darcs, or mercurial
Haskell
28
star
27

pandoc-website

Source files for pandoc's website
Lua
28
star
28

illuminate

An efficient syntax highlighting library in Haskell, using alex-generated lexers
Haskell
26
star
29

emojis

Haskell library for emojis
Haskell
25
star
30

markdown-peg

A Haskell implementation of markdown using a PEG grammar
Haskell
24
star
31

pandoc-server

Simple server app for pandoc conversions.
Haskell
20
star
32

doclayout

A prettyprinting library designed for laying out plain text documents
Haskell
20
star
33

standalone-html

Incorporates external dependencies into HTML file using data: URI scheme
Haskell
19
star
34

pandoc-tex2svg

Pandoc filter to convert math to SVG using MathJax-node's tex2svg
HTML
19
star
35

cloudlib

tools for keeping a library of books and articles on Amazon's S3 and SimpleDB
Ruby
19
star
36

cmark-lua

Lua bindings to libcmark CommonMark parser
C
17
star
37

HeX

a flexible text macro system
Haskell
17
star
38

djoths

Haskell parser for the djot light markup language
Haskell
17
star
39

unicode-collation

Haskell implementation of the Unicode Collation Algorithm
Haskell
16
star
40

sep-offprint

Creates formatted "offprints" of Stanford Encyclopedia of Philosophy entries.
15
star
41

BayHac2014

Slides for my presentation on pandoc at BayHac2014
TeX
14
star
42

cmarkpdf

Steps towards a PDF renderer for cmark using libharu
C
14
star
43

lunamark-standalone

Standalone version of lunamark (compiled with no library dependencies)
C
12
star
44

commonmarker

Ruby wrapper for libcmark (CommonMark parser)
Ruby
12
star
45

ipynb

Data structures and JSON serializer/deserializer for Jupyter notebooks (.ipynb) format.
Jupyter Notebook
11
star
46

hsb2hs

Preprocessor for inserting literals with binary blobs into Haskell programs.
Haskell
11
star
47

gogar

Computer implementation of Robert Brandom's "game of giving and asking for reasons," from Making It Explicit, chapter 3.
Ruby
10
star
48

emacsd

emacs configuration
Emacs Lisp
9
star
49

hscommonmark

pure Haskell CommonMark parser
Haskell
9
star
50

recaptcha

Haskell library for using the reCAPTCHA service
Haskell
8
star
51

select-meta

Pandoc lua filter for constructing metadata from YAML data sources using queries
Lua
8
star
52

html2cmark

Lua library to convert HTML5 to commonmark
Lua
8
star
53

citeproc-hs-bin

Command-line interface to the citeproc-hs CSL citation processing library
Haskell
8
star
54

grammata

Well-typed system for generating documents in multiple formats
Haskell
7
star
55

hw2gitit

Script to convert haskellwiki pages to a gitit wiki
Haskell
7
star
56

ecstatic

Static website management using tenjin templates and YAML data files
Ruby
7
star
57

hsgit

A higher-level interface to libgit2 functions than hlibgit2
Haskell
6
star
58

trypandoc

Live demo of pandoc
JavaScript
6
star
59

pandoc-highlight

Filter and library for using pandoc with highlighting-kate
Haskell
6
star
60

commonmark-lua

Lua binding to libcmark commonmark parser
Lua
5
star
61

rfc5051

Haskell implementation of RFC5051, simple unicode collation.
Haskell
5
star
62

jgm.github.com

jgm's web pages on github
4
star
63

rocks

luarocks repository
4
star
64

cmark-fuzz-data

A minimal fuzz test suite for cmark created by american fuzzy lop and afl-cmin
3
star
65

GHCUnicodeAlt

Improved version of GHC.Unicode, with benchmarks
Haskell
3
star
66

luacmark

Lua binding to CommonMark
C
2
star
67

typst-symbols

Defines symbols and emoji used in typst
Haskell
2
star