erlxml - Erlang XML parsing library based on pugixml

pugixml is the fastest dom parser available in c++ based on the benchmarks available here. The streaming parsing is implemented by splitting the stream into independent stanzas which are parsed using pugixml. The algorithm for splitting is pretty fast but in order to keep it simple as possible adds some limitations at this moment for the streaming mode:

• not supporting CDATA
• not supporting comments with special xml characters inside
• not supporting DOCTYPE

All above limitations applies only to streaming mode and not for DOM parsing mode.

erlxml:parse(<<"<foo attr1='bar'>Some Value</foo>">>).

Which results in

{ok,{xmlel,<<"foo">>,
           [{<<"attr1">>,<<"bar">>}],
           [{xmlcdata,<<"Some Value">>}]}}

Xml = {xmlel,<<"foo">>,
    [{<<"attr1">>,<<"bar">>}],  % Attributes
    [{xmlcdata,<<"Some Value">>}]   % Elements
},
erlxml:to_binary(Xml).

Which results in

<<"<foo attr1=\"bar\">Some Value</foo>">>

Chunk1 = <<"<stream><foo attr1=\"bar">>,
Chunk2 = <<"\">Some Value</foo></stream>">>,
{ok, Parser} = erlxml:new_stream(),
{ok,[{xmlstreamstart,<<"stream">>,[]}]} = erlxml:parse_stream(Parser, Chunk1),
Rs = erlxml:parse_stream(Parser, Chunk2),
{ok,[{xmlel,<<"foo">>,
        [{<<"attr1">>,<<"bar">>}],
        [{xmlcdata,<<"Some Value">>}]},
     {xmlstreamend,<<"stream">>}]} = Rs.

When you create a stream using new_stream/1 you can specify the following options:

• stanza_limit - Specify the maximum size a stanza can have. In case the library parses more than this amount of bytes without finding a stanza will return and error {error, {max_stanza_limit_hit, binary()}}. Example: {stanza_limit, 65000}. By default is 0 which means unlimited.

• strip_non_utf8 - Will strip from attributes values and node values elements all invalid utf8 characters. This is considered user input and might have malformed chars. Default is false.

The benchmark code is inside the benchmark folder. You need to get exml from Erlang Solutions and fast_xml from ProcessOne as dependencies because all measurements are against this libraries.

• exml version used: 3.0.1
• fast_xml version used: 1.1.30

All tests are run with 3 concurrency levels (how many erlang processes are spawn)

• C1 (concurrency level 1)
• C5 (concurrency level 5)
• C10 (concurrency level 10)

Parse the same stanza defined in benchmark/benchmark.erl for 600000 times:

benchmark:bench_parsing(erlxml|exml, 600000, 1|5|10).

Note:

• Starting version 3.0.0, exml improved a lot by replacing Expat with RapidXML, for example before this switch the results were 26704.861 ms for C1, 7094.698 for C5 and 5812.703 for C10. Now results are comparable with erlxml.
• Difference between erlxml and exml performances because is so small we can say they offer same performance from speed point of view.

Encode the same erlang term defined in benchmark/benchmark.erl for 600000 times:

benchmark:bench_encoding(erlxml|exml, 600000, 1|5|10).

Will load all stanza's from a file and run the parsing mode over that stanza's for 30000 times (total bytes processed in my test is around 1.38 GB) :

benchmark_stream:bench(exml, "/Users/silviu/Desktop/example.txt", 30000, 1).
### 32933.493 ms 42.90 MB/sec total bytes processed: 1.38 GB
benchmark_stream:bench(exml, "/Users/silviu/Desktop/example.txt", 30000, 5).
### 9231.714 ms 153.05 MB/sec total bytes processed: 1.38 GB
benchmark_stream:bench(exml, "/Users/silviu/Desktop/example.txt", 30000, 10).
### 9693.043 ms 145.77 MB/sec total bytes processed: 1.38 GB

benchmark_stream:bench(erlxml, "/Users/silviu/Desktop/example.txt", 30000, 1). 
### 10580.888 ms 133.53 MB/sec total bytes processed: 1.38 GB
benchmark_stream:bench(erlxml, "/Users/silviu/Desktop/example.txt", 30000, 5).
### 2662.581 ms 530.66 MB/sec total bytes processed: 1.38 GB
benchmark_stream:bench(erlxml, "/Users/silviu/Desktop/example.txt", 30000, 10).
### 2579.559 ms 547.74 MB/sec total bytes processed: 1.38 GB

Notes:

• Starting version 3.0.0, exml improved by replacing Expat with RapidXML with arount 27% but is still way behind erlxml.