Spicy — Robust Parsers for Protocols & File Formats

Stable Development Documentation Spicy Manual Changelog Release Notes CHANGES Build status

Overview

Spicy is a parser generator that makes it easy to create robust C++ parsers for network protocols, file formats, and more. Spicy is a bit like a "yacc for protocols", but it's much more than that: It's an all-in-one system enabling developers to write attributed grammars that describe both syntax and semantics of an input format using a single, unified language. Think of Spicy as a domain-specific scripting language for all your parsing needs.

The Spicy toolchain turns such grammars into efficient C++ parsing code that exposes an API to host applications for instantiating parsers, feeding them input, and retrieving their results. At runtime, parsing proceeds fully incrementally—and potentially highly concurrently—on input streams of arbitrary size. Compilation of Spicy parsers takes place either just-in-time at startup (through a C++ compiler); or ahead-of-time either by creating pre-compiled shared libraries, or by giving you generated C++ code that you can link into your application.

Spicy comes with a Zeek plugin that enables adding new protocol and file analyzers to Zeek without having to write any C++ code. You define the grammar, specify which Zeek events to generate, and Spicy takes care of the rest. There's also a Zeek analyzers package that provides Zeek with several new, Spicy-based analyzers.

See our collection of example grammars to get a sense of what Spicy looks like.

Installation

We provide pre-built Spicy binaries for several Linux platforms, as well as a Homebrew formula (and also binaries) for installation on macOS. You can also pull a Docker image from Docker Hub, or leverage one of several included Docker files as a starting point. Of course, you can also just build Spicy from source directly. See the installation instructions for more information on any of these options.

Documentation

Please read the Spicy Manual, which provides the following sections:

• Installation
• Getting Started
• FAQ
• Tutorial: A Real Analyzer
• Programming in Spicy
• Toolchain
• Zeek Integration
• Custom Host Applications
• Release Notes
• Developer's Manual

Getting in touch

Having trouble using Spicy? Have ideas how to make Spicy better? We'd like to hear from you!

• Report issues on the GitHub ticket tracker.

• Ask the #spicy channel on Zeek's Slack.

• Propose ideas, and show what you're doing, on GitHub's Discussions.

• Visit the Zeek community to discuss Spicy under the Spicy tag.

• To follow development, subscribe to the commits mailing list (it can be noisy!).

License

Spicy is open source and released under a BSD license, which allows for pretty much unrestricted use as long as you leave the license header in place. You fully own any parsers that Spicy generates from your grammars.

History

Spicy was originally developed as a research prototype at the International Computer Science Institute with funding from the U.S. National Science Foundation. Since then, Spicy has been rebuilt from the ground up by Corelight, which has contributed the new implementation to the Zeek Project.