Chymyst — declarative concurrency in Scala

This repository hosts Chymyst Core — a domain-specific language for purely functional, declarative concurrency, implemented as a Scala library. Chymyst is a framework-in-planning that will build upon Chymyst Core to enable creating concurrent applications declaratively.

Chymyst (pronounced “chemist”) is a clean-room implementation of the chemical machine paradigm, known in the academic world as Join Calculus (JC). JC has the same expressive power as CSP (Communicating Sequential Processes) and the Actor model, but is easier to use and reason about, more high-level and more declarative. (See also Conceptual overview of concurrency.)

The code is extensively tested under Oracle JDK 8 with Scala 2.11.8, 2.11.11, and 2.12.2-2.12.6.

Version history and roadmap

Overview of Chymyst and the chemical machine programming paradigm

Concurrency in Reactions: Get started with this extensive tutorial book

• Download the tutorial book in PDF

• Download the tutorial book in EPUB format

  • Manage book details (requires login)

Selected tutorial chapters from the book:

Project documentation at Github Pages

From actors to reactions: a guide for those familiar with the Actor model

Show me the code: An ultra-short guide to Chymyst

A "Hello, world" project

Presentations on Chymyst and on the chemical machine programming paradigm

Nov. 18, 2017: Declarative concurrent programming with Chymyst. Presented at the Scale by the Bay conference

• Talk slides with audio — Long version
• Talk slides in PDF
• Code examples used in the talk.

Oct. 16, 2017: Declarative concurrent programming with Join Calculus. Presented at the Scala Bay meetup:

• Talk slides with audio
• Talk slides (PDF)
• Code examples used in the talk.

July 2017: Industry-Strength Join Calculus: Declarative concurrent programming with Chymyst: Draft of a paper describing Chymyst and its approach to join calculus

Nov. 11, 2016: Concurrent Join Calculus in Scala. Presented at Scalæ by the Bay 2016:

• Video presentation of early version of Chymyst, then called JoinRun
• See also the talk slides revised for the current syntax.

Main features of the chemical machine

Comparison of the chemical machine vs. academic Join Calculus

Comparison of the chemical machine vs. the Actor model

Comparison of the chemical machine vs. the coroutines / channels approach (CSP)

Developer documentation for Chymyst Core

Example: the "dining philosophers" problem

The following code snippet is a complete runnable example that implements the logic of “dining philosophers” in a fully declarative and straightforward way.

import io.chymyst.jc._

object Main extends App {
   /** Print message and wait for a random time interval. */
  def wait(message: String): Unit = {
    println(message)
    Thread.sleep(scala.util.Random.nextInt(20))
  }
  
  val hungry1 = m[Unit]
  val hungry2 = m[Unit]
  val hungry3 = m[Unit]
  val hungry4 = m[Unit]
  val hungry5 = m[Unit]
  val thinking1 = m[Unit]
  val thinking2 = m[Unit]
  val thinking3 = m[Unit]
  val thinking4 = m[Unit]
  val thinking5 = m[Unit]
  val fork12 = m[Unit]
  val fork23 = m[Unit]
  val fork34 = m[Unit]
  val fork45 = m[Unit]
  val fork51 = m[Unit]
  
  site (
    go { case thinking1(_) ⇒ wait("Socrates is thinking");  hungry1() },
    go { case thinking2(_) ⇒ wait("Confucius is thinking"); hungry2() },
    go { case thinking3(_) ⇒ wait("Plato is thinking");     hungry3() },
    go { case thinking4(_) ⇒ wait("Descartes is thinking"); hungry4() },
    go { case thinking5(_) ⇒ wait("Voltaire is thinking");  hungry5() },
  
    go { case hungry1(_) + fork12(_) + fork51(_) ⇒ wait("Socrates is eating");  thinking1() + fork12() + fork51() },
    go { case hungry2(_) + fork23(_) + fork12(_) ⇒ wait("Confucius is eating"); thinking2() + fork23() + fork12() },
    go { case hungry3(_) + fork34(_) + fork23(_) ⇒ wait("Plato is eating");     thinking3() + fork34() + fork23() },
    go { case hungry4(_) + fork45(_) + fork34(_) ⇒ wait("Descartes is eating"); thinking4() + fork45() + fork34() },
    go { case hungry5(_) + fork51(_) + fork45(_) ⇒ wait("Voltaire is eating");  thinking5() + fork51() + fork45() }
  )
  // Emit molecules representing the initial state:
  thinking1() + thinking2() + thinking3() + thinking4() + thinking5()
  fork12() + fork23() + fork34() + fork45() + fork51()
  // Now reactions will start and print messages to the console.
}

Status of the project

The Chymyst Core library is in alpha pre-release, with very few API changes envisioned for the future.

The semantics of the chemical machine (restricted to single-host, multi-core computations) is fully implemented and tested on many nontrivial examples.

The library JAR is published to Maven Central.

Extensive tutorial and usage documentation is available.

Unit tests (more than 500 at the moment) exercise all aspects of the DSL provided by Chymyst. Test coverage is 100% according to codecov.io.

Test suites also complement the tutorial book and include examples such as barriers, asynchronous and synchronous rendezvous, local critical sections, parallel “or”, parallel map/reduce, parallel merge-sort, “dining philosophers”, as well as many other concurrency algorithms.

Performance benchmarks indicate that Chymyst Core can schedule about 100,000 reactions per second per CPU core, and the performance bottleneck is in submitting jobs to threads (a distant second bottleneck is pattern-matching in the internals of the library).

Run unit tests

sbt test

The tests will print some error messages and exception stack traces — this is normal, as long as all tests pass.

Some tests are timed and will fail on a slow machine.

Run the benchmark application

sbt benchmark/run will run the benchmarks.

To build the benchmark application as a self-contained JAR, run

sbt clean benchmark/assembly

Then run it as

java -jar benchmark/target/scala-2.11/benchmark-assembly-*.jar

To run with FlightRecorder:

sbt benchmark/runFR

This will create the file benchmark.jfr in the current directory. Open that file with jmc (Oracle's "Java Mission Control" tool) to inspect Code and then the "Hot Methods" (places where most time is spent).

Use Chymyst Core in your programs

Chymyst Core is published to Maven Central. To pull the dependency, add this to your build.sbt at the appropriate place:

libraryDependencies += "io.chymyst" %% "chymyst-core" % "latest.integration"

To use the chemical machine DSL, add import io.chymyst.jc._ in your Scala sources.

See the "hello, world" project for an example.

Build the library JARs

To build the library JARs:

sbt core/package core/package-doc

This will prepare JAR assemblies as well as their Scaladoc documentation packages.

The main library is in the core JAR assembly (core/target/scala-2.11/core-*.jar). User code should depend on that JAR only.

Prepare new release

• Edit the version string at the top of build.sbt
• Make sure there is a description of changes for this release at the top of docs/roadmap.md
• Commit everything to master and add tag with release version
• Push everything (with tag) to master; build must pass on CI

Publish to Sonatype

$ sbt
> project core
> +publishSigned
> sonatypeRelease

If sonatypeRelease fails due to any problems with POM files while +publishSigned succeeded, it is possible to release manually on the Sonatype web site (requires login). Go to "Staging Repositories" and execute the actions to "promote" the release.

Trivia

This drawing is by Robert Boyle, who was one of the founders of the science of chemistry. In 1661 he published a treatise titled “The Sceptical Chymist”, from which the Chymyst framework borrows its name.