gnieh/diffson

Gnieh Diffson

A Scala implementation of the RFC-6901, RFC-6902, and RFC-7396. It also provides methods to compute diffs between two Json values that produce valid Json patches or merge patches.

Note: if you still want to use the 3.x.y series (without cats), please see this documentation

• Getting Started
• Json Library
  • Note on (de)serialization
• Json Patch (RFC-6902)
  • Basic Usage
  • Simple diffs
  • Remembering old values
• Json Merge Patches (RFC-7396)

This library is published in the Maven Central Repository. You can add it to your sbt project by putting this line into your build description:

libraryDependencies += "org.gnieh" %% f"diffson-$jsonLib" % "4.1.1"

where jsonLib is either:

• spray-json
• play-json
• circe
• ujson for ujson/upickle

These versions are built for Scala 2.12, 2.13, and 3.

Scala.JS is also supported for Scala 2.12, 2.13, and 3. To use it, add this dependency to your build file:

libraryDependencies += "org.gnieh" %%% f"diffson-$jsonLib" % "4.1.1"

Diffson was first developed for spray-json, however, it is possible to use it with any json library of your liking. The only requirement is to have a Jsony for your json library. Jsony is a type class describing what operations are required to compute diffs and apply patches to Json-like types.

At the moment, diffson provides instances for spray-json, Play! Json, and circe. To use these implementations you need to link with the correct module and import the instance:

// spray-json
import diffson.sprayJson._
// play-json
import diffson.playJson._
// circe
import diffson.circe._

If you want to add support for your favorite Json library, you may only depend on diffson core module diffson-core and all you need to do then is to implement the Jsony class, which provides all the operations for diffson to be able to compute diffs and apply patches. Contribution of new Json libraries in this repository are more than welcome.

The purpose of diffson is to create and manipulate diffs and patch for Json like structures. However the supported patch formats can also be represented as Json objects. The core library doesn't mention any of this, as its sole purpose is the diff/patch computations. Given the variety of Json libraries out there and there various ways of implementing the way of (de)serializing Json values, there is no good abstraction that fits this general purpose library, and this is up to the library user to do it in the most appropriate approach given the Json library of their choosing. The various supported Json libraries in diffson provide an idiomatic way of (de)serializing the different element for each of them (e.g. the circe module provide Decoders and Encoders for all the patch types).

For instance to get circe encoder and decoder instances, you need to

import io.circe._
import diffson.circe._
import diffson.jsonpatch._

val decoder = Decoder[JsonPatch[Json]]
val encoder = Encoder[JsonPatch[Json]]

For Play! Json, you need to

import play.api.libs.json._
import diffson.playJson._
import diffson.playJson.DiffsonProtocol._
import diffson.jsonpatch._

val format = Json.format[JsonPatch[JsValue]]

For Spray Json, you need to

import spray.json._
import diffson.sprayJson._
import diffson.sprayJson.DiffsonProtocol._
import diffson.jsonpatch._

val format = implicitly[JsonFormat[JsonPatch[JsValue]]]

Although the library is quite small and easy to use, here comes a summary of its basic usage. Diffson uses a type-class approach based on the cats library. All operations that may fail are wrapped in type with a MonadError instance.

There are two different entities living in the diffson.jsonpatch and one on diffson.jsonpointer package useful to work with Json patches:

• Pointer which allows to parse and manipulate Json pointers as defined in RFC-6901,
• JsonPatch which allows to parse, create and apply Json patches as defined in RFC-6902,
• JsonDiff which allows to compute the diff between two Json values and create Json patches.

Basically if someone wants to compute the diff between two Json objects, they can execute the following:

import diffson._
import diffson.lcs._
import diffson.circe._
import diffson.jsonpatch._
import diffson.jsonpatch.lcsdiff._

import io.circe._
import io.circe.parser._

import cats._
import cats.implicits._

implicit val lcs = new Patience[Json]

val json1 = parse("""{
                    |  "a": 1,
                    |  "b": true,
                    |  "c": ["test", "plop"]
                    |}""".stripMargin)

val json2 = parse("""{
                    |  "a": 6,
                    |  "c": ["test2", "plop"],
                    |  "d": false
                    |}""".stripMargin)

val patch =
  for {
    json1 <- json1
    json2 <- json2
  } yield diff(json1, json2)

which will return a patch that can be serialized in json as:

[{
  "op":"replace",
  "path":"/a",
  "value":6
},{
  "op":"remove",
  "path":"/b"
},{
  "op":"replace",
  "path":"/c/0",
  "value":"test2"
},{
  "op":"add",
  "path":"/d",
  "value":false
}]

This example computes a diff based on an LCS, so we must provide an implicit instance of Lcs. In that case we used the Patience instance, but other could be used. See package diffson.lcs to see what implementations are available by default, or provide your own.

You can then apply an existing patch to a Json object as follows:

import scala.util.Try

import cats.implicits._

val json2 = patch[Try](json1)

which results in a json like:

{
  "d":false,
  "c":"test2",
  "a":6
}

which we can easily verify is the same as json2 modulo reordering of fields.

A patch may fail, this is why the apply method wraps the result in an F[_] with a MonadError. In this example, we used the standard Try class, but any type F with the appropriate MonadError[F, Throwable] instance in scope can be used.

The example above uses an LCS based diff, which makes it possible to have smart diffs for arrays. However, depending on your use case, this feature might not be what you want:

• LCS can be intensive to compute if you have huge arrays;
• you might want to see a modified array as a single replace operation.

To do so, instead of importing diffson.jsonpatch.lcsdiff._, import diffson.jsonpatch.simplediff._ and you do not need to provide an Lcs instance. Resulting diff will be bigger in case of different arrays, but quicker to compute.

For instance, the resulting simple diff for the example above is:

[
  {
    "op" : "replace",
    "path" : "/a",
    "value" : 6
  },
  {
    "op" : "remove",
    "path" : "/b"
  },
  {
    "op" : "replace",
    "path" : "/c",
    "value" : [
      "test2",
      "plop"
    ]
  },
  {
    "op" : "add",
    "path" : "/d",
    "value" : false
  }
]

Note the replace operation for the entire array, instead of the single modified element.

Whether you use the LCS based or simple diff, you can make it remember old values for remove and replace operations.

To that end, you just need to import diffson.jsonpatch.lcsdiff.remembering._ or diffson.jsonpatch.simplediff.remembering._ instead. The generated diff will add an old field to remove and replace operations in the patch, containing the previous version of the field in original object. Taking the first example with the new import, we have similar code.

import diffson._
import diffson.lcs._
import diffson.circe._
import diffson.jsonpatch._
import diffson.jsonpatch.lcsdiff.remembering._

import io.circe._
import io.circe.parser._

import cats._
import cats.implicits._

implicit val lcs = new Patience[Json]

val json1 = parse("""{
                    |  "a": 1,
                    |  "b": true,
                    |  "c": ["test", "plop"]
                    |}""".stripMargin)

val json2 = parse("""{
                    |  "a": 6,
                    |  "c": ["test2", "plop"],
                    |  "d": false
                    |}""".stripMargin)

val patch =
  for {
    json1 <- json1
    json2 <- json2
  } yield diff(json1, json2)

which results in a result with the old value remembered in the patch:

[
  {
    "op" : "replace",
    "path" : "/a",
    "value" : 6,
    "old" : 1
  },
  {
    "op" : "remove",
    "path" : "/b",
    "old" : true
  },
  {
    "op" : "replace",
    "path" : "/c/0",
    "value" : "test2",
    "old" : "test"
  },
  {
    "op" : "add",
    "path" : "/d",
    "value" : false
  }
]

Patches produced with this methods are still valid according to the RFC, as the new field must simply be ignored by implementations that are not aware of this encoding, so interoperability is not broken.

There are two different entities living in the diffson.jsonmergepatch package useful to work with Json merge patches:

• JsonMergePatch which allows to parse, create and apply Json merge patches as defined in RFC-7396,
• JsonMergeDiff which allows to compute the diff between two Json values and create Json merge patches.

Basically if someone wants to compute the diff between two Json objects, they can execute the following:

import diffson._
import diffson.circe._
import diffson.jsonmergepatch._

import io.circe.parser._
import io.circe.syntax._

val json1 = parse("""{
              |  "a": 1,
              |  "b": true,
              |  "c": "test"
              |}""".stripMargin)

val json2 = parse("""{
              |  "a": 6,
              |  "c": "test2",
              |  "d": false
              |}""".stripMargin)

val patch =
  for {
    json1 <- json1
    json2 <- json2
  } yield diff(json1, json2)

which will return the following Json Merge Patch:

{
  "a": 6,
  "b": null,
  "c": "test2",
  "d": false
}

You can then apply the patch to json1:

val json3 = patch(json1)

which will create the following Json:

{
  "d":false,
  "c":"test2",
  "a":6
}

which we can easily verify is the same as json2 modulo reordering of fields.