yakivy/poppet

Poppet is a minimal, type-safe RPC Scala library.

Essential differences from autowire:

• has no explicit macro application .call, result of a consumer is an instance of original trait
• has no restricted HKT Future, you can specify any monad (has cats.Monad typeclass) as HKT for the provider/consumer
• has no forced codec dependencies uPickle, you can choose from the list of predefined codecs or easily implement your own codec
• has robust failure handling mechanism
• supports Scala 3 (however method/class generation with macros is still an experimental feature)

1. Quick start
2. Customizations
   1. Logging
   2. Failure handling
3. Manual calls
4. Limitations
5. API versioning
6. Examples
7. Changelog

Put cats and poppet dependencies in the build file, let's assume you are using SBT:

val version = new {
    val cats = "2.10.0"
    val circe = "0.14.6"
    val poppet = "0.3.5"
}

libraryDependencies ++= Seq(
    "org.typelevel" %% "cats-core" % version.cats,

    //to use circe
    "io.circe" %% "circe-core" % version.circe,
    "com.github.yakivy" %% "poppet-circe" % version.poppet,

    //"com.github.yakivy" %% "poppet-upickle" % version.poppet, //to use upickle
    //"com.github.yakivy" %% "poppet-play-json" % version.poppet, //to use play json
    //"com.github.yakivy" %% "poppet-jackson" % version.poppet, //to use jackson
    //"com.github.yakivy" %% "poppet-core" % version.poppet, //to build custom codec
)

Define service trait and share it between provider and consumer apps:

case class User(email: String, firstName: String)
trait UserService {
    def findById(id: String): Future[User]
}

Implement service trait with actual logic:

class UserInternalService extends UserService {
    override def findById(id: String): Future[User] = {
        //emulation of business logic
        if (id == "1") Future.successful(User(id, "Antony"))
        else Future.failed(new IllegalArgumentException("User is not found"))
    }
}

Create service provider (can be created once and shared for all incoming calls), keep in mind that only abstract methods of the service type will be exposed, so you need to explicitly specify a trait type:

import cats.implicits._
import io.circe._
import io.circe.generic.auto._
import poppet.codec.circe.all._
import poppet.provider.all._

//replace with serious pool
implicit val ec: ExecutionContext = ExecutionContext.global

val provider = Provider[Future, Json]()
    .service[UserService](new UserInternalService)
    //.service[OtherService](otherService)

Create service consumer (can be created once and shared everywhere):

import cats.implicits._
import io.circe._
import io.circe.generic.auto._
import poppet.codec.circe.all._
import poppet.consumer.all._
import scala.concurrent.ExecutionContext

//replace with serious pool
implicit val ec: ExecutionContext = ExecutionContext.global
//replace with actual transport call
val transport: Transport[Future, Json] = request => provider(request)

val userService = Consumer[Future, Json](transport)
    .service[UserService]

Enjoy 👌

userService.findById("1")

The library is build on following abstractions:

• [F[_]] - is your service HKT, can be any monad (has cats.Monad typeclass);
• [I] - is an intermediate data type that your coding framework works with, can be any serialization format, but it would be easier to choose from existed codec modules as they come with a bunch of predefined codecs;
• poppet.consumer.Transport - used to transfer the data between consumer and provider apps, technically it is just a function from [I] to [F[I]], so you can use anything as long as it can receive/pass the chosen data type;
• poppet.Codec - used to convert [I] to domain models and vice versa. Poppet comes with a bunch of modules, where you will hopefully find a favourite codec. If it is not there, you can always try to write your own by providing 2 basic implicits like here;
• poppet.CodecK - used to convert method return HKT to [F] and vice versa. It's needed only if return HKT differs from your service HKT, compilation errors will hint you what codecs are absent;
• poppet.FailureHandler[F[_]] - used to handle internal failures, more info you can find here;
• poppet.Peek[F[_], I] - used to decorate a function from Request[I] to F[Response[I]]. Good fit for logging, more info you can find here.

Both provider and consumer take Peek[F, I] as an argument, that allows to inject logging logic around the Request[I] => F[Response[I]] function. Let's define simple logging peek:

val peek: Peek[Id, Json] = f => request => {
    println("Request: " + request)
    val response = f(request)
    println("Response: " + response)
    response
}

All meaningful failures that can appear in the library are being transformed into poppet.Failure, after what, handled with poppet.FailureHandler. Failure handler is a simple polymorphic function from failure to lifted result:

trait FailureHandler[F[_]] {
    def apply[A](f: Failure): F[A]
}

by default, throwing failure handler is being used:

def throwing[F[_]]: FailureHandler[F] = new FailureHandler[F] {
    override def apply[A](f: Failure): F[A] = throw f
}

so if your don't want to deal with JVM exceptions, you can provide your own instance of failure handler. Let's assume you want to pack a failure with EitherT[Future, String, *] HKT, then failure handler can look like:

type SR[A] = EitherT[Future, String, A]
val SRFailureHandler = new FailureHandler[SR] {
    override def apply[A](f: Failure): SR[A] = EitherT.leftT(f.getMessage)
}

For more info you can check Http4s with Circe example project, it is built around EitherT[IO, String, *] HKT.

If your codec has a human-readable format (JSON for example), you can use a provider without consumer (mostly for debug purposes) by generating requests manually. Here is an example of curl call:

curl --location --request POST '${providerUrl}' \
--data-raw '{
    "service": "poppet.UserService", #full class name of the service
    "method": "findById", #method name
    "arguments": {
        "id": "1" #argument name: encoded value
    }
}'

You can generate consumer/provider almost from any Scala trait (or Java interface 😲). It can have non-abstract members, methods with default arguments, methods with multiple argument lists, varargs, etc... But there are several limitations:

• you cannot overload methods with the same argument names, because for the sake of simplicity argument names are being used as a part of the request, for more info check manual calls section:

//compiles
def apply(a: String): Boolean = ???
def apply(b: Int): Boolean = ???

// doesn't compile
def apply(a: String): Boolean = ???
def apply(a: Int): Boolean = ???

• trait/method type parameters should be fully qualified, because codecs are resolved at consumer/provider generation rather than at the method call:

//compiles
trait A[T] {
    def apply(t: T): Boolean
}

//doesn't compile
trait A {
    def apply[T](t: T): Boolean
}
trait A {
    type T
    def apply(t: T): Boolean
}

• trait should not have arguments

The goal of the library is to closely resemble typical Scala traits, so same binary compatibility approaches can also be applied for API versioning, for example:

• when you want to change method signature, add new method and deprecate old one, (important note: argument name is a part of signature in poppet, for more info check limitations section):

@deprecared def apply(a: String): Boolean = ???
def apply(b: Int): Boolean = ???

• if you are tolerant to binary incompatible changes, you can modify argument/return types without creating new method, but ensure that codecs are compatible:

def apply(a: String): Boolean = ???
//if Email is serialized as a String, method can be updated to
def apply(a: Email): Boolean = ???

• when you want to remove method, deprecate it and remove after all consumers are updated to the new version
• when you want to change service name, provide new service (you can extend it from the old one) and deprecate old one:

@deprecated trait A
trait B extends A

Provider[..., ...]()
    .service[A](bImpl)
    .service[B](bImpl)

• run desired example:
  • Http4s with Circe: https://github.com/yakivy/poppet/tree/master/example/http4s
    • run provider: ./mill example.http4s.provider.run
    • run consumer: ./mill example.http4s.consumer.run
  • Play Framework with Play Json: https://github.com/yakivy/poppet/tree/master/example/play
    • run provider: ./mill example.play.provider.run
    • run consumer: ./mill example.play.consumer.run
    • remove RUNNING_PID file manually if services are conflicting with each other
  • And even Spring Framework with Jackson 😲: https://github.com/yakivy/poppet/tree/master/example/spring
    • run provider: ./mill example.spring.provider.run
    • run consumer: ./mill example.spring.consumer.run
• put http://localhost:9002/api/user/1 in the address bar

• simplify transport and provider response, use Request => Response instead of I (remove Peek?)
• add action (including argument name) to codec
• throw an exception on duplicated service processor
• separate .service[S] and .service[G[_], S] to simplify codec resolution
• don't create ObjectMapper in the lib, use implicit one
• check that passed class is a trait and doesn't have arguments to prevent obscure error from compiler
• check that all abstract methods are public

• fix compilation errors for methods with varargs
• fix several compilation errors for Scala 3
• fix codec resolution for id (I => I) codecs
• reset DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES jackson object mapping property to default value to close DoS vulnerability
• add Scala 3 support

• fix compilation error message for ambiguous implicits
• fix processor compilation for complex types
• migrate to mill build tool
• add Scala JS and Scala Native support
• add more details to Can't find processor exception
• make FailureHandler explicit
• rename poppet.coder package to poppet.codec
• various refactorings and cleanups