Clip

Table of Contents

Background
Usage
How to

Clip is an inversion of control API which minimizes impact on your code. It is an alternative to integrant, mount or component.

Background

Project Status

Alpha. The core API is unlikely to change significantly, but there may be some small changes to the system-config format. Design feedback is still welcome, there is still space for the rationale to change/expand. While bugs are avoided, the immaturity of the project makes them more likely. Please provide design feedback and report bugs on the issue tracker.

Rationale

System as data

Systems may fail during startup, which can leave you in a state where a port is in use, but you have no handle to that partially-started system in order to close it.

(let [db (get-db)]
      http (start-http-server db)
      ;; What happens if make-queue-listener throws an exception?
      queue-listener (make-queue-listener)]
  …)

This can lead to re-starting the REPL, which is a major interruption to flow. Clip provides a data model for your system, and will rethrow exceptions with the partially-started components attached in order to allow automatic recovery in the REPL or during tests.

You may choose to make your system very granular. For example, you may choose to provide each web request handler with it’s dependencies directly, rather than passing them via the router. In these cases, you will find your system quickly grows large, having your system as data reduces the effort to maintain & understand the relationships between components.

EDN is a natural way to express data in Clojure. A data-driven system should naturally work with EDN without any magic.

Boilerplate

Existing dependency injection libraries require boilerplate for defining your components. This either comes in the form of multi-methods which must have their namespaces required for side-effects, or the creation of records in order to implement a protocol. These extension mechanisms are used to extend existing var calls in libraries. Instead of doing that, we can take a reference to a function (e.g. ring.adapter.jetty/run-jetty) and call it directly. This means that effort to wrap an existing library is minimal, you simply have to specify what to call. This also removes the problem of inventing new strategies for tying a keyword back to a namespace as Integrant has to, by directly using the fully qualified symbol which already includes the namespace as required. In addition, normal functions support doc-strings, making for easy documentation. Finally, this use of vars means that your library is not coupled to Clip in any way, yet it’s easy to use directly from Clip.

Transitions

Side-effects are part of a startup process. The most common example is seeding a database. Before other components can function (such as a health check) the migration must have run. Existing approaches require us to either taint our component’s start with additional side-effect code (complecting connection with migration) or to create side-effecting components which others must then depend upon.

Clip by default, provides a :pre and :post phase for start-up, enabling you to run setup before/after starting the component. For example, you may need to call datomic.api/create-database before connecting to it, and you may want to call my.app/seed-db after starting it, but before passing it around.

Clip is also simple, it separates out running many actions on your system. This allows you to define custom phases for components, if you need them.

Async

Async is a significant part of systems when doing ClojureScript applications. Something as simple as reading data from disk or fetching the user from a remote endpoint will cause your entire system to be aware of the callback.

Interceptors have shown that async can be a layered-on concern, rather than being intrinsically present in all consumption of the result. Clip provides multiple "executors" for running actions against your system, providing out of the box support for sync (no async), promesa, and manifold. If you need an additional executor, they are fairly simple to write.

Obvious

Obvious connections between actions are easier to understand than obscure ones. Use of inheritance for references or relying on implicit dependencies increases the obscurity of your API. Component and Integrant allow you to spread out your references through the use of using and prep-key. Instead Clip encourages you to make references live in the system, making it always obvious how components connect together.

Comparison with other libraries

Name	Need/has wrappers for libs	Extension mechanism	Suspension	System location	Multiple parallel systems?	Transparent async between components?
Component	Yes	Protocol	Via library	Map	Yes	No
Integrant	Yes	Multi-method	Yes	Map	Yes	No
Clip	No	Code/code as data	Coming	Map	Yes	Yes
Mount	No	Code	No	Namespace	No	No

Example Applications

Want to add one? Open an issue or pull request.

Usage

Defining a system configuration

You define a system configuration with data. A system configuration contains a :components key and an optional :executor key. :components is a map from anything to a map of properties about that component. Note the use of ` in the example below, this is to prevent execution, you might find it easier to use Use with EDN to define your system configuration.

(def system-config
  {:components
   {:db {:pre-start `(d/create-database "datomic:mem://newdb") ;; (1)
         :start `(d/connect "datomic:mem://newdb") ;; (2)
         :post-start `seed-conn} ;; (3)
    :handler {:start `(find-seed-resource (clip/ref :db))} ;; (4)
    :http {:start `(yada/listener (clip/ref :handler))
           :stop '((:close this)) ;; (5)
           :resolve :server} ;; (6)
    :foo {:start '(clip/ref :http)}}})

:pre-start will be run before :start for your component. Here we use it to run the required create-database in datomic.
:start is run and returns the value that other components will refer to.
:post-start is run before passing the component to any other components. Here, we use it to seed the connection. Because we provided a symbol, it will be resolved and called like so (seed-conn conn) where conn is the started component.
Here we use (clip/ref) to refer to the :db component. This will be provided positionally to the function.
:stop has access to the variable this to refer to the started component.
You can control how a component is referenced by other components. Here the :server key will be passed to other components referencing it (e.g. :foo).

`:components` reference

Out of the box, there are a handful of keys supported for a component. In the future this may be more extensible (please open an issue if you have a use-case!).

Many of the values of these keys take code as data. This means that if you were to create them programmatically you have to create them using either list or quotes. Supporting code as data means that EDN-based systems can be defined, but also that there’s special execution rules.

Code as data means that you don’t need to use actual function references, you can use symbols and these will be required & resolved by Clip. Requiring and resolving isn’t supported in ClojureScript, see workaround in ClojureScript.

Code is either executed with an "implicit target" or not. An example of an implicit target is the started instance. If an implicit target is available, you can provide a symbol or function without a list and it will be called with an argument which is the implicit target. If the function to call has multiple arguments, then you can use this to change where the implicit target will be placed.

Key	Implicit Target	Description
`:pre-start`	No	Run before starting the component
`:start`	No	Run to start the component, this will be what ends up in the system
`:post-start`	Started instance	Run with the started component, a useful place to perform migrations or seeding
`:stop`	Started instance (to stop)	Run with the started component, should be used to shut down the component. Optional to add. If not specified and value is AutoCloseable, then .close will be run on it
`:resolve`	Started instance	Run with the started component used by other components to get the value for this component when using `(clip/ref)`

Supported values for code as data with implicit target:

Type	Description
Symbol	Resolved to function then called with target
Function	Resolved to function then called with target
Keyword	Used to get the key out of the target

Supported values for code as data without implicit target:

Type	Description	Example(s)
Symbol	Resolved to function and called with no arguments	`'myapp.core/start-web-server`
Function	Called with no arguments	`myapp.core/start-web-server`
Vector	Recursed into, with arguments resolved	`[(clip/ref :foo)]`
List	Called as if code	`(list 'myapp.core/start-web-server {:port 8000})` `'(myapp.core/start-web-server {:port 8000})`
:else	Returned unchanged

Async Components

In Clip, async is achieved by using alternative executors. Out of the box support is provided for promesa and manifold. Open an issue if you’d like to see support for another popular library.

Executors are specified on your system and must be a function. juxt.clip.edn/load will convert your :executor from a symbol to a function.

Example 1. Promesa Async Example

{:executor juxt.clip.promesa/exec
 :components
 {:a {:start `(promesa.core/resolved 10)}
  :b {:start `(inc (clip/ref :a))}}}

Note that :b does not need to be aware that :a returns an async value. It will be called at the appropriate time with the resolved value.

Example 2. Manifold Example

(require '[manifold.deferred :as d])

{:executor juxt.clip.manifold/exec
 :components
 {:a {:start `(d/chain 10)}
  :b {:start `(inc (clip/ref :a))}}}

Reloaded Workflow

Clip provides a namespace for easily setting up a reloaded workflow. You will need to add a dependency on tools.namespace to your project.

You should call set-init! with a function which will return your system-config. Usually you will have such a function defined in another namespace that takes a "profile" or "config" in order to be parameterized to development or production. If you are using Use with EDN to load your system, ensure your function calls clip.edn/load.

(ns dev
  (:require
    [app.system]
    [juxt.clip.repl :refer [start stop reset set-init! system]]))

(set-init! #(app.system/system-config :dev))

Async executors

If you’re using an async executor with the repl namespace, you may need to make it sync. Out of the box, the repl namespace will do it’s best to work with anything supported by deref. If you need to override the deref that the repl namespace uses, you can supply a symbol or function in the key :juxt.clip.repl/deref. It should take one argument: the system to deref.

You won’t need to tweak this for Promesa or Manifold.

(set-init!
  (constantly {:executor juxt.clip.awkward-async/exec
               :juxt.clip.repl/deref some.ns.awkard-async/deref
               …}))

ClojureScript

ClojureScript has limitations with taking code-forms as data. This will continue to be an active research topic, but until resolved the usage is still reasonably concise. You must use list to create a list-form.

Example 3. ClojureScript Usage

(ns frontend.core
  (:require [juxt.clip.core :as clip]))

(def system-config
  {:components
    {:foo {:start 200}
     :bar {:start (list inc (clip/ref :foo))}}})

Caution

The following macro is experimental, feedback on use is welcome. However, of the following experimental options it is currently the forerunner.

There is a macro called with-deps that allows you to write a code-form and bind the dependencies required. This is useful when using Clip from a code (rather than data) context. It’s also particularly useful in ClojureScript where symbols cannot be resolved back to functions.

with-deps takes bindings and a body, much like fn. The first of the bindings must be to the deps you want. You must use associative destructuring.

Example 4. with-deps Usage

(ns frontend.core
  (:require [juxt.clip.core :as clip :include-macros true]))

(def system-config
  {:components
    {:foo {:start 200}
     :bar {:start 300}
     :baz {:start (clip/with-deps [{:keys [foo bar]}]
                    (+ foo bar))}}})

Caution

The following macro is extremely experimental, feed-back on use is welcome.

You can also bring in the deval macro. This macro will convert lists of code it finds into non-evaluated lists, which can later be interpreted by Clip.

Example 5. Deval Usage

(ns frontend.core
  (require '[juxt.clip.core :as clip :include-macros true]))

(def system-config
  (clip/deval
    {:components
      {:foo {:start 200}
       :bar {:start (inc (clip/ref :foo))}}}))

How to

Use with EDN

Clip works very well with EDN. First, you must call juxt.clip.edn/load on your EDN system to produce a system Clip can understand. This will convert your EDN structure into a system compatible with juxt.clip.core functions by, e.g. converting :executor from a symbol to a function.

It was designed to be used with a library such as aero in order to make dev/prod changes to your system. Here’s a minimal example system-config configured with aero:

config.edn

{:system-config
 {:components
  {:db {:start (hikari-cp.core/make-datasource
                 #profile
                 {:dev
                  {:adapter "h2"
                   :url "jdbc:h2:~/test"}
                  :prod
                  {:jdbc-url "jdbc:sqlite:db/database.db"}})
        :stop (hikari-cp.core/close-datasource this)}}}}

Use from -main

When starting your application from -main there’s a few considerations:

Blocking forever (Use @(promise) to do this)
Storing the system for REPLing in later
Whether to shutdown the system or not

Simplest version, blocking forever

(ns myapp.main
  (:require
    [myapp.system]
    [juxt.clip.core :as clip]))

(defn -main
  [& _]
  (clip/start (myapp.system/system-config :prod))
  @(promise))

Storing system for later

(ns myapp.main
  (:require
    [myapp.system]
    [juxt.clip.core :as clip]))

(def system nil)

(defn -main
  [& _]
  (let [system (clip/start (myapp.system/system-config :prod))]
    (alter-var-root #'system (constantly system)))
  @(promise))

Stopping system on shutdown

(ns myapp.main
  (:require
    [myapp.system]
    [juxt.clip.core :as clip]))

(def system nil)

(defn -main
  [& _]
  (let [system-config (myapp.system/system-config :prod)
        system (clip/start system-config)]
    (alter-var-root #'system (constantly system))
    (.addShutdownHook
     (Runtime/getRuntime)
     (Thread. #(clip/stop system-config system))))
  @(promise))

AOT compile namespaces

You need to involve Clip in the AOT process in order to require any namespaces required by your system.

(binding [*compile-files* true] ;; (1)
  (-> ((requiring-resolve 'myapp.system/get-system)) ;; (2)
      clip.edn/load ;; (3)
      clip/require ;; (4)
      ))

Tell Clojure that when we require code, we’d like to AOT it.
We must require myapp.system inside of the binding otherwise it won’t be AOT’d.
Optional, only do this if your system is in EDN.
This is what requires the namespaces used by your system.

Test systems

Clip provides 2 useful mechanisms for testing:

select to get a subset of a system-config with only the specified components. start also takes a list of components to start as a convenience.
with-system macro which can start a system, and tries very hard to close it.

For example, you may wish to test a single ring handler, and pass it any required arguments to start:

(ns my.app.handler-test
  (:require
    [clojure.test :refer [deftest is]]
    [juxt.clip.core :as clip]
    [my.app.system :refer [get-system]]
    [ring.mock.request :as mock]))

(deftest handler-test
  (clip/with-system [system (clip/select (get-system) [:my-handler])]
    (try
      (is (= {} (:my-handler system)))
      (finally
        (clip/stop system)))))

Custom reloaded workflow

Alternatively you can roll your own Reloaded workflow quite easily, but you will miss out on convenient features in the built-in one like auto-cleanup.

(ns dev
  (:require [juxt.clip.core :as clip]
            [clojure.tools.namespace.repl :refer [refresh]]))

(def system-config {:a {:start 1}})
(def system nil)

(defn go []
  (alter-var-root #'system (constantly (clip/start system-config))))

(defn stop []
  (alter-var-root #'system
    (fn [s] (when s (clip/stop system-config s)))))

(defn reset []
  (stop)
  (refresh :after 'dev/go))

juxt/clip

juxt

Reviews

Repository Details

Clip

Background

Project Status

Rationale

System as data

Boilerplate

Transitions

Async

Obvious

Comparison with other libraries

Example Applications

Usage

Defining a system configuration

`:components` reference

Async Components

Reloaded Workflow

Async executors

ClojureScript

How to

Use with EDN

Use from -main

AOT compile namespaces

Test systems

Custom reloaded workflow

More Repositories

juxt/clip

juxt

Reviews

Repository Details

Clip

Background

Project Status

Rationale

System as data

Boilerplate

Transitions

Async

Obvious

Comparison with other libraries

Example Applications

Usage

Defining a system configuration

:components reference

Async Components

Reloaded Workflow

Async executors

ClojureScript

How to

Use with EDN

Use from -main

AOT compile namespaces

Test systems

Custom reloaded workflow

More Repositories

`:components` reference