• This repository has been archived on 05/Dec/2022
  • Stars
    star
    117
  • Rank 291,070 (Top 6 %)
  • Language
    Clojure
  • License
    Apache License 2.0
  • Created about 9 years ago
  • Updated almost 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Utility library for writing microservices in Clojure, with support for Swagger and OAuth

friboo

Maven Central Build Status codecov

Friboo is a lightweight utility library for writing microservices in Clojure. It provides several components that you can use with Stuart Sierra's Component lifecycle framework.

Friboo encourages an "API First" approach based on the Swagger specification. As such, the REST API is defined as YAML.

Leiningen dependency

[org.zalando.stups/friboo 2.0.0]

Why Friboo?

  • Friboo allows you to first define your API in a portable, language-agnostic format, and then implement it (with the help of swagger1st).
  • It contains ready-made components/building blocks for your applications: An HTTP server, DB access layer, metrics registry, Hystrix dashboard (in case you have compliance requirements to follow), and more. See Components.
  • Pluggable support for all authentication mechanisms (basic, OAuth 2.0, API keys).
  • It contains the "glue code" for you, and there is already a recommended way of doing things.

Development Status

In our production we use an extension library that is based on Friboo: friboo-ext-zalando. See the list at the end of this page. However, there is always room for improvement, so we're very much open to contributions. For more details, see our contribution guidelines and check the Issues Tracker for ways you can help.

Getting Started

Requirements

Starting a New Project

To start a new project based on Friboo, use the Leiningen template:

$ lein new friboo com.example/friboo-is-awesome

This will generate a sample project containing some "foobar" logic that can serve as a starting point in your experiments.

A new directory with name friboo-is-awesome will be created in the current directory, containing the following files:

friboo-is-awesome
├── README.md
├── dev
│   └── user.clj
├── dev-config.edn
├── project.clj
├── resources
│   └── api
│       └── api.yaml
├── src
│   └── com
│       └── example
│           └── friboo_is_awesome
│               ├── api.clj
│               └── core.clj
└── test
    └── com
        └── example
            └── friboo_is_awesome
                ├── api_test.clj
                └── core_test.clj
  • README.md contains some pregenerated development tips for the new project.
  • dev/user.clj contains functions for Reloaded Workflow.
  • dev-config.edn contains environment variables that will be used during reloaded workflow (instead of putting them into profiles.clj).
  • project.clj contains the project definition with all dependencies and some additional plugins.
  • resources/api.yaml contains the Swagger API definition in .yaml format.
  • src directory contains these files:
    • core.clj is the system definition.
    • api.clj contains API endpoint handlers.
  • the test directory contains unit test examples using both clojure.test and Midje.

How Friboo works

There are two core parts in any Friboo application:

  • loading configuration by aggregating many sources
  • starting the system

Both these parts are taken care of in core.clj in run function. The name "run" is not fixed, it can be anything.

Let's put configuration aside for now. A minimal run function might look like this:

(require '[com.stuartsierra.component :as component]
         '[org.zalando.stups.friboo.system.http :as http]
         '[org.zalando.stups.friboo.system :as system])

(defn run []
  (let [system (component/map->SystemMap
                 {:http (http/make-http "api.yaml" {})})]
    (system/run {} system)))

Here we declare a system that has just one component created by make-http function. When started, this component will expose a RESTful API where requests are routed according to the Swagger definition in api.yaml, which is taken from the classpath (usually resources/api.yaml).

Then we call run from -main:

(defn -main [& args]
  (try
    (run)
    (catch Exception e
      (println "Could not start the system because of" (str e))
      (System/exit 1))))

run function does not block, it immediately returns the started system that can later be stopped (as reloaded workflow suggests).

This already works, but it's not too flexible.

Parsing configuration options

According to https://12factor.net/config, configuration should be provided via environment variables. However, with REPL-driven reloaded workflow you would have to restart the JVM every time you need to change a configuration value. That's less than perfect.

Friboo supports several sources of configuration:

  • environment variables: HTTP_PORT=8081
  • JVM properties: http.port=8081
  • development configuration from dev-config.edn: {:http-port 8081}
  • default configurations per component (hardcoded)

Another challenge is — how to give components only the configuration they need? What if more than one component would like to use PORT variable? Friboo solves this with namespacing of configuration parameters. Namespace in this case is just a known prefix: HTTP_, API_, ENGINE_ etc.

Configuration is in loaded inside run by load-config function before defining the system:

(defn run [args-config]
  (let [config (config/load-config
                 (merge default-http-config
                        args-config)
                 [:http])
        system (component/map->SystemMap
                 {:http (http/make-http "api.yaml" (:http config))})]
    (system/run config system)))

Here we make run accept a configuration map as an argument, it acts as an additional source of configuration (and it is used by Reloaded Workflow to inject reloadable configuration, see dev/user.clj).

load-config takes 2 arguments:

  • map of default configuration that looks like this:
{:http-port   8081
 :db-password "1q2w3e4r5t"
 :foo-bar     "foobar"
  • list of namespaces, which are known prefixes configuration variables that we expect:
[:http :db]

Configuration parameters' names are normalized in the following way (this is actually done by environ):

  • HTTP_PORT becomes :http-port
  • http.port becomes :http-port

load-config normalizes names in all configuration sources, merges them (real environment overrules the default config), filters the parameters by known prefixes and returns a nested map:

{:http {:port 8081}
 :db   {:password "1q2w3e4r5t"}}

Note that :foo-bar parameter did not make it into the output, because it does not start with :http- nor :db-.

After we have this configuration loaded, it's very straightforward to give each component its part:

{:http (http/make-http "api.yaml" (:http config))
 :db   (db/make-db (:db config))}

system/run also takes the entire configuration as the first argument and uses the :system part of it.

Components

HTTP Component

HTTP component starts a HTTP server and routes the requests based on the Swagger API definition. It lives in org.zalando.stups.friboo.system.http namespace.

It has an optional dependency :controller that is given to all API handlers as first argument. The use case is to make it contain some configuration and dependencies that the handlers should have access to.

paths:
  '/hello/{name}':
    get:
      operationId: "com.example.myapp.api/get-hello"
      responses: {}

Part of system map (we make :api component to be a simple map, it's not necessary for every component to implement com.stuartsierra.component/Lifecycle protocol):

:http      (component/using
             (http/make-http "api.yaml" (:http config))
             {:controller :api})
:api       {:configuration (:api config)}

{:controller :api} means that :api component will be available to :http under the name :controller, that's what it expects.

In com.example.myapp.api namespace:

(defn get-hello [{:keys [configuration]} {:keys [name]} request]
  (response {:message (str "Hello " name)}))

get-hello (and every other API handler function) is called with 3 arguments:

  • :controller (:api component in our example)
  • merged parameters map from path, query and body parameters
  • raw request map

Every handler function is expected to return a map representing a HTTP response:

{:body    {:message "Hello Michael"}
 :headers {}
 :status  200

In our example we use ring.util.response/response to create a HTTP 200.

Configuration Options

{:port        8081
 :cors-origin "*.zalando.de"}

DB Component

DB component encapsulates JDBC connection pool and provides Flyway to support schema migrations.

When the component starts, it will have additional :datasource key that contains an implementation of javax.sql.DataSource. You can use it as you like.

One of the examples is in friboo-ext-zalando:

$ lein new friboo-ext-zalando db-example

Take a look at the following files:

example
├── resources
│   └── db
│       ├── migration
│       │   └── V1__initial_schema.sql
│       └── queries.sql
└── src
    └── db_example
        ├── api.clj
        ├── core.clj
        └── sql.clj

Configuration Options

For available options please refer to org.zalando.stups.friboo.system.db/start-component

Metrics Component

The metrics component initializes a Dropwizard MetricsRegistry to measure frequency and performance of the Swagger API endpoints; see HTTP component.

Management HTTP component

This component starts another embedded Jetty at a different port (default 7979) and exposes endpoints used to monitor and manage the application:

  • /metrics: A JSON document containing all metrics, gathered by the metrics component
  • /hystrix.stream: The Hystrix stream (can be aggregated by Turbine)
  • /monitor/monitor.html: The Hystrix dashboard

Configuration Options

All Jetty configuration options.

Real-World Usage

There are multiple examples of real-world usages of Friboo, including among Zalando's STUPS components:

TODO HINT: set java.util.logging.manager= org.apache.logging.log4j.jul.LogManager to have proper JUL logging.

License

Copyright © 2016 Zalando SE

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

More Repositories

1

patroni

A template for PostgreSQL High Availability with Etcd, Consul, ZooKeeper, or Kubernetes
Python
6,058
star
2

postgres-operator

Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
Go
3,686
star
3

skipper

An HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress
Go
3,005
star
4

zalenium

A flexible and scalable container based Selenium Grid with video recording, live preview, basic auth & dashboard.
Java
2,380
star
5

restful-api-guidelines

A model set of guidelines for RESTful APIs and Events, created by Zalando
CSS
2,067
star
6

SwiftMonkey

A framework for doing randomised UI testing of iOS apps
Swift
1,945
star
7

tailor

A streaming layout service for front-end microservices
JavaScript
1,726
star
8

logbook

An extensible Java library for HTTP request and response logging
Java
1,684
star
9

tech-radar

Visualizing our technology choices
1,491
star
10

spilo

Highly available elephant herd: HA PostgreSQL cluster using Docker
Python
1,225
star
11

intellij-swagger

A plugin to help you easily edit Swagger and OpenAPI specification files inside IntelliJ IDEA
Java
1,160
star
12

problem-spring-web

A library for handling Problems in Spring Web MVC
Java
997
star
13

nakadi

A distributed event bus that implements a RESTful API abstraction on top of Kafka-like queues
Java
928
star
14

zally

A minimalistic, simple-to-use API linter
Kotlin
873
star
15

problem

A Java library that implements application/problem+json
Java
851
star
16

zalando-howto-open-source

Open Source guidance from Zalando, Europe's largest online fashion platform
799
star
17

go-keyring

Cross-platform keyring interface for Go
Go
689
star
18

gin-oauth2

Middleware for Gin Framework users who also want to use OAuth2
Go
556
star
19

zappr

An agent that enforces guidelines for your GitHub repositories
JavaScript
543
star
20

pg_view

Get a detailed, real-time view of your PostgreSQL database and system metrics
Python
488
star
21

engineering-principles

Our guidelines for building new applications and managing legacy systems
363
star
22

gulp-check-unused-css

A build tool for checking your HTML templates for unused CSS classes
CSS
359
star
23

zmon

Real-time monitoring of critical metrics & KPIs via elegant dashboards, Grafana3 visualizations & more
Shell
355
star
24

expan

Open-source Python library for statistical analysis of randomised control trials (A/B tests)
Python
325
star
25

PGObserver

A battle-tested, flexible & comprehensive monitoring solution for your PostgreSQL databases
Python
315
star
26

riptide

Client-side response routing for Spring
Java
285
star
27

jackson-datatype-money

Extension module to properly support datatypes of javax.money
Java
240
star
28

grafter

Grafter is a library to configure and wire Scala applications
Scala
240
star
29

opentracing-toolbox

Best-of-breed OpenTracing utilities, instrumentations and extensions
Java
178
star
30

elm-street-404

A fun WebGL game built with Elm
Elm
176
star
31

tokens

Java library for conveniently verifying and storing OAuth 2.0 service access tokens
Java
169
star
32

innkeeper

Simple route management API for Skipper
Scala
166
star
33

public-presentations

List of public talks by Zalando Tech: meetup presentations, recorded conference talks, slides
165
star
34

python-nsenter

Enter kernel namespaces from Python
Python
139
star
35

dress-code

The official style guide and framework for all Zalando Brand Solutions products
CSS
129
star
36

faux-pas

A library that simplifies error handling for Functional Programming in Java
Java
128
star
37

beard

A lightweight, logicless templating engine, written in Scala and inspired by Mustache
Scala
121
star
38

spring-cloud-config-aws-kms

Spring Cloud Config add-on that provides encryption via AWS KMS
Java
99
star
39

zalando.github.io

Open Source Documentation and guidelines for Zalando developers
HTML
80
star
40

failsafe-actuator

Endpoint library for the failsafe framework
Java
53
star
41

package-build

A toolset for building system packages using Docker and fpm-cookery
Ruby
35
star
42

ghe-backup

Github Enterprise backup at ZalandoTech (Kubernetes, AWS, Docker)
Shell
30
star
43

rds-health

discover anomalies, performance issues and optimization within AWS RDS
Go
18
star
44

backstage-plugin-api-linter

API Linter is a quality assurance tool that checks the compliance of API's specifications to Zalando's API rules.
TypeScript
12
star
45

.github

Standard github health files
1
star