• Stars
    star
    26
  • Rank 930,752 (Top 19 %)
  • Language
    Clojure
  • License
    Other
  • Created over 3 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Leiningen plugin for consuming and compiling protobuf schemas

lein-protodeps

Clojars Project

A Leiningen plugin to automate compilation of Protobuf and gRPC stubs.

This plugin allows to define your project's protobuf dependencies, by stating their source location. These locations can currently point to a git repository, or a local filesystem.

When run, the plugin will resolve these locations and compile the desired .proto files (and their dependencies) using the correct protoc compiler and gRPC plugin, according to the versions specified in your project's configuration. The plugin will automatically download the correct versions if they are not already installed.

Table of Contents

Usage

Put [com.appsflyer/lein-protodeps "1.0.5"] into the :plugins vector of your project.clj.

Once installed, run lein protodeps generate to run the plugin.

The plugin requires some configuration in your project.clj in order to run.

An example configuration:

(def proto-version "3.12.4") ;; protobuf version -- should be used when declaring protobuf dependencies
(def grpc-version "1.30.2")  ;; gRPC version -- should be used when declaring gRPC dependencies

(defproject my-cool-project "0.1.0"
  ...
  ...
  ;; plugin configuration
  :lein-protodeps {:output-path   "src/java/generated" ;; where to place the generated files? Should reside within your `java-source-paths`
                   :proto-version ~proto-version
                   :grpc-version  ~grpc-version
                   :compile-grpc? true ;; whether to activate the gRPC plugin during the stub generation process
                   ;; Repositories configuration. Each entry in this map is an entry mapping a logical repository name
                   ;; to its configuration.
                   :repos         {:af-schemas {:repo-type :git ;; a git repo
                                                :config   {:clone-url   "git@localhost:test/repo.git" ;; url to clone from
                                                           ;; rev - can point to a commit hash, tag name or branch name. The repo will be cloned
                                                           ;; to this version of itself. If unspecified, will point to origin's HEAD (i.e, master).
                                                           :rev         "mybranch"}
                                                ;; a vector of proto-paths relative to the directory root. May use an empty string if the root
                                                ;; level is a proto path in itself.
                                                :proto-paths ["products"]
                                                ;; a vector of dependencies which control what stubs to compile. Each dependency vector
                                                ;; contains a directory under one of the proto paths. All files in this directory and their
                                                ;; dependencies will be compiled.
                                                :dependencies [products/events
                                                               products/adrevenue]}

                                   :some-other-schemas {:repo-type    :filesystem ;; read files directly from filesystem instead of git.
                                                        :config       {:path "../schemas"} ;; path, either relative or absolute
                                                        :proto-paths  ["products"]
                                                        :dependencies [products/foo
                                                                       products/bar]}}}

Cross-repository compilation

lein-protodeps also supports cross-repo compilation, for example when a .proto file dependency in one repo imports a file residing in a different repo.

To enable cross-repo compilation, simply add both repos to the :repos config map.

Git HTTP authentication

To use HTTP authentication using username and password, provide them in the clone url: "https://<myuser>:<mypass>@github.com/whatever/cool_repo.git"

It is recommended to use environment variables rather than hardcoding their values in plaintext. Environment variables are accessible via the ${:env/<var_name>} interpolation syntax, which allows us to write the former as: "https://${:env/GIT_USERNAME}:${:env/GIT_PASSWORD}@github.com/whatever/cool_repo.git".

protoc and gRPC binaries retrieval

The plugin will download the protoc and gRPC plugin binaries according to the versions set in :proto-version and :grpc-version, respectively, and install them under ~/.lein-protodeps/.

By default, the plugin will use https://github.com/protocolbuffers/protobuf/releases/download/ for protoc and https://repo1.maven.org/maven2/io/grpc/protoc-gen-grpc-java/ for gRPC. However, it is possible to override these to other endpoints by setting :protoc-zip-url-template and :grpc-exe-url-template options in the plugin configuration.

The values of these options are URL templates that will be interpolated at runtime with the following variables to produce download URLs:

  • :os-name host OS (i.e, linux, osx)
  • :os-arch host architecture (i.e, x86_64, aarch64)
  • :semver version string as defined in :protoc-version or :grpc-version
  • :major major part of :semver
  • :minor minor part of :semver
  • :patch patch part of :semver

For example, to override the gRPC URL you may set :grpc-exe-url-template to https://some-other-place.com/artifacts/grpc/${:semver}/protoc-gen-grpc-java-${:semver}-${:os-name}-${:os-arch}. Note that currently this feature does not support any method of authentication, in case your endpoints require it.

Configuration Reference

Plugin Configuration Options

Key Type Req? Notes
:output-path string required Path to which to compile stubs. This usually needs to be under :java-source-paths
:proto-version string required Version of protoc to use
:grpc-version string optional Version of gRPC plugin to use. Required if :compile-grpc? is true
:compile-grpc? boolean optional Whether to compile gRPC stubs. Defaults to false
:protoc-zip-url-template string optional URL template from which to retrieve protoc's zip release (if needed). See also protoc and gRPC binaries retrieval
:grpc-exe-url-template string optional URL template from which to retrieve gRPC's executable release (if needed). See also protoc and gRPC binaries retrieval

Repo Options

Key Type Req? Notes
<:repo-name>.:repo-type :git :filesystem required
<:repo-name>.:config.:clone-url string required (for git repos) Either SSH or HTTP endpoints are supported, see also Git HTTP authentication
<:repo-name>.:config.:rev string optional (for git repos) commit hash/tag name/branch name. Not specifying a rev will default to cloning the main branch (it is generally encouraged to use a fixed version)
<:repo-name>.:config.:path string required (for filesystem repos) path to directory containing files (absolute or relative to project directory)

Proto options

Key Type Req? Notes
<:repo-name>.:proto-paths vector of strings required Relative proto paths to the directory root (where protoc will search for imports, see protoc --help for more information)
<:repo-name>.:dependencies vector of symbols optional List of paths which contain files to compile to stubs. Each of these paths needs to be prefixed with one of the proto paths defined under :proto-paths, see example above. If unspecified or empty, nothing in this repo will be compiled, but it may still be used for finding imports required by other repos. See also cross-repository compilation

More Repositories

1

go-sundheit

A library built to provide support for defining service health for golang services. It allows you to register async health checks for your dependencies and the service itself, provides a health endpoint that exposes their status, and health metrics.
Go
549
star
2

donkey

Modern Clojure HTTP server and client built for ease of use and performance
Java
291
star
3

ketu

A clojure kafka client with core.async integration.
Clojure
121
star
4

pronto

Clojure support for protocol buffers
Clojure
107
star
5

terra-crust

Terra Crust was created to allow Platform teams to expose Terraform as the main API to developers
Go
65
star
6

mate-clj

Clojure library for debugging core functions
Clojure
58
star
7

aerospike-clj

Clojure client for the Aerospike database.
Clojure
29
star
8

kafka-mirror-tester

A tool to test the performance and correctness of kafka mirroring.
Go
25
star
9

wrk3

A golang generic benchmarking tool based mostly on Gil Tene's wrk2, only rewritten in go, and extended to allow running arbitrary protocols
Go
17
star
10

go-consul-resolver

A library of composable layers that is designed to provide client-side load balancing for (but not limited to) HTTP client-server communication, using Consul as the service discovery layer
Go
12
star
11

local-pvc-releaser

A Kubernetes controller designed to oversee Persistent Volume Claims (PVCs) associated with local storage on worker nodes. Its purpose is to enhance resilience and facilitate automatic recovery in the event of node termination.
Go
11
star
12

elb-log-replay

Replays an ELB log to a provided host
Go
7
star
13

unleash-client-clojure

Unleash Client SDK for Clojure
Clojure
7
star
14

srealip

Go package for securely extracting HTTP client's real public IP
Go
6
star
15

engineering-org-resources

List of talks, abstracts, speaker profiles, and resources from the AppsFlyer Engineering Bakery
5
star
16

go-sundheit-opentelemetry

Open telemetry metrics support for go-sundheit
3
star
17

AppsFlyerUnityPlugin

AppsFlyer Unity Plugin (iOS)
Perl
2
star
18

DevOpsDaysTLV2020Challenge

Programming challenge for DevOpsDays Tel Aviv 2020
2
star
19

appsflyer.github.io

Quick access to AppsFlyer's public & open source repos
JavaScript
1
star
20

af-sast-clojure

Clojure
1
star