• Stars
    star
    301
  • Rank 135,154 (Top 3 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created almost 8 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Wrap up your bash scripts into a modern CLI today. Graduate to a full-blown golang app tomorrow.

Variant

image

Build modern command line applications in YAML and any scripting language of your choice, and eventually enhance it with golang

CircleCI

Integrations: GitHub Actions

$ cat <<EOF | variant init mycmd
tasks:
  hello:
   parameters:
   - name: target
   script: |
     echo Hello {{ get "target" }}!
EOF
#!/usr/bin/env variant

tasks:
  hello:
   parameters:
   - name: target
   script: |
     echo Hello {{ get "target" }}!
$ ./mycmd hello --target variant
mycmd ≫ starting task hello
Hello variant!

You can then build a single go executable of your command and finally enhance it with golang code.

Rationale

Automating DevOps workflows is difficult because it often involve multiple executables like shell/ruby/perl/etc scripts and commands.

Because those executables vary in:

  • Their quality; from scripts written in a day, intended as a one-off command, but which wind up sticking around for months or even years, to serious commands which are well-designed and written in richer programming languages with adequate tests.
  • Their interface; some passing parameters via environment variables, others having application specific command-line flags, or configuration files.

Writing a single tool which

  • wires up all the executables
  • re-implements all the things currently done in various tools

is time-consuming.

Install

To install the latest version, run:

curl -sL https://raw.githubusercontent.com/variantdev/get/master/get | INSTALL_TO=/usr/local/bin sh

To install a specific version, run with the VERSION shell variable:

curl -sL https://raw.githubusercontent.com/variantdev/get/master/get | INSTALL_TO=/usr/local/bin VERSION=v0.35.1 sh

Getting Started

Create a yaml file named myfirstcmd containing:

#!/usr/bin/env variant

tasks:
  bar:
    script: |
      echo "dude"
  foo:
    parameters:
    - name: bar
      type: string
      description: "the bar"
    - name: environment
      type: string
      default: "heaven"
    script: |
      echo "Hello {{ get "bar" }} you are in the {{ get "environment" }}"

Now run your command by:

$ chmod +x ./myfirstcmd
$ ./myfirstcmd
Usage:
  myfirstcmd [command]

Available Commands:
  bar
  env         Print currently selected environment
  foo
  help        Help about any command
  ls          test
  version     Print the version number of this command

Flags:
  -c, --config-file string   Path to config file
  -h, --help                 help for myfirstcmd
      --logtostderr          write log messages to stderr (default true)
  -o, --output string        Output format. One of: json|text|bunyan (default "text")
  -v, --verbose              verbose output

Use "myfirstcmd [command] --help" for more information about a command.

Each task in the myfirstcmd is given a sub-command. Run myfirstcmd foo to run the task named foo:

$ ./myfirstcmd foo
Hello dude you are in the heaven

Look at the substring dude contained in the output above. The value dude is coming from the the parameter bar of the task foo. As we didn't specify the value for the parameter, variant automatically runs the task bar to fulfill it.

To confirm that the task bar is emitting the value dude, try running it:

$ ./myfirstcmd bar
INFO[0000] ≫ sh -c echo "dude"
dude

To specify the value, use the corresponding command-line flag automatically created and named after the parameter bar:

$ ./myfirstcmd foo --bar=folk
Hello folk you are in the heaven

Alternatively, you can source the value from a YAML file.

Create myfirstcmd.yaml containing:

foo:
  bar: variant

Now your task sources variant as the value for the parameter:

$ ./myfirstcmd foo
Hello variant you are in the heaven

Releasing a variant-made command

While Variant makes it easy for you to develop a modern CLI without recompiling, it is able to produce a single executable binary of your command.

Example: examples/hello

Write a small shell script that wraps your variant command into a simple golang program:

$ cat <<EOF > main.go
package main
import "github.com/mumoshu/variant/cmd"
func main() {
    cmd.YAML(\`
$(cat yourcmd)
\`)
}
EOF

$ cat <<EOF > Gopkg.toml
[[constraint]]
  name = "github.com/mumoshu/variant"
  version = "v0.24.0"
EOF

And then build with the standard golang toolchain:

$ dep ensure
$ go build -o dist/yourcmd .
$ ./mycli --target variant
Hello variant!

It is recommended to version-control the produced Gopkg.toml and Gopkg.lock because it is just more straight-forward than managing embedded version of em in the shell snippet.

It is NOT recommended to version-control main.go. One of the benefits of Variant is you don't need to recompile while developing. So it is your Variant command written in YAML that should be version-controlled, rather than main.go which is necessary only while releasing.

How it works

Variant is a framework to build a CLI application which becomes the single entry point to your DevOps workflows.

It consists of:

  • YAML-based DSL
    • to define a CLI app's commands, inputs
    • which allows splitting commands into separate source files, decoupled from each others
  • Ways to configure your apps written using Variant via:
    • defaults
    • environment variables
    • command-line parameters
    • application specific configuration files
    • environment specific configuration files
  • DI container
    • to implicitly inject required inputs to a commands from configuration files or outputs from another commands
    • to explicit inject inputs to commands and its dependencies via command-line parameters

Features

  • Default Command
  • Task grouping
  • Dependency injection

Default Command

The top-level script is executed whenever there's no sub-task that matches the provided command-line arguments.

In the below example, ./mycmd bar runs the task bar, while ./mycmd foo bar fails with an "unknown command" error:

tasks:
  bar:
    script: |
      echo bar

While in the next example, ./mycmd foo bar runs the root task(=the top-level script):

script: |
  echo {{ index .args 0 }}


tasks:
  bar:
    script: |
       echo bar

Dependency injection

An input named myinput for the task mytask can be one of follows, in order of precedence:

  • Value of the command-line option --myinput
  • Value of the configuration variable mytask.myinput
    • from the environment specific config file: config/environments/<environment name>.yaml
    • from the common config file: <command name>.yaml(normally var.yaml)
  • Output of the task myinput

Environments

You can switch environment (or context) in which a task is executed by running var env set <env name>.

$ var env set dev
$ var test
#=> reads inputs from var.yaml + config/environments/dev.yaml

$ var env set prod
$ var test
#=> reads inputs from var.yaml + config/environments/prod.yaml

Environment Variables

variant takes a few envvars for configuration.

VARIANT_RUN: Additional command-line arguments to be added to the actual args. For instance, VARIANT_RUN="bar baz" variant foo --color=false is equivalent to variant foo --color=false bar baz.

VARIANT_RUN_TRIM_PREFIX: Prefix to be removed from the VARIANT_RUN. For intance, VARIANT_RUN="/myslashcmd --foo=bar" variant mycmd is equivalent to variant mycmd --foo=bar.

VARIANT_GITHUB_COMMENT(_ON_[SUCCESS|FAILURE]): (GitHub Actions v2 only) When this variables is set to a non-empty value, variant tries to obtain the "source" GitHub issue/pull request that triggered the run, and sends a issue/pr comment containing the result. Great for giving feedbacks to whom run the variant task from e.g. GitHub comment.

FAQ

Please see the collection of answered questions in our GitHub issues labeled "question".

Integrations and useful companion tools

  • Use liujianping/job for timeouts, retries, scheduled runs, etc.
  • Use davidovich/summon to bundle assets into your variant command by using the golang module system and gobin

Alternatives

Interesting Readings

Future Goals

  • Runners to run tasks in places other than the host running your Variant app
    • Docker
    • Kubernetes
    • etc
  • Tools/instructions to package your Variant app for easier distribution
    • Single docker image containing
      • all the scripts written directly in the yaml
      • maybe all the scripts referenced from scripts in the yaml
      • maybe all the commands run via the host runner
  • Integration with job queues
    • to ensure your tasks are run reliably, at-least-once, tolerating temporary failures

License

Apache License 2.0

Attribution

We use:

  • semtag for automated semver tagging. I greatly appreciate the author(pnikosis)'s effort on creating it and their kindness to share it!

More Repositories

1

kube-airflow

A docker image and kubernetes config files to run Airflow on Kubernetes
Python
649
star
2

aws-secret-operator

A Kubernetes operator that automatically creates and updates Kubernetes secrets according to what are stored in AWS Secrets Manager.
Go
309
star
3

terraform-provider-eksctl

Manage AWS EKS clusters using Terraform and eksctl
Go
229
star
4

helm-x

Treat any Kustomization or K8s manifests directory as a Helm chart
Go
169
star
5

play2-memcached

A memcached plugin for Play 2.x
Scala
162
star
6

variant2

Turn your bash scripts into a modern, single-executable CLI app today
Go
134
star
7

terraform-provider-helmfile

Deploy Helmfile releases from Terraform
Go
115
star
8

kube-ssm-agent

Secure, access-controlled, and audited terminal sessions to EKS nodes without SSH
Dockerfile
106
star
9

kube-spot-termination-notice-handler

Moved to https://github.com/kube-aws/kube-spot-termination-notice-handler
Shell
97
star
10

concourse-aws

Auto-scaling Concourse CI v2.2.1 on AWS with Terraform (since 2016.04.14)
Go
72
star
11

play2-typescript

TypeScript assets handling for Play 2.0. Compiles .ts files under the /assets dir along with the project.
Scala
72
star
12

crossover

Minimal sufficient Envoy xDS for Kubernetes that knows https://smi-spec.io/
Go
71
star
13

helmfile-operator

Kubernetes operator that continuously syncs any set of Chart/Kustomize/Manifest fetched from S3/Git/GCS to your cluster
Go
62
star
14

okra

Hot-swap Kubernetes clusters while keeping your service up and running.
Go
48
star
15

decouple-apps-and-eks-clusters-with-tf-and-gitops

Smarty
31
star
16

kodedeploy

CodeDeploy for EKS. Parallel and Continuous (re)delivery to multiple EKS clusters. Blue-green deployment "of" EKS clusters.
Shell
29
star
17

config-registry

Switch between kubeconfigs and avoid unintentional operation on your production clusters.
Go
28
star
18

gosh

(Re)write your bash script in Go and make it testable too
Go
24
star
19

tiny-mmo

A work-in-progress example of server-side implementation of a tiny massive multi-player online game, implemented with Scala 2.10.2 and Akka 2.2
Java
23
star
20

geohash-scala

Geohash library for scala
Scala
23
star
21

prometheus-process-exporter

Helml chart for Prometheus process-exporter
Mustache
22
star
22

waypoint-plugin-helmfile

Helmfile deployment plugin for HashiCorp Waypoint
Go
15
star
23

kube-fluentd

An ubuntu-slim/s6-overlay/confd based docker image for running fluentd in Kubernetes pods/daemonsets
Makefile
15
star
24

kube-magic-ip-address

daemonset that assigns a magic IP address to pods. Useful for implementing node-local services on Kubernetes. Use in combination with dd-agent, dd-zipkin-proxy, kube2iam, kiam, and Elastic's apm-server
Shell
15
star
25

node-detacher

Practically and gracefully stop your K8s node on (termination|scale down|maintenance)
Go
13
star
26

kube-node-init

Kubernetes daemonset for node initial configuration. Currently for modifying files and systemd services on eksctl nodes without changing userdata
Smarty
13
star
27

lambda_bot

JavaScript
12
star
28

argocd-clusterset

A command-line tool and Kubernetes controller to sync EKS clusters into ArgoCD cluster secrets
Go
12
star
29

dcind

Docker Compose (and Daemon) in Docker
Shell
12
star
30

flux-repo

Enterprise-grade secrets management for GitOps
Go
10
star
31

brigade-helmfile-demo

Demo for building an enhanced GitOps pipeline with Flux, Brigade and Helmfile
JavaScript
9
star
32

kubeherd

A opinionated toolkit to automate herding your cattles = ephemeral Kubernetes clusters
Shell
9
star
33

sopsed

Spawning and storage of secure environments powered by sops, inspired from vaulted. Out-of-box support for kubectl, kube-aws, helm, helmfile
Go
9
star
34

helmfile-gitops

8
star
35

nodelocaldns

Temporary location of the nodelocaldns chart until it is upstreamed to helm/charts
Smarty
8
star
36

kube-logrotate

An ubuntu-slim/s6-overlay/confd based docker image for running logrotate via Kubernetes daemonsets
Makefile
7
star
37

idea-play

A plugin for IntelliJ IDEA which supports development of Play framework applications.
Java
7
star
38

actions-runner-controller-ci

Test repo for https://github.com/summerwind/actions-runner-controller
Go
6
star
39

kargo

Deploy your Kubernetes application directly or indirectly
Go
6
star
40

shoal

Declarative, Go-embeddable, and cross-platform package manager powered by https://gofi.sh/
Go
6
star
41

ingress-daemonset-controller

Build your own highly available and efficient external ingress loadbalancer on Kubernetes
Go
6
star
42

conflint

Unified lint runners for various configuration files
Go
5
star
43

fluent-plugin-datadog-log

Sends logs to Datadog Log Management: A port/translation of https://github.com/DataDog/datadog-log-agent to a Fluentd plugin
Ruby
5
star
44

play20-zentasks-tests

Added spec2 tests to Play20's 'zentasks
Scala
4
star
45

versapack

Versatile package manager. Version and host anything in Helm charts repositories
4
star
46

nomidroid

Android app to find nearby restaurants, bars, etc with Recruit Web API
Java
4
star
47

vote4music-play20

A Play 2.0 RC1 port of https://github.com/loicdescotte/vote4music
Scala
4
star
48

homebrew-anyenv

Ruby
4
star
49

terraform-provider-kubectl

Run kubectl against multiple K8s clusters in a single terraform-apply. AWS auth/AssumeRole supported.
Go
3
star
50

kube-node-index-prioritizing-scheduler

Kubernetes scheduler extender that uses sorted nodes' positions as priorities. Use in combination with resource request/limit to implement low-latency highly available front proxy cluster
Go
3
star
51

depot

Ruby
2
star
52

kokodoko

JavaScript
2
star
53

brigade-cd

Go
2
star
54

scripts

my scripts for linux machines
Shell
2
star
55

courier

Go
2
star
56

nodejs-chat

JavaScript
2
star
57

ooina

JavaScript
2
star
58

play-websocket-growl-sample

A sample application notifies the server on Growl, and other users via WebSocket
Java
2
star
59

heartbeat-operator

Monitor any http and tcp services via Kubernetes custom resources
Shell
2
star
60

brigade-automation

TypeScript
2
star
61

nbxmpfilter

xmpfilter for NetBeans
2
star
62

wy

Go
2
star
63

play-scala-lobby-example

Java/Scala mixed app with a WebSocketController in Java, while others are in Scala.
JavaScript
2
star
64

docker-squid-gem-proxy

Dockerfile for building docker image to run Squid-based reverse proxies to rubygems.org, for speeding-up `gem install` or `bundle install`
Shell
2
star
65

hcl2-yaml

YAML syntax for HashiCorp Configuration Language Version 2
Go
2
star
66

tweet_alarm_clock_android

Java
2
star
67

testkit

A toolkit for writing Go tests for your cloud apps, infra-as-code, and gitops configs
Go
1
star
68

elasticdrone

A set of tools to configure auto-scaled Drone.io cluster on AWS in ease
JavaScript
1
star
69

kubeaws-cicd-pipeline-golang

An example of the opinionated deployment pipeline for your Kubernetes-on-AWS-native apps. Secure, declarative, customizable, open app deployments at your hand.
Shell
1
star
70

akka-parallel-tasks

Scala
1
star
71

sing

1
star
72

good-code-is-red-example-for-idea-scala-plugin

Scala
1
star
73

share-snippets

JavaScript
1
star
74

scala-scalable-programming-sidenote

Scala
1
star
75

teleport-on-minikube

Shell
1
star
76

genrun

Go
1
star
77

nomidroid_test

tests for nomidroid the android app.
Java
1
star
78

web_smartphonizer

Scala
1
star
79

idobata4s

The Scala client library for idobata.io API
Scala
1
star
80

specs2-sandbox

Scala
1
star
81

kube-distributed-test-runner

Go
1
star
82

syaml

Set YAML values with `cat your.yaml | syaml a.b.c newValue`
Go
1
star
83

delimited-continuation

An example usage of Delimited Continuation in Scala,;implementing an async HTTP client.
Scala
1
star
84

play2-unit-tests-sample

1
star
85

javascript-missiles-and-lasers

JavaScript
1
star
86

scala-debian-package

1
star
87

glabel

A GOverlay implementation for putting text labels on GMap2
JavaScript
1
star
88

gitops-demo

Makefile
1
star
89

division

Control-plane of your microservices and Kubernetes clusters
Go
1
star
90

mumoauth

Yet another implementation of OAuth aiming to cover both OAuth 1.0a and 2, from Client to Server.
Scala
1
star
91

sam-containerized-go-example

Go
1
star
92

kube-veneur

Sidecar container to run local veneur and Kubernetes service for global https://github.com/stripe/veneur. Uses confd and s6-overlay under the hood.
Makefile
1
star
93

tweet_alarm_clock_android_test

Java
1
star
94

javascript-hue_circle

Draw a hue circle in JavaScript
JavaScript
1
star
95

thread_local

An implementation of the thread-local variable for Ruby
Ruby
1
star
96

play-multijpa

a Play! plugin to switch databases for each play.db.jpa.Model subclass
Java
1
star
97

argocd-example-apps

Jsonnet
1
star
98

logrus-bunyan-formatter

Logrus Bunyan Log Formatter
Go
1
star
99

crdb

Custom Resource DB - Kubernetes custom resources without Kubernetes or self-hosted databases. For any cloud, by leveraging cloudprovider-specific managed databases like AWS DynamoDB
Go
1
star