• Stars
    star
    200
  • Rank 195,325 (Top 4 %)
  • Language
    Go
  • License
    GNU Affero Genera...
  • Created over 4 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

CLI for executing Cirrus tasks locally and in any CI

Cirrus CLI

Build Status

Cirrus CLI is a tool for running containerized tasks reproducibly in any environment. Most commonly, Cirrus tasks are used as part of continuous integration workflows but can also be used as part of local development process as a hermetic replacement of helper scripts/Makefiles. Cirrus CLI runs your tasks locally the same way they are executed in CI or on your colleague's machine. Immutability of containers ensures the tasks will be executed the same way years from now regardless what versions of packages you'll have locally.

Cirrus CLI Demo

Installation

Usage

Cirrus CLI reuses the same YAML configuration format as Cirrus CI which allows to reuse a large list of examples created by Cirrus CI community.

Note: Cirrus CLI can be used in any environment that has Docker or Podman installed. It can be your laptop or any CI system you already have like Jenkins, GitHub Actions, Travis CI, etc. With Cirrus CLI it's no longer a requirement to use Cirrus CI in order to benefit from Cirrus configuration format that we (Cirrus Labs) have crafted for so long and really proud of.

Here is an example of .cirrus.yml configuration file for testing a Go application with several Go versions:

task:
  env:
    matrix:
      VERSION: 1.15
      VERSION: 1.14
  name: Tests (Go $VERSION)
  container:
    image: golang:$VERSION
  modules_cache:
    fingerprint_script: cat go.sum
    folder: $GOPATH/pkg/mod
  get_script: go get ./...
  build_script: go build ./...
  test_script: go test ./...

Running Cirrus Tasks

To run Cirrus tasks, simply switch to a directory where the .cirrus.yml is located and run:

cirrus run

By default, working directory will be rsynced into a container while respecting .gitignore configuration. This makes sure Cirrus Tasks are executed from a clean state only with source code changes.

In case rsync-ing the whole working directory is too costly, you can pass a --dirty flag which will result in all operations being done against the actual working directory (and not it's rsynced copy):

cirrus run --dirty Lint

Since most linters and code-analysis tools are read-only by their nature there is no need in extra precautions and the potentially costly rsync-ing can be safely avoided.

It is also possible to run a particular task by name:

cirrus run "Tests (Go 1.15)"

Or pass some extra environment variables with -e flag:

cirrus run -e CIRRUS_TAG="test-release" Release

Note: Cirrus CLI only supports Linux container and macos_instance VMs at the moment. Linux containers support the Dockerfile as a CI environment feature.

Validating Cirrus Configuration

To validate a Cirrus configuration, simply switch to a directory where the .cirrus.yml is located and run:

cirrus validate

Caching

By default, Cirrus CLI stores blob artifacts produced by the cache instruction in the user-specific cached data folder. Similar to Cirrus Cloud the CLI can use a caching HTTP server for more efficient sharing of cached artifacts between tasks executed on different physical hosts.

Caching HTTP server should support a single /<key> REST endpoint with PUT, GET and HEAD methods available for uploading, downloading and checking availability of a cached artifact under <key> key respectively. There are reference implementations of such HTTP servers for Google Cloud Storage and AWS S3 and Azure's Blob Storage.

To start using your own HTTP caching server simply pass it's hostname as CIRRUS_HTTP_CACHE_HOST to run command:

cirrus run --environment CIRRUS_HTTP_CACHE_HOST=http-cache-host.internal:8080

Security

Cirrus CLI aims to run in different environments, but in some environments we choose to provide more usability at the cost of some security trade-offs:

  • SELinux
    • both the task container and the helper container (that copies the project directory into a per-task container volume using rsync) run unconfined

Please open an issue if your use-case requires a different approach.

FAQ

What is the relationship between Cirrus CI and Cirrus CLI?

Cirrus CI was released in the early 2018 with an idea to bring some innovation to CI space. A lot of things have changed in CI-as-a-service space since then but Cirrus CI pioneered many ideas in CI-as-a-service space including per-second billing and support for Linux, Windows and macOS all together.

Over the past two and a half years we heard only positive feedback about Cirrus CI's YAML configuration format. Users liked how concise their configuration looked and that it was easy to reason about.

Another feedback we heard from users was that it's hard to migrate from one CI to another. There is a need to rewrite CI configurations from one format into another that basically still locks into another vendor.

And now in 2020 with Cirrus CLI we are trying to solve the "vendor lock" problem by popularizing Cirrus configuration format and building community around it. Stay tuned for the upcoming option to use Starlark templates instead of YAML!

Think of Cirrus CLI as an executor of Cirrus Tasks on a single machine only in Docker containers for simple CI scenarious. And Cirrus CI as an option for more specific cases where Cirrus Tasks can be executed in containers and VMs using a variety of supported compute services or using a managed infrastructure with per-second billing.

More Repositories

1

tart

macOS and Linux VMs on Apple Silicon to use in CI and other automations
Swift
3,743
star
2

docker-images-flutter

Docker Images for Flutter
Shell
443
star
3

echelon

hierarchical progress bars in terminal on steroids
Go
371
star
4

cirrus-ci-docs

Documentation for Cirrus CI 📚
HTML
350
star
5

macos-image-templates

HCL
205
star
6

orchard

Orchestrator for running Tart Virtual Machines on a cluster of Apple Silicon devices
Go
190
star
7

cirrus-ci-web

Web front end for Cirrus CI
TypeScript
92
star
8

packer-plugin-tart

Packer builder for Tart VMs
Go
63
star
9

vetu

Create, publish and virtualize ephemeral Linux VMs with ease
Go
63
star
10

gitlab-tart-executor

GitLab Runner executor to run jobs in Tart VMs
Go
55
star
11

docker-images-android

Android SDK Docker Images
Dockerfile
39
star
12

http-cache-action

HTTP Caching service for GitHub Actions
Go
23
star
13

softnet

Software networking with isolation for Tart
Rust
22
star
14

gh-submit-queue

GitHub App to Introduce Submit Queue
Kotlin
19
star
15

bazel-project-generator

Bazel BUILD files generator for Kotlin/Java projects.
Kotlin
14
star
16

cirrus-ci-agent

Agent to execute Cirrus CI tasks
Go
13
star
17

linux-image-templates

HCL
13
star
18

vagrant-tart

Go
11
star
19

google-storage-proxy

HTTP proxy with REST API to interact with Google Cloud Storage Buckets
Go
10
star
20

tart-buildkite-plugin

Run Buildkite steps inside Tart Virtual Machines
Shell
10
star
21

terminal

A service that powers Cirrus CI's task terminal access
Go
8
star
22

swiftlint-action

Run SwiftLint from your GitHub Actions with ease
TypeScript
8
star
23

vm-images

Common images to use with Cirrus CI
7
star
24

azure-blob-storage-proxy

HTTP proxy with REST API to interact with Azure Blob Storage
Go
7
star
25

ansible-orchard

Ansible roles to configure Orchards workers
Jinja
7
star
26

aws-s3-proxy

HTTP S3 proxy
Go
6
star
27

docker-images-windows

Base Windows Docker images for Cirrus CI
Dockerfile
6
star
28

homebrew-cli

Homebrew formula for Cirrus CLI
Ruby
6
star
29

cirrus-ci-annotations

Parse and Create Cirrus CI Annotations
Go
6
star
30

intellij-starlark

Python
4
star
31

go-java-glob

A port of the Java's NIO globbing functionality to Golang
Go
3
star
32

anka-controller-extended

Bridge for Anka Controller's API to expend functionality
Kotlin
3
star
33

docker-wget

alpine-based Docker image with working SSL
Dockerfile
2
star
34

gradle-example

Kotlin
2
star
35

podmanapi

Swagger-generated Podman REST API client
Go
2
star
36

cirrus-action

GitHub Action to run Cirrus Tasks
TypeScript
1
star
37

backbone-services

Abstract backbone services for your apps
Starlark
1
star
38

core-services

Useful adapters for Pub/Sub and key-value services by different vendors
Go
1
star
39

docker-images-bazel

Bazel Docker Images
Shell
1
star
40

tart-ecid-hacking

Go
1
star
41

cirrus-webhooks-server

Example of a simple server that processes webhook events from Cirrus CI
Go
1
star
42

actions-playground

Shell
1
star