FairwindsOps/gemini

Gemini is a Kubernetes CRD and operator for managing VolumeSnapshots. This allows you to create a snapshot of the data on your PersistentVolumes on a regular schedule, retire old snapshots, and restore snapshots with minimal downtime.

Installation

The Gemini Helm chart will install both the CRD and the operator into your cluster

kubectl create ns gemini
helm repo add fairwinds-stable https://charts.fairwinds.com/stable
helm install gemini fairwinds-stable/gemini --namespace gemini

Prerequisites

You'll need to have the VolumeSnapshot API available in your cluster. This API is in beta as of Kubernetes 1.17, and was introduced as alpha in 1.12.

To check if your cluster has VolumeSnapshots available, you can run

kubectl api-resources | grep volumesnapshots

• To enable on v1.12-16, set the flag --feature-gates=VolumeSnapshotDataSource=true on the API server binary source
• To enable VolumeSnapshots on kops, see our instructions here
• Depending on your environment, you may need to configure the VolumeSnapshot API as well as the CSI. Fortunately, some managed Kubernetes providers like DigitalOcean support VolumeSnapshots by default, even on older versions

Before getting started with Gemini, it's a good idea to make sure you're able to create a VolumeSnapshot manually.

Upgrading to V2

Version 2.0 of Gemini updates the CRD from v1beta1 to v1. There are no substantial changes, but v1 adds better support for PersistentVolumeClaims on Kubernetes 1.25.

If you want to keep the v1beta1 CRD available, you can run:

kubectl apply -f https://raw.githubusercontent.com/FairwindsOps/gemini/main/pkg/types/snapshotgroup/v1beta1/crd-with-beta1.yaml

before upgrading, and add --skip-crds when running helm install.

Usage

Snapshots

Gemini can schedule snapshots for an existing PVC, or create a new PVC to back up.

Schedules

The schedule parameter tells Gemini how often to create snapshots, and how many historical snapshots to keep.

For example, the following schedule tells Gemini to create a snapshot every day, keeping two weeks worth of history:

apiVersion: gemini.fairwinds.com/v1beta1
kind: SnapshotGroup
metadata:
  name: test-volume
spec:
  persistentVolumeClaim:
    claimName: postgres
  schedule:
    - every: day
      keep: 14

For a more complex example, Gemini can create new snapshots every 10 minutes, always keep the last 3 snapshots, and preserve historical hourly, daily, monthly, and yearly snapshots.

apiVersion: gemini.fairwinds.com/v1beta1
kind: SnapshotGroup
metadata:
  name: test-volume
spec:
  persistentVolumeClaim:
    claimName: postgres
  schedule:
    - every: 10 minutes
      keep: 3
    - every: hour
      keep: 1
    - every: day
      keep: 1
    - every: month
      keep: 1
    - every: year
      keep: 1

Note that keep specifies how many historical snapshots you want, in addition to the most recent snapshot. This way the schedule

- every: 10 minutes
  keep: 3

will always give you at least 30 minutes of snapshot coverage. But you will see four snapshots at any given time. E.g. right after a new snapshot is created, you'll see snapshots for

• 0m ago
• 10m ago
• 20m ago
• 30m ago

Using an Existing PVC

See the extended example The following example schedules snapshots every 10 minutes for a pre-existing PVC named postgres.

apiVersion: gemini.fairwinds.com/v1beta1
kind: SnapshotGroup
metadata:
  name: test-volume
spec:
  persistentVolumeClaim:
    claimName: postgres
  schedule:
    - every: 10 minutes
      keep: 3

Creating a New PVC

You can also specify an entire PVC spec inside the SnapshotGroup if you'd like Gemini to create the PVC for you.

apiVersion: gemini.fairwinds.com/v1beta1
kind: SnapshotGroup
metadata:
  name: test-volume
spec:
  persistentVolumeClaim:
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 1Gi
  schedule:
    - every: 10 minutes
      keep: 3

The PVC will have the same name as the SnapshotGroup, (in this example, test-volume)

Snapshot Spec

You can use the spec.template field to set the template for any VolumeSnapshots that get created, most notably the name of the snapshot class you want to use.

apiVersion: gemini.fairwinds.com/v1beta1
kind: SnapshotGroup
metadata:
  name: test-volume
spec:
  persistentVolumeClaim:
    claimName: postgres
  schedule:
    - every: "10 minutes"
      keep: 3
  template:
    spec:
      volumeSnapshotClassName: test-snapshot-class      

Restore

Caution: you cannot alter a PVC without some downtime! You can restore your PVC to a particular point in time using an annotation.

First, check out what VolumeSnapshots are available:

$ kubectl get volumesnapshot
NAME                           AGE
test-volume-1585945609         15s

Next, you'll need to remove any Pods that are using the PVC:

$ kubectl scale all --all --replicas=0

Then, copy the timestamp from the first step, and use that to annotate the SnapshotGroup:

$ kubectl annotate snapshotgroup/test-volume --overwrite \
  "gemini.fairwinds.com/restore=1585945609"

Finally, you can scale your Pods back up:

$ kubectl scale all --all --replicas=1

End-to-End Example

To see gemini working end-to-end, check out the CodiMD example

Caveats

• Like the VolumeSnapshot API it builds on, Gemini is currently in beta
• Be sure to test out both the snapshot and restore process to ensure Gemini is working properly
• VolumeSnapshots simply grab the current state of the volume, without respect for things like in-flight database transactions. You may find you need to stop the application in order to get a consistently usable VolumeSnapshot.

Join the Fairwinds Open Source Community

The goal of the Fairwinds Community is to exchange ideas, influence the open source roadmap, and network with fellow Kubernetes users. Chat with us on Slack join the user group to get involved!

Other Projects from Fairwinds

Enjoying Gemini? Check out some of our other projects:

• Polaris - Audit, enforce, and build policies for Kubernetes resources, including over 20 built-in checks for best practices
• Goldilocks - Right-size your Kubernetes Deployments by compare your memory and CPU settings against actual usage
• Pluto - Detect Kubernetes resources that have been deprecated or removed in future versions
• Nova - Check to see if any of your Helm charts have updates available
• rbac-manager - Simplify the management of RBAC in your Kubernetes clusters