• Stars
    star
    162
  • Rank 232,284 (Top 5 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created about 2 years ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Generic libraries for building idiomatic Kubernetes controllers

controller-idioms

Go Report Card Go Documentation Discord Server Twitter

Watch a short overview of controller-idioms on CNCF's YouTube

controller-idioms is a collection of generic libraries that complement and extend fundamental Kubernetes libraries (e.g. controller-runtime) to implement best practices for Kubernetes controllers.

These libraries were originally developed by Authzed to build the SpiceDB Operator and their internal projects powering Authzed Dedicated.

Available idioms include:

  • adopt: efficiently watch resources the controller doesn't own (e.g. references to a secret or configmap)
  • bootstrap: install required CRDs and default CRs typically for CD pipelines
  • component: manage and aggregate resources that are created on behalf of another resource
  • fileinformer: an InformerFactory that watches local files typically for loading config without restarting
  • hash: hashing resources to detect modifications
  • metrics: metrics for resources that implement standard metav1.Condition arrays
  • pause: handler that allows users stop the controller reconciling a particular resource without stopping the controller
  • static: controller for "static" resources that should always exist on startup

Have questions? Join our Discord.

Looking to contribute? See CONTRIBUTING.md.

Overview

Handlers

A Handler is a small, composable, reusable piece of a controller's state machine. It has a simple, familiar signature:

func (h *MyHandler) Handle(context.Context) {
	// do some work
}

Handlers are similar to an http.Handler or a grpc.UnaryHandler, but can pass control to another handler as needed.
This allows handlers to compose in nice ways:

func mainControlLoop(ctx context.Context) {
	handler.Chain(
		validateServiceAccount,
		handler.Parallel(
			createJob,
			createPersistentVolume, 
        )
    ).Handle(ctx)
}

The handler package contains utilities for building, composing, and decorating handlers, and for building large state machines with them. See the docs for more details.

Handlers take some inspiration from statecharts to deal with the complexity of writing and maintaining controllers, while staying close to golang idioms.

Typed Context

Breaking a controller down into small pieces with Handlers means that each piece either needs to re-calculate results from other stages or fetch the previously computed result from context.

The typedctx package provides generic helpers for storing / retrieving values from a context.Context.

var CtxExpensiveObject = typedctx.NewKey[ExpensiveComputation]()

func (h *ComputeHandler) Handle(ctx context.Context) {
    ctx = CtxExpensiveObject.WithValue(ctx, myComputedExpensiveObject)
}

func (h *UseHandler) Handle(ctx context.Context) {
    precomputedExpensiveObject = CtxExpensiveObject.MustValue(ctx)
	// do something with object
}

Handlers are typically chained in a way that preserves the context between handlers, but not always.

For example:

var CtxExpensiveObject = typedctx.NewKey[ExpensiveComputation]()

func (h *ComputeHandler) Handle(ctx context.Context) {
    ctx = CtxExpensiveObject.WithValue(ctx, myComputedExpensiveObject)
}

func (h *DecorateHandler) Handle(ctx context.Context) {
	ComputeHandler{}.Handle(ctx)

    // this fails, because the ctx passed into the wrapped handler isn't passed back out 
    CtxExpensiveObject.MustValue(ctx)	
}

To deal with these cases, typedctx provides a Boxed context type that instead stores a pointer to the object, with additional helpers for making a "space" for the pointer to be filled in later.

var CtxExpensiveObject = typedctx.Boxed[ExpensiveComputation](nil)

func (h *ComputeHandler) Handle(ctx context.Context) {
    ctx = CtxExpensiveObject.WithValue(ctx, myComputedExpensiveObject)
}

func (h *DecorateHandler) Handle(ctx context.Context) {
	// adds an empty pointer
	ctx = CtxExpensiveObject.WithBox(ctx)
	
	// fills in the pointer - note that the implementation of ComputeHandler didn't change
	ComputeHandler{}.Handle(ctx)

	// now this succeeds, and returns the unboxed value 
	CtxExpensiveObject.MustValue(ctx)	
}

Typed Informers, Listers, Indexers

The typed package converts (dynamic) kube informers, listers, and indexers into typed counterparts via generics.

informerFactory := dynamicinformer.NewFilteredDynamicSharedInformerFactory(client, defaultResync, namespace, tweakListOptions)
indexer := informerFactory.ForResource(corev1.SchemeGroupVersion.WithResource("secrets")).Informer().Indexer()
secretIndexer := typed.NewIndexer[*corev1.Secret](indexer)

// secrets is []*corev1.Secret instead of unstructured
secrets, err := secretIndexer.ByIndex("my-index-name", "my-index-value")

Controllers and Managers

The manager package provides an optional lightweight controller Manager abstraction (similar to kubernetes controller manager, or the manager from controller runtime). It also provides a simple Controller abstraction and some basic implementations.

The rest of controller-idioms can be used without using these if you are already using another solution.

Manager

  • provides a way to start and stop a collection of controllers together
  • controllers can be dynamically added/removed after the manager has started
  • starts health / debug servers that tie into the health of the underlying controllers

BasicController

  • provides default implementations of health and debug servers

OwnedResourceController

  • implements the most common pattern for a controller: reconciling a single resource type via a workqueue
  • has a single queue managing a single type
  • on start, starts processing objects from the queue
  • doesn't start any informers

Informer Factory Registry

It can be useful to access the informer cache of one controller from another place, so that multiple controllers in the same binary don't need to open separate connections against the kube apiserver and maintain separate caches of the same objects.

The typed package provides a Registry that synchronizes access to shared informer factories across multiple controllers.

Queue

The queue package provides helpers for working with client-go's workqueues.

queue.OperationsContext can be used from within a Handler to control the behavior of the queue that has called the handler.

The queue operations are:

  • Done (stop processing the current key)
  • Requeue (requeue the current key)
  • RequeueAfter (wait for some period of time before requeuing the current key)
  • ReqeueueErr (record an error and requeue)
  • RequeueAPIError (requeue after waiting according to the priority and fairness response from the apiserver)

If calling these controls from a handler, it's important to return immediately so that the handler does not continue processing a key that the queue thinks has stopped.

Middleware

Middleware can be injected between handlers with the middleware package.

middleware.ChainWithMiddleware(
    middleware.NewHandlerLoggingMiddleware(4),
)(
    c.checkPause,
    c.adoptSecret,
    c.validate
).Handle(ctx)

More Repositories

1

spicedb

Open Source, Google Zanzibar-inspired permissions database to enable fine-grained access control for customer applications
Go
4,405
star
2

zed

Official command-line tool for managing SpiceDB
Go
101
star
3

authzed-go

Official SpiceDB client library for Go
Go
66
star
4

zanzibar-annotated

Google's Zanzibar paper annotated for folks outside of Google
TypeScript
64
star
5

spicedb-operator

Kubernetes controller for managing instances of SpiceDB
Go
57
star
6

awesome-spicedb

An awesome list for the SpiceDB ecosystem
54
star
7

authzed-node

Official SpiceDB client library for NodeJS
TypeScript
42
star
8

consistent

gRPC Balancer that routes requests using a consistent hashring
Go
37
star
9

prom-authzed-proxy

A Prometheus proxy that performs SpiceDB permission checks based on labels
Go
33
star
10

authzed-py

Official SpiceDB client library for Python
Python
29
star
11

examples

A collection of examples for SpiceDB users
Go
26
star
12

crdbpool

node aware connection pooling for CockroachDB
Go
21
star
13

spicedb-kubeapi-proxy

Secure access to the Kubernetes API using SpiceDB
Go
21
star
14

animated-code-example-component

Embeddable component for displaying an animated code example, as well as a REPL and a browser window
TypeScript
17
star
15

authzed-java

Official SpiceDB client library for JVM languages
Java
16
star
16

authzed-rb

Official SpiceDB client library for Ruby
Ruby
15
star
17

api

Protocol Buffers & gRPC Services used by SpiceDB
13
star
18

action-spicedb-validate

GitHub Action for validating your SpiceDB schema
Dockerfile
11
star
19

connector-postgresql

Import PostgreSQL foreign key relationships into SpiceDB
Go
9
star
20

grpcutil

various utilities to simplify common gRPC APIs
Go
8
star
21

action-spicedb

GitHub Action for integration testing your application with SpiceDB
Shell
8
star
22

docs

Documentation website for Authzed & SpiceDB
MDX
7
star
23

spicedb-vscode

VS Code extension adding SpiceDB syntax highlighting, linting, and more
TypeScript
6
star
24

zed-testserver

Unit and Integration Testing server for Authzed
Dockerfile
4
star
25

action-testserver

@github action for running an @authzed test server
Dockerfile
3
star
26

actions

GitHub Actions used on various Authzed projects
JavaScript
3
star
27

servok

Serve endpoint metadata for client side load balancing
Go
3
star
28

cla

The agreements and signatures of contributors of the Authzed CLA
2
star
29

.github

Community defaults for our projects
1
star
30

homebrew-tap

a collection of @homebrew formula for @authzed
Ruby
1
star
31

tree-sitter-spicedb

tree-sitter grammar for SpiceDB schemas
JavaScript
1
star