• Stars
    star
    154
  • Rank 233,693 (Top 5 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created about 6 years ago
  • Updated over 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

An Kubernetes validating admission webhook that rejects pods that use environment variables.

Getting Started with Kubernetes Validating Admission Webhooks the FaaS Way

Kubernetes is a platform for running and managing application containers and has slowly evolved into a platform for building platforms, largely thanks to its extensible API. Kubernetes has many extension points including extensions that enable you to define custom resource types, cloud provider and container runtime integrations.

Another, less well known, set of extension points are the admission controllers. An admission controller is a piece of code that intercepts requests to the Kubernetes API prior to persistence of the object, but after the request is authenticated and authorized. Most admission controllers are built into Kubernetes and cover a range of functionality.

To understand how admission controllers work you need to see them in action. Take the namespace exists admission controller for example, it rejects all requests that attempt to create resources in a namespace that does not exist.

If you were to list the active namespaces available to a new Kubernetes install you would see two or three namespaces including the default and kube-system namespaces. The kube-system namespace is where things like Kubernetes DNS and the Kubernetes Dashboard live.

kubectl get ns

output:

NAME          STATUS    AGE
default       Active    3m
kube-public   Active    3m
kube-system   Active    3m

If you tried to create a deployment in a namespace that did not exist you would get an error because the namespace exists admission controller would reject it.

kubectl run nginx --image nginx --namespace does-not-exist

output:

Error from server (NotFound): namespaces "does-not-exist" not found

With a basic understanding of how admission controllers work it's time to take a look at the newest admission controller, the validating admission webhook. As it names implies, the validating admission webhook allows you to intercept and validate requests to the Kubernetes API using an external webhook, but not mutate them. That last point is critical; because validating admission webhooks can't mutate resources it's safe to run them in parallel, and quickly reject a request if any of the webhooks fail.

Before you can use them you'll need access to a Kubernetes cluster.

Kubernetes the Easy Way

Before you can use validating admission webhooks you need access to a Kubernetes cluster.

There are many ways to provision a Kubernetes cluster, but I'm going to assume you want me to tell you exactly what to do so we can get back to building a validating admission webhook. This is where I point you to Google Kubernetes Engine and swear it's not a vendor pitch. While I'm recommending GKE, everything should work on minikube or Docker support for Kubernetes.

If you've chosen to follow along with GKE, run the following commands and wait for them to finish.

First create a 1.9.2+ Kubernetes cluster. We are going to spin a 1 node cluster to help save you some cash, and lighten the blow those bitcoin crashes are taking on your wallet.

Use the gcloud command to create the k0 Kubernetes cluster:

gcloud container clusters create k0 \
  --cluster-version 1.9.2-gke.1 \
  --zone us-central1-a \
  --num-nodes 1

With the k0 cluster in place it's time to design and build a validating admission webhook.

How to Write a Validating Admission Webhook

We are going to write our first validating admission webhook using nocode.

I'll give you a minute.

That's right, don't write a single line of code until you determine what rules you plan to use for accepting or rejecting a Kubernetes resource. For our first webhook we are going to keep things simple. We are going to reject all pods that leverage environment variables for application configuration.

You can write validating admission webhooks in just about any programming language, except nocode, and deploy it on any platform, including Kubernetes. But remember, the only goal is to validate incoming Kubernetes resources, not build a generic web application. In theory we only need to write a small bit of code to make that happen.

function denyenv (req, res) {
  // Review the Kubernetes Pod resource and reject it if any
  // of the containers are using environment variables.
}

Given the minimual requirements for our validating admission webhook, we are going to skip containers altogether and deploy our webhook using a Function as a Service (FaaS) platform. Also know as Serverless.

Serverless to the Rescue

We are going to implement our validating admission webhook using Node.js and deploy it to Google Cloud Functions.

Now it's time to write the validating admission webhook. Create a directory named denyenv and move into it:

mkdir denyenv && cd denyenv

Next, save the following code block to a file named index.js:

'use strict';

exports.denyenv = function denyenv (req, res) {
  var admissionRequest = req.body;

  // Get a reference to the pod spec
  var object = admissionRequest.request.object;

  console.log(`validating the ${object.metadata.name} pod`);

  var admissionResponse = {
    allowed: false
  };

  var found = false;
  for (var container of object.spec.containers) {
    if ("env" in container) {
      console.log(`${container.name} is using env vars`);

      admissionResponse.status = {
        status: 'Failure',
        message: `${container.name} is using env vars`,
        reason: `${container.name} is using env vars`,
        code: 402
      };

      found = true;
    };
  };

  if (!found) {
    admissionResponse.allowed = true;
  }

  var admissionReview = {
    response: admissionResponse
  };

  res.setHeader('Content-Type', 'application/json');
  res.send(JSON.stringify(admissionReview));
  res.status(200).end();
};

Run the following command to create a new function named denyenv:

gcloud beta functions deploy denyenv --trigger-http

Retrieve the HTTPS URL that triggers the denyenv function as we'll need it later when configuring our Kubernetes cluster to use it.

HTTPS_TRIGGER_URL=$(gcloud beta functions describe denyenv \
  --format 'value(httpsTrigger.url)')

Validating Admission Webhook Configuration

With the denyenv function in place, it’s time to configure the Kubernetes cluster to use it.

Start by generating a validating webhook configuration and submitting it to the Kubernetes API:

cat <<EOF | kubectl apply -f -
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: denyenv
webHooks:
  - name: denyenv.hightowerlabs.com
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - v1
        operations:
          - CREATE
        resources:
          - pods
    failurePolicy: Fail
    clientConfig:
      url: "${HTTPS_TRIGGER_URL}"
EOF

output:

validatingwebhookconfiguration "denyenv" created

At this point we are all set to start testing the denyenv webhook.

Testing

With the the denyenv validating admission webhook in place we need to ensure that it’s working.

First let’s make sure pods without env vars can be deployed to our Kubernetes clusters:

kubectl run nginx --image=nginx

output:

deployment "nginx" created

List the pods:

kubectl get pods

output:

NAME                   READY     STATUS    RESTARTS   AGE
nginx-8586cf59-qmdm4   1/1       Running   0          25s

It works. We can review the logs of the denyenv cloud function to show that it was indeed called:

gcloud beta functions logs read denyenv

output:

D      denyenv  wb9bmjb34uyh  2018-02-09 02:06:35.534  Function execution started
I      denyenv  wb9bmjb34uyh  2018-02-09 02:06:35.542  validating the nginx-8586cf59-qmdm4 pod
D      denyenv  wb9bmjb34uyh  2018-02-09 02:06:35.545  Function execution took 12 ms, finished with status code: 200

Next we need to ensure our webhook actually does the thing it’s designed to do, which is prevent pods with containers using env vars.

kubectl run nginx-with-env --image=nginx --env="PASSWORD=fail"

output:

deployment "nginx-with-env" created

List the pods:

kubectl get pods

output:

NAME                   READY     STATUS    RESTARTS   AGE
nginx-8586cf59-qmdm4   1/1       Running   0          1m

Its not there. What's going on?

The best way to determine why the nginx-with-env pod is not running is to review the Kubernetes event stream:

kubectl get events

output:

Warning   FailedCreate  replicaset-controller Error creating: admission webhook "denyenv.hightowerlabs.com" denied the request: nginx-with-env is using env vars

Looks like the nginx-with-env pod was denied by the denyenv admission webhook. We can confirm by fetching the logs for the denyenv function.

gcloud beta functions logs read denyenv
I      denyenv  s1ij4wfykerr  2018-02-09 02:09:31.457  validating the nginx-with-env-76498b5b66-nw259 pod
I      denyenv  s1ij4wfykerr  2018-02-09 02:09:31.459  nginx-with-env is using env vars
D      denyenv  s1ij4wfykerr  2018-02-09 02:09:31.461  Function execution took 8 ms, finished with status code: 200

Conclusion

Validating admission webhooks are one of easiest ways of extending Kubernetes with new policy controls. Building and running admission webhooks using a FaaS platform can help streamline the development process, and make it easy to enforce policy across multiple Kubernetes clusters using a single function.

More Repositories

1

nocode

The best way to write secure and reliable applications. Write nothing; deploy nowhere.
Dockerfile
59,013
star
2

kubernetes-the-hard-way

Bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts.
38,169
star
3

confd

Manage local application configuration files using templates and data from etcd or consul
Go
8,274
star
4

envconfig

Golang library for managing configuration data from environment variables
Go
4,881
star
5

kube-cert-manager

Manage Lets Encrypt certificates for a Kubernetes cluster.
Go
1,087
star
6

intro-to-kubernetes-workshop

Intro to Kubernetes Workshop
1,016
star
7

pipeline

A step by step guide on creating build and deployment pipelines for Kubernetes.
747
star
8

consul-on-kubernetes

Running HashiCorp's Consul on Kubernetes
Shell
598
star
9

konfd

Manage application configuration using Kubernetes secrets, configmaps, and Go templates.
Go
494
star
10

kubernetes-cluster-federation

Kubernetes cluster federation tutorial
458
star
11

vault-controller

Automate the creation of unique Vault tokens for Kubernetes Pods using init containers.
Go
448
star
12

compose2kube

Convert docker-compose service files to Kubernetes objects.
Go
417
star
13

serverless-vault-with-cloud-run

Guide to running Vault on Cloud Run
Shell
392
star
14

vault-on-google-kubernetes-engine

How to guide on running HashiCorp's Vault on Google Kubernetes Engine
Shell
387
star
15

app

Example 12 Facter App
Go
379
star
16

terminus

Get facts about a Linux system.
Go
361
star
17

nomad-on-kubernetes

Tutorial on running Nomad on Kubernetes.
Shell
342
star
18

istio-ingress-tutorial

How to run the Istio Ingress Controller on Kubernetes
Shell
322
star
19

kubernetes-initializer-tutorial

Hands-on tutorial for building and deploying Kubernetes Initializers.
Go
321
star
20

kubestack

Manage Kubernetes with Packer and Terraform on Google Compute Engine.
297
star
21

grpc-hello-service

grpc examples
Go
272
star
22

coreos-ipxe-server

CoreOS iPXE server
Go
218
star
23

kubernetes-redis-cluster

Kubernetes Redis Cluster configs and tutorial
Shell
215
star
24

standalone-kubelet-tutorial

Standalone Kubelet Tutorial
Go
204
star
25

grafeas-tutorial

A step by step guide for getting started with Grafeas and Kubernetes.
Go
191
star
26

docker-kubernetes-tls-guide

Step by step guide on how to secure Docker and Kubernetes using TLS with CloudFlare’s CFSSL
191
star
27

google-cloud-functions-go

Google Cloud Function tutorial and hacks to enable the use of Go.
Go
180
star
28

helloworld

Go
180
star
29

scheduler

Toy Kubernetes Scheduler
Go
174
star
30

craft-kubernetes-workshop

Craft Kubernetes Workshop
Shell
172
star
31

ingress-with-static-ip

Tutorial on creating a Kubernetes Ingress Resource with a Static IP Address in GCP or GKE
165
star
32

mesh

Cloud native service mesh for the rest of us.
160
star
33

lobsters-on-kubernetes

Lobsters, the Hacker News clone, on Kubernetes
Shell
156
star
34

vault-init

Automate the initialization and unsealing of HashiCorp Vault on Google Cloud Platform.
Go
151
star
35

certificate-init-container

Bootstrap TLS certificates for Pods using the Kubernetes certificates API.
Go
146
star
36

kubeadm-single-node-cluster

How to bootstrap a single-node Kubernetes cluster on Google Compute Engine using kubeadm.
Shell
140
star
37

hashiapp

Demo 12 Factor application that utilizes Hashicorp tools.
Go
134
star
38

kubernetes-envoy-sds

Kubernetes Envoy Service Discovery Service.
Go
133
star
39

setup-network-environment

Create an environment file with system networking information.
Go
131
star
40

kargo

Go
125
star
41

contributors

Display GitHub contributors for a specific repo.
Go
123
star
42

self-deploying-hello-universe

What if applications could deploy themselves?
Go
123
star
43

konfig

Go
119
star
44

event-gateway-on-kubernetes

How to guide on running Serverless.com's Event Gateway on Kubernetes
Go
119
star
45

gke-service-accounts-tutorial

A tutorial on using Google Cloud service account with Google Container Engine (GKE).
Go
109
star
46

run

Package run provides helper functions for building Cloud Run applications.
Go
107
star
47

hashiconf-eu-2016

HashiConf EU 2016
Shell
106
star
48

talks

Shell
99
star
49

badger

Generate build status images for Google Cloud Build
Go
98
star
50

riff-tutorial

How-to guide for testing the riff FaaS platform and Istio on Google Kubernetes Engine.
Go
97
star
51

cri-o-tutorial

A guided tutorial for the cri-o (ocid) Kubernetes container runtime.
95
star
52

lambda-on-cloud-run

Tutorial: Running Lambda Functions on Cloud Run
Dockerfile
83
star
53

gophercon-2018

Kelsey's GopherCon 2018 Keynote: Going Serverless
Go
72
star
54

12-fractured-apps

Example code for the 12 Fractured Apps blog posts.
Go
71
star
55

etcd-production-setup

Setting up etcd to run in production.
69
star
56

cmd-tutorial

68
star
57

oscon-2017-kubernetes-tutorial

OSCON 2017 Kubernetes Tutorial
68
star
58

jira-on-kubernetes

Notes: Running Atlassian's Jira on Kubernetes
67
star
59

conf2kube

conf2kube can read and create Kubernetes secrets based on the contents of configuration files.
Go
64
star
60

endpoints

Kubernetes endpoints load balancer.
Go
62
star
61

oscon-metrics-tutorial

OSCON Metrics Tutorial
60
star
62

echo

echo prints the first positional argument to stdout
Assembly
59
star
63

memkv

Simple in-memory key/value store backed by a map
Go
58
star
64

motorboat

Dynamically sync Nginx Plus backends from Kubernetes service endpoints.
Go
57
star
65

app-healthz

Example app with a healthz endpoint
Go
55
star
66

dynamic-ports-tutorial

A prototype of using dynamic ports with Kubernetes.
Go
54
star
67

ipxed

Web interface and api for ipxed
Go
51
star
68

helloworld-infrastructure-production

51
star
69

intro-to-go-workshop

Intro to Go Workshop
Go
50
star
70

pipeline-application

Go
48
star
71

reposync

Sync GitHub and Google Cloud Source Repos
Go
45
star
72

journal-2-logentries

Ship systemd journal entries to logentries.com
Go
45
star
73

pm

Package manager
Go
44
star
74

time

Educational package to teach aspiring programmers about history through the lens of time.
Go
44
star
75

container-instance-metadata-server

Cloud Run Container Instance Metadata Server Emulator
Go
41
star
76

kube

Basic single node Kubernetes using a bash script.
Shell
40
star
77

helloworld-infrastructure-staging

36
star
78

gcscache

GCS Cache implements the autocet.Cache interface using Google Cloud Storage.
Go
35
star
79

helloworld-infrastructure-qa

35
star
80

memq

In memory message queue prototype.
Go
32
star
81

kubestack-solo

Create a single node Kubernetes image for local testing with VMware Fusion.
32
star
82

jsonrpc-server

Complete example on using net/rpc over HTTP with the jsonrpc encoding
Go
32
star
83

opa-on-cloud-run

Tutorial: Open Policy Agent on Cloud Run
Shell
31
star
84

confidence

Example application which demonstrates various configuration options for modern applications.
Go
30
star
85

cloud-functions-min-instances-tutorial

Cloud Functions Min Instances Tutorial
Go
30
star
86

hello-cuelang

Learning CUE in public
Go
29
star
87

kube-rsa

Generate self-signed TLS certificates for Kubernetes
Go
29
star
88

kur

Kubernetes Up and Running
Shell
28
star
89

echod

A small echo server written in x86-32 asm
Assembly
28
star
90

redis-enterprise-on-kubernetes

How to deploy Redis Enterprise on Kubernetes
Shell
27
star
91

kubernetes-letsencrypt-tutorial

WIP: Kubernetes Lets Encrypt Tutorial
Go
27
star
92

krane

Convert Google Compute Engine (GCE) autoscaling instance groups to Kubernetes autoscaling deployments.
Go
27
star
93

twelve

Go
26
star
94

buildinfo

Go
25
star
95

etcd-pod-gen

Generate etcd pod specs for running under the Kubernetes Kubelet
Go
24
star
96

istio-initializer

Kubernetes Initializer that injects the Istio sidecar into pods.
Go
24
star
97

config-connector-policy-demo

Kubernetes Config Connector Policy Demo.
Open Policy Agent
24
star
98

dialogflow

The best way to create Dialogflow webhooks in Go.
Go
23
star
99

functions

Google Cloud Functions Helpers
Go
23
star
100

terraform-kcc-demo

Terraform KCC Demo
Shell
22
star