Kubediff
Kubediff is a tool for Kubernetes to show you the differences between your running configuration and your version controlled configuration.
Kubediff can be run from the command line:
$ ./kubediff
usage: kubediff [-h] [--kubeconfig KUBECONFIG] [--context CONTEXT] [--namespace NAMESPACE] [--json] [--no-error-on-diff] [paths ...]
_ _ _ _ __ __
| |__ _ _ | |__ ___ __| |(_) / _| / _|
| / /| || || '_ \/ -_)/ _` || || _|| _|
|_\_\ \_,_||_.__/\___|\__,_||_||_| |_|
Compare yaml files in path(s) to running state in kubernetes and print the
differences. This is useful to ensure you have applied all your changes
to the appropriate environment. This tools runs kubectl, so unless your
~/.kube/config is configured for the correct environment, you will need
to supply the kubeconfig for the appropriate environment.
positional arguments:
paths path(s) from which kubediff will look for configuration files
optional arguments:
-h, --help show this help message and exit
--kubeconfig KUBECONFIG, -k KUBECONFIG
path to kubeconfig
--context CONTEXT, -c CONTEXT
name of kubeconfig context to use
--namespace NAMESPACE, -n NAMESPACE
Namespace to assume for objects where it is not specified (default = Kubernetes default for current context)
--json, -j output in json format
--no-error-on-diff, -e
don't exit with 2 if diff exists
For example:
$ ./kubediff k8s
Checking ReplicationController 'kubediff'
*** .spec.template.spec.containers[0].args[0]: '-repo=https://github.com/weaveworks/kubediff' != '-repo=https://github.com/<your github repo>'
Checking Secret 'kubediff-secret'
Checking Service 'kubediff'
Make sure the dependencies are installed first:
$ pip install -r requirements.txt
Kubediff can also be run as a service on Kubernetes, periodically downloading the latest configuration from Github, comparing it to the running configuration. In this mode Kubediff will also offers a very simple UI showing the output and export the result to Prometheus, all courtesy to prom-run.
To deploy to Kubernetes, you much first make a copy of the YAML files in k8s
and update the following fields:
kubediff-rc.yamlthe first argument to git-sync must be the location of the config repo, and the last argument to kubediff must the the location in this repo of your config.kubediff-secret.yamlthe username and password must be set to valid github OAuth token.
Once you have updated the config, the following commands should bring up the service:
$ kubectl create -f k8s
replicationcontroller "kubediff" created
secret "kubediff-secret" created
service "kubediff" created
And to view the UI, run the follow command and go to http://localhost:4040
`$ kubectl port-forward $(kubectl get pod --selector=name=kubediff -o jsonpath={.items..metadata.name}) 4040:80`
This service exports the exit code of the kubediff as a Prometheus metric; a suitable alert can be setup for persistent differences:
ALERT Kubediff
IF max(command_exit_code{job="kubediff"}) != 0
FOR 2h
LABELS { severity="warning" }
ANNOTATIONS {
summary = "Kubediff has detected a difference in running config.",
description = "Kubediff has detected a difference in running config.",
}
These alerts can be sent to Slack, for example:
compare-images
To quickly see how two sets of configurations differ, purely in terms of images:
$ ./compare-images ../service-conf/k8s/dev/ ../service-conf/k8s/prod/
Image dev prod
----------------------------- -------------------- --------------------
quay.io/weaveworks/grafana master-0fc7cc2 master-08fd09d
quay.io/weaveworks/prometheus master-0fc7cc2 master-4fb2aed
quay.io/weaveworks/ui-server master-2899c36 master-45d67b3
tomwilkie/prometheus frankenstein-8a5ec1b frankenstein-ebe5808
weaveworks/scope master-1a1021c master-14d0e4e
Build
mkdir -p $GOPATH/src/github.com/prometheus && cd "$_"
git clone [email protected]:prometheus/client_golang.git
mkdir -p $GOPATH/src/github.com/weaveworks && cd "$_"
git clone [email protected]:weaveworks/kubediff.git
cd kubediff
make
Getting Help
If you have any questions about, feedback for or problems with kubediff:
- Invite yourself to the Weave Users Slack.
- Ask a question on the #general slack channel.
- File an issue.
Weaveworks follows the CNCF Code of Conduct. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting a Weaveworks project maintainer, or Alexis Richardson ([email protected]).
Your feedback is always welcome!