• Stars
    star
    253
  • Rank 160,776 (Top 4 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created almost 7 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Declarative way of managing machines for Kubernetes cluster

machine-controller-manager

CI Build status Go Report Card

Note One can add support for a new cloud provider by following Adding support for new provider.

Overview

Machine Controller Manager aka MCM is a group of cooperative controllers that manage the lifecycle of the worker machines. It is inspired by the design of Kube Controller Manager in which various sub controllers manage their respective Kubernetes Clients. MCM gives you the following benefits:

  • seamlessly manage machines/nodes with a declarative API (of course, across different cloud providers)
  • integrate generically with the cluster autoscaler
  • plugin with tools such as the node-problem-detector
  • transport the immutability design principle to machine/nodes
  • implement e.g. rolling upgrades of machines/nodes

MCM supports following providers. These provider code is maintained externally (out-of-tree), and the links for the same are linked below:

It can easily be extended to support other cloud providers as well.

Example of managing machine:

kubectl create/get/delete machine vm1

Key terminologies

Nodes/Machines/VMs are different terminologies used to represent similar things. We use these terms in the following way

  1. VM: A virtual machine running on any cloud provider. It could also refer to a physical machine (PM) in case of a bare metal setup.
  2. Node: Native kubernetes node objects. The objects you get to see when you do a "kubectl get nodes". Although nodes can be either physical/virtual machines, for the purposes of our discussions it refers to a VM.
  3. Machine: A VM that is provisioned/managed by the Machine Controller Manager.

Design of Machine Controller Manager

The design of the Machine Controller Manager is influenced by the Kube Controller Manager, where-in multiple sub-controllers are used to manage the Kubernetes clients.

Design Principles

It's designed to run in the master plane of a Kubernetes cluster. It follows the best principles and practices of writing controllers, including, but not limited to:

  • Reusing code from kube-controller-manager
  • leader election to allow HA deployments of the controller
  • workqueues and multiple thread-workers
  • SharedInformers that limit to minimum network calls, de-serialization and provide helpful create/update/delete events for resources
  • rate-limiting to allow back-off in case of network outages and general instability of other cluster components
  • sending events to respected resources for easy debugging and overview
  • Prometheus metrics, health and (optional) profiling endpoints

Objects of Machine Controller Manager

Machine Controller Manager reconciles a set of Custom Resources namely MachineDeployment, MachineSet and Machines which are managed & monitored by their controllers MachineDeployment Controller, MachineSet Controller, Machine Controller respectively along with another cooperative controller called the Safety Controller.

Machine Controller Manager makes use of 4 CRD objects and 1 Kubernetes secret object to manage machines. They are as follows:

Custom ResourceObject Description
MachineClass A MachineClass represents a template that contains cloud provider specific details used to create machines.
Machine A Machine represents a VM which is backed by the cloud provider.
MachineSet A MachineSet ensures that the specified number of Machine replicas are running at a given point of time.
MachineDeployment A MachineDeployment provides a declarative update for MachineSet and Machines.
Secret A Secret here is a Kubernetes secret that stores cloudconfig (initialization scripts used to create VMs) and cloud specific credentials.

See here for CRD API Documentation

Components of Machine Controller Manager

Controller Description
MachineDeployment controller Machine Deployment controller reconciles the MachineDeployment objects and manages the lifecycle of MachineSet objects. MachineDeployment consumes provider specific MachineClass in its spec.template.spec which is the template of the VM spec that would be spawned on the cloud by MCM.
MachineSet controller MachineSet controller reconciles the MachineSet objects and manages the lifecycle of Machine objects.
Safety controller There is a Safety Controller responsible for handling the unidentified or unknown behaviours from the cloud providers. Safety Controller:
  • freezes the MachineDeployment controller and MachineSet controller if the number of Machine objects goes beyond a certain threshold on top of Spec.replicas. It can be configured by the flag --safety-up or --safety-down and also --machine-safety-overshooting-period`.
  • freezes the functionality of the MCM if either of the target-apiserver or the control-apiserver is not reachable.
  • unfreezes the MCM automatically once situation is resolved to normal. A freeze label is applied on MachineDeployment/MachineSet to enforce the freeze condition.

Along with the above Custom Controllers and Resources, MCM requires the MachineClass to use K8s Secret that stores cloudconfig (initialization scripts used to create VMs) and cloud specific credentials. All these controllers work in an co-operative manner. They form a parent-child relationship with MachineDeployment Controller being the grandparent, MachineSet Controller being the parent, and Machine Controller being the child.

Development

To start using or developing the Machine Controller Manager, see the documentation in the /docs repository, please find the index here.

FAQ

An FAQ is available here

Cluster-api Implementation

More Repositories

1

gardener

Homogeneous Kubernetes clusters at scale on any infrastructure using hosted control planes.
Go
2,869
star
2

etcd-backup-restore

Collection of components to backup and restore the etcd of a Kubernetes cluster.
Go
285
star
3

dashboard

Web-based GUI for Gardener installations.
JavaScript
209
star
4

terraformer

Executes Terraform configuration as job/pod inside a Kubernetes cluster.
Go
133
star
5

external-dns-management

Environment to manage external DNS entries for a kubernetes cluster
Go
80
star
6

garden-setup

Describes Gardener components for installation of a Gardener landscape using sow
Shell
78
star
7

etcd-druid

An etcd operator to configure, provision, reconcile and monitor etcd clusters.
Go
70
star
8

landscaper

Development of Landscaper - A deployer for K8S workloads with integrated data flow engine.
Go
56
star
9

oidc-webhook-authenticator

Kubernetes Webhook Authenticator that allows for dynamic registration of OpenID Connect providers
Go
47
star
10

documentation

Documentation and website
Shell
34
star
11

cert-management

Manages TLS certificates in Kubernetes clusters using custom resources
Go
32
star
12

hvpa-controller

Controller for weight based horizontal and vertical scaling
Go
31
star
13

gardener-extension-provider-openstack

Gardener extension controller for the OpenStack cloud provider (https://openstack.org).
Go
25
star
14

gardenctl-v2

Command-line client for the Gardener with focus on operations
Go
24
star
15

cc-utils

CI/CD utils for gardener project
Python
21
star
16

gardener-extension-provider-aws

Gardener extension controller for the AWS cloud provider (https://aws.amazon.com).
Go
18
star
17

hyperkube

Shell
17
star
18

logging

Components needed for Gardener logging
Go
15
star
19

ops-toolbelt

Useful tools and operations guide for gardener landscapes
Shell
14
star
20

docforge

Scalable build tool for distributed documentation sources
Go
13
star
21

gardener-extension-networking-cilium

Gardener extension controller for the Cilium CNI network plugin.
Go
13
star
22

gardener-extension-provider-gcp

Gardener extension controller for the GCP cloud provider (https://cloud.google.com).
Go
13
star
23

apiserver-proxy

SNI Passthrough proxy for kube-apiservers
Shell
13
star
24

kupid

Inject scheduling criteria into target pods orthogonally by policy definition.
Go
11
star
25

test-infra

Test machinery for orchestration of integration/e2e/smoke style tests
Go
11
star
26

sow

A tiny installation tool based on spiff and plugins
Shell
11
star
27

controller-manager-library

Library easing implementation of kubernetes controllers
Go
11
star
28

gardener-extension-provider-alicloud

Gardener extension controller for the Alibaba cloud provider (https://alibabacloud.com).
Go
10
star
29

website-generator

Tools to generate the public facing gardener website
SCSS
10
star
30

gardenlogin

kubectl credential plugin for shoot cluster admin authentication
Go
10
star
31

gardener-extension-provider-azure

Gardener extension controller for the Azure cloud provider (https://azure.microsoft.com).
Go
10
star
32

gardener-extension-shoot-cert-service

Gardener extension controller for certificate services for shoot clusters.
Go
10
star
33

gardener-extension-os-gardenlinux

Gardener extension controller for the Garden Linux operating system
Go
9
star
34

gardener-extension-provider-equinix-metal

Gardener extension controller for the Equinix Metal cloud provider (https://equinixmetal.com/).
Go
8
star
35

gardener-extension-provider-vsphere

Gardener extension controller for the vSphere cloud provider (https://www.vmware.com).
Go
8
star
36

gardener-extension-networking-calico

Gardener extension controller for the Calico CNI network plugin.
Go
8
star
37

machine-controller-manager-provider-aws

Gardener machine controller manager provider for AWS
Go
8
star
38

gardener-extension-shoot-dns-service

Gardener extension controller for DNS services for shoot clusters.
Go
8
star
39

homebrew-tap

Provides Gardener tools via Homebrew package
Ruby
7
star
40

gardener-extension-registry-cache

Gardener extension controller which deploys pull-through caches for container registries.
Go
7
star
41

diki

Go
7
star
42

ci-infra

Test infrastructure for the Gardener project.
Go
6
star
43

machine-controller-manager-provider-azure

This repository is the out of tree implementation of the machine driver for Azure cloud provider
Go
6
star
44

service-account-issuer-discovery

A simple server that exposes the OpenID discovery documents of a Kubernetes cluster.
Go
6
star
45

landscapercli

Go
6
star
46

etcd-wrapper

Configures and starts an embedded ETCD
Go
5
star
47

chaos-engineering

Chaos engineering tools for Gardener-managed clusters
Python
5
star
48

machine-controller-manager-provider-vsphere

Gardener machine controller manager provider for VMware vSphere
Go
5
star
49

gardener-extension-os-suse-chost

Gardener extension controller for the SUSE Container Host operating system (CHost).
Go
5
star
50

gardener-extension-shoot-oidc-service

Gardener extension controller for OpenID Connect services for shoot clusters.
Go
5
star
51

vpn2

Network connector between the control plane (deployed in a Seed cluster) and a Shoot cluster superseding the vpn repository.
Go
5
star
52

machine-controller-manager-provider-equinix-metal

Out of tree (controller based) implementation for `Equinix Metal` as provider.
Go
5
star
53

remedy-controller

Remedy controller for automatic application of remedies for platform issues
Go
4
star
54

gardener-metrics-exporter

A Prometheus exporter for Gardener service-level metrics.
Go
4
star
55

landscaper-examples

Landscaper Examples
Shell
4
star
56

network-problem-detector

Network problem detector agent and cli
Go
4
star
57

machine-controller-manager-provider-openstack

Out of tree implementation for machine-controller-manager's provider-openstack
Go
4
star
58

terminal-controller-manager

Used for the webterminal feature of the gardener/dashboard
Go
4
star
59

gardener-extension-os-coreos

Gardener extension controller for the CoreOS/FlatCar Container Linux operating system.
Go
4
star
60

gardener-extension-runtime-gvisor

Gardener extension controller for the gVisor container runtime sandbox (https://gvisor.dev).
Go
4
star
61

machine-controller-manager-provider-alicloud

Gardener machine controller manager provider for Alicloud
Go
4
star
62

landscaper-service

A service managing multi-tenant landscaper installations.
Go
4
star
63

dependency-watchdog

This controller checks the status of etcd and restarts control plane components which are in a state of crashloop-backoff over an extensive period of time.
Go
4
star
64

gardener-extension-shoot-lakom-service

A k8s admission controller verifying pods are using signed images (cosign signatures) and a gardener extension to install it for shoots and seeds.
Go
3
star
65

machine-controller-manager-provider-sampleprovider

Out of tree implementation for SampleProvider as a new cloud provider
Go
3
star
66

machine-controller-manager-provider-gcp

Gardener machine controller manager provider for GCP
Go
3
star
67

etcd-custom-image

Custom etcd container image
Shell
3
star
68

k8syncer

Syncing k8s resources from the cluster into some kind of storage.
Go
3
star
69

gardener-extension-shoot-networking-problemdetector

Gardener extension for deploying network problem detector
Go
3
star
70

gardener-extension-shoot-networking-filter

Gardener extension controller for networking policy filter.
Go
2
star
71

gardener-extension-os-ubuntu

Gardener extension controller for the Ubuntu operating system.
Go
2
star
72

website

2
star
73

alpine-conntrack

Alpine image with pre-installed conntrack tools
Dockerfile
2
star
74

ext-authz-server

External authorization server for envoy.
Shell
2
star
75

oidc-apps-controller

A kubernetes controller enabling OIDC authentication and RBAC authorisation proxies to target workloads.
Go
2
star
76

chocolatey-packages

This repository contains chocolatey packages for tools from `github.com/gardener`.
PowerShell
1
star
77

monitoring

Components needed for Gardener monitoring
1
star
78

aws-custom-route-controller

Custom route controller for AWS
Go
1
star
79

aws-ipam-controller

AWS ipam controller for kubernetes
Go
1
star
80

etcd-druid-api

External APIs to manage an etcd cluster and its individual members
Go
1
star
81

falco-event-ingestor

Receive, validate, and store events from Falco deployed by Gardener.
Go
1
star
82

quic-reverse-http-tunnel

A reverse HTTP Tunnel using QUIC protocol
Go
1
star
83

kube-rbac-proxy-watcher

A configuration watcher managing the lifecycle of a child process upon changes on the watched resources.
Go
1
star
84

ingress-default-backend

Default ingress backend for Shoot clusters.
Shell
1
star