• Stars
    star
    2,519
  • Rank 17,703 (Top 0.4 %)
  • Language HCL
  • Created over 2 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Ampernetacle

This is a Terraform configuration to deploy a Kubernetes cluster on Oracle Cloud Infrastructure. It creates a few virtual machines and uses kubeadm to install a Kubernetes control plane on the first machine, and join the other machines as worker nodes.

By default, it deploys a 4-node cluster using ARM machines. Each machine has 1 OCPU and 6 GB of RAM, which means that the cluster fits within Oracle's (pretty generous if you ask me) free tier.

It is not meant to run production workloads, but it's great if you want to learn Kubernetes with a "real" cluster (i.e. a cluster with multiple nodes) without breaking the bank, and if you want to develop or test applications on ARM.

Getting started

  1. Create an Oracle Cloud Infrastructure account (just follow this link).
  2. Have installed or install kubernetes.
  3. Have installed or install terraform.
  4. Have installed or install OCI CLI .
  5. Configure OCI credentials. If you obtain a session token (with oci session authenticate), make sure to put the correct region, and when prompted for the profile name, enter DEFAULT so that Terraform finds the session token automatically.
  6. Download this project and enter its folder.
  7. terraform init
  8. terraform apply

That's it!

At the end of the terraform apply, a kubeconfig file is generated in this directory. To use your new cluster, you can do:

Linux

export KUBECONFIG=$PWD/kubeconfig
kubectl get nodes

Windows

$env:KUBECONFIG="$pwd\kubeconfig"
kubectl get nodes

The command above should show you 4 nodes, named node1 to node4.

You can also log into the VMs. At the end of the Terraform output you should see a command that you can use to SSH into the first VM (just copy-paste the command).

Windows

It works with Windows 10/Powershell 5.1.

It may be necesssary to change the execution policy to unrestricted.

PowerShell ExecutionPolicy

Customization

Check variables.tf to see tweakable parameters. You can change the number of nodes, the size of the nodes, or switch to Intel/AMD instances if you'd like. Keep in mind that if you switch to Intel/AMD instances, you won't get advantage of the free tier.

Stopping the cluster

terraform destroy

Implementation details

This Terraform configuration:

  • generates an OpenSSH keypair and a kubeadm token
  • deploys 4 VMs using Ubuntu 20.04
  • uses cloud-init to install and configure everything
  • installs Docker and Kubernetes packages
  • runs kubeadm init on the first VM
  • runs kubeadm join on the other VMs
  • installs the Weave CNI plugin
  • transfers the kubeconfig file generated by kubeadm
  • patches that file to use the public IP address of the machine

Caveats

This doesn't install the OCI cloud controller manager, which means that you cannot create services with type: LoadBalancer; or rather, if you create such services, their EXTERNAL-IP will remain <pending>.

To expose services, use NodePort.

Likewise, there is no ingress controller and no storage class.

These might be added in a later iteration of this project. Meanwhile, if you want to install it manually, you can check the OCI cloud controller manager github repository.

Remarks

Oracle Cloud also has a managed Kubernetes service called Container Engine for Kubernetes (or OKE). That service doesn't have the caveats mentioned above; however, it's not part of the free tier.

What does "Ampernetacle" mean?

It's a porte-manteau between Ampere, Kubernetes, and Oracle. It's probably not the best name in the world but it's the one we have! If you have an idea for a better name let us know. 😊

Possible errors and how to address them

Authentication problem

If you configured OCI authentication using a session token (with oci session authenticate), please note that this token is valid 1 hour by default. If you authenticate, then wait more than 1 hour, then try to terraform apply, you will get authentication errors.

Symptom

The following message:

 Error: 401-NotAuthenticated
│ Service: Identity Compartment
│ Error Message: The required information to complete authentication was not provided or was incorrect.
│ OPC request ID: [...]
│ Suggestion: Please retry or contact support for help with service: Identity Compartment

Solution

Authenticate or re-authenticate, for instance with oci session authenticate.

If prompted for the profile name, make sure to enter DEFAULT so that Terraform automatically uses the session token.

If you previously used oci session authenticate, you should be able to refresh the session with oci session refresh --profile DEFAULT.

Capacity issue

Symptom

If you get a message like the following one:

Error: 500-InternalError
│ ...
│ Service: Core Instance
│ Error Message: Out of host capacity.

It means that there isn't enough servers available at the moment on OCI to create the cluster.

Solution

One solution is to switch to a different availability domain. This can be done by changing the availability_domain input variable. (Thanks @uknbr for the contribution!)

Note 1: some regions have only one availability domain. In that case you cannot change the availability domain.

Note 2: OCI accounts (especially free accounts) are tied to a single region, so if you get that problem and cannot change the availability domain, you can create another account.

Using the wrong region

Symptom

When doing terraform apply, you get this message:

oci_identity_compartment._: Creating...
╷
│ Error: 404-NotAuthorizedOrNotFound
│ Service: Identity Compartment
│ Error Message: Authorization failed or requested resource not found
│ OPC request ID: [...]
│ Suggestion: Either the resource has been deleted or service Identity Compartment need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│
│
│   with oci_identity_compartment._,
│   on main.tf line 1, in resource "oci_identity_compartment" "_":
│    1: resource "oci_identity_compartment" "_" {
│
╵

Solution

Edit ~/.oci/config and change the region= line to put the correct region.

To know what's the correct region, you can try to log in to https://cloud.oracle.com/ with your account; after logging in, you should be redirected to an URL that looks like https://cloud.oracle.com/?region=us-ashburn-1 and in that example the region is us-ashburn-1.

Troubleshooting cluster creation

After the VMs are created, you can log into the VMs with the ubuntu user and the SSH key contained in the id_rsa file that was created by Terraform.

Then you can check the cloud init output file, e.g. like this:

tail -n 100 -f /var/log/cloud-init-output.log

More Repositories

1

pipework

Software-Defined Networking tools for LXC (LinuX Containers)
Shell
4,139
star
2

container.training

Slides and code samples for training, tutorials, and workshops about Docker, containers, and Kubernetes.
Shell
3,559
star
3

nsenter

Shell
2,582
star
4

dind

Docker in Docker
Shell
2,474
star
5

dockvpn

Recipe to build an OpenVPN image for Docker
Shell
833
star
6

squid-in-a-can

Python
357
star
7

pxe

Dockerfile to build a PXE server in a Docker container
Shell
250
star
8

minimage

Minimal Docker images: a collection of Dockerfiles illustrating how to reduce container image size.
Shell
206
star
9

griode

Griode + Novation Launchpad + Raspberry Pi = a music instrument!
Python
139
star
10

registrish

Dirty hack to run a read-only, public Docker registry on almost any static file hosting service (e.g. NGINX, Netlify, S3...)
Shell
136
star
11

shpod

Container image to get a consistent training environment to work on Kubernetes.
Dockerfile
134
star
12

critmux

Docker + CRIU + tmux = magic!
Dockerfile
117
star
13

dockercoins

Python
98
star
14

docker-busybox

Busybox for Stackbrew
Shell
94
star
15

sekexe

Separate Kernel Execution: execute a process within user-mode-linux and retrieve its output and status code
Shell
79
star
16

dessine-moi-un-cluster

Instructions to build a Kubernetes control plane one piece at a time, for learning purposes.
Shell
78
star
17

gunsub

Get your github notifications under control!
Python
74
star
18

syslogdocker

70
star
19

stevedore

Containerize your development environments
Shell
68
star
20

hamba

Shell
67
star
21

obs-docker

OBS-Studio (and a few extra tools) in containers
Python
62
star
22

docker2docker

Shell
26
star
23

wordsmith

Java
23
star
24

intro-to-docker

CSS
23
star
25

jpetazzo.github.io

HTML
22
star
26

go-docker-

20
star
27

trainingwheels

HTML
18
star
28

snakedeck

Elgato StreamDeck controller for Linux, in Python.
Python
17
star
29

nsplease

Tiny Kubernetes operator to create Namespaces on demand (for CI/CD, for instance)
Shell
16
star
30

django

Django on DotCloud tutorial
Python
16
star
31

whisperfiles

A bunch of Dockerfiles for OpenAI Whisper, to illustrate various image optimization techniques
Shell
15
star
32

foundation-example

Shell
14
star
33

httpenv

Tiny HTTP server showing the environment variables
Go
14
star
34

buildkit-demos

Dockerfile
13
star
35

dctrl

Shell
13
star
36

orchestration-workshop

We have moved! We are now at → https://github.com/jpetazzo/container.training
HTML
12
star
37

decoup

Python
11
star
38

layeremove

Surgically remove layers from a Docker image (with a chainsaw)
Python
11
star
39

znc-on-dotcloud

Shell
11
star
40

tilestream-on-dotcloud

Python
10
star
41

littleboxes

Just for fun scripts to manage local cloud-like VMs with KVM
Shell
10
star
42

django-and-mongodb-on-dotcloud

Django on DotCloud tutorial, using MongoDB to store objects!
Python
10
star
43

kubercoins

8
star
44

sstk

Shell
8
star
45

geodjango-on-dotcloud

Python
7
star
46

meteor-on-dotcloud

7
star
47

solr-on-dotcloud

JavaScript
7
star
48

seleterm

Selenium for terminal applications
Python
6
star
49

snap-on-dotcloud

Shell
6
star
50

httplat

Minimalist Prometheus exporter to collect the latency of an HTTP target
Go
6
star
51

boggle

Solver for the Boggle Word Game
Python
6
star
52

mume

Python
6
star
53

gitorial

(Ab)use git history to write tutorials!
Python
6
star
54

postgresql-on-dotcloud

Python
5
star
55

django-on-gpaas

Django on GANDI PAAS
Python
5
star
56

scangraph

Retrieve point coordinates from a raster plot
JavaScript
5
star
57

hano

Online IDE for Node.js on dotCloud
Shell
5
star
58

pyramid-on-dotcloud

Python
5
star
59

traefik-compose

Quick demo showing how to run web sites (like Wordpress) on Docker with Traefik
4
star
60

plumber

Shell
4
star
61

consul

jpetazzo's Consul image
Shell
4
star
62

zwave-exporter

Prometheus exporter for Z-Wave sensors
JavaScript
4
star
63

jenkins-on-dotcloud

Shell
4
star
64

color

Go
4
star
65

busyhttp

A trivial HTTP server that eats CPU cycles at each request.
Python
4
star
66

tinydocgen

Tiny document generator using Jinja2, Markdown, and WeasyPrint.
Makefile
3
star
67

ngrok

3
star
68

charliebot

Python
3
star
69

prettypictures

3
star
70

usb-webcam-analyzer

Python
3
star
71

rickroll-in-docker

HTML
3
star
72

dnc

Domain Name Command-line tool
Python
3
star
73

python-simple-logging

Python
3
star
74

replay.container.training

Shell
3
star
75

riak-on-dotcloud-ALPHA

Shell
3
star
76

ucengine-on-dotcloud-ALPHA

Shell
3
star
77

django-r2d2

R2D2 (RRDDashboard) is a Django application to draw graphs from metrics coming from e.g. collectd.
Python
3
star
78

pieuvre

Distributed HTTP proxy in Node.js
JavaScript
2
star
79

tmp-sealedsecret-juin-2022

2
star
80

couchdb-on-dotcloud-ALPHA

Shell
2
star
81

escapehash

Python
2
star
82

dockerhubratelimit

Shell
2
star
83

workflows

2
star
84

python-worker-on-dotcloud

Shell
2
star
85

tcl-on-dotcloud-ALPHA

Shell
2
star
86

elastic-gke

HCL
2
star
87

dampmam

Docker-Apache-MySQL-PHP but without Apache and MySQL
JavaScript
2
star
88

watchdns

Shell
2
star
89

highfive

Dockerfile
2
star
90

memcached-on-dotcloud

2
star
91

pawd

PulseAudio Web Daemon
2
star
92

pingr

HTTP server to ping other servers and report their status
Go
2
star
93

dotfiles-old

Config files for various environments
Shell
2
star
94

pgpool-II-on-dotcloud

2
star
95

ls

An ls image for the Docker Fundamentals training
Shell
2
star
96

jetty-on-dotcloud

Reimplementation of dotCloud java service using the custom build API
Shell
2
star
97

dockage

Shell
2
star
98

jira-on-dotcloud

Shell
2
star
99

tornado-on-dotcloud

Python
2
star
100

aiguillage

Nginx
2
star