istio/installer istio/installer

  • This repository has been archived on 08/Oct/2020
  • Stars
    star
    112
  • Rank 312,240 (Top 7 %)
  • Language Smarty
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated over 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
istio/installer
github

istio

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A modular, a-la-carte installer for Istio components. MOVED to https://github.com/istio/istio/tree/master/manifests

Istio Installer

Istio installer is a modular, 'a-la-carte' installer for Istio. It is based on a fork of the Istio helm templates, refactored to increase modularity and isolation.

Goals:

  • Improve upgrade experience: users should be able to gradually roll upgrades, with proper canary deployments for Istio components. It should be possible to deploy a new version while keeping the stable version in place and gradually migrate apps to the new version.

  • More flexibility: the new installer allows multiple 'environments', allowing applications to select a set of control plane settings and components. While the entire mesh respects the same APIs and config, apps may target different 'environments' which contain different instances and variants of Istio.

  • Better security: separate Istio components reside in different namespaces, allowing different teams or roles to manage different parts of Istio. For example, a security team would maintain the root CA and policy, a telemetry team may only have access to Mixer-telemetry and Prometheus, and a different team may maintain the control plane components (which are highly security sensitive).

The install is organized in 'environments' - each environment consists of a set of components in different namespaces that are configured to work together. Regardless of 'environment', workloads can talk with each other and obey the Istio configuration resources, but each environment can use different Istio versions and different configuration defaults.

istioctl kube-inject or the automatic sidecar injector are used to select the environment. In the case of the sidecar injector, the namespace label istio-env: <NAME_OF_ENV> is used instead of the conventional istio-injected: true. The name of the environment is defined as the namespace where the corresponding control plane components (config, discovery, auto-injection) are running. In the examples below, by default this is the istio-control namespace. Pod annotations can also be used to select a different 'environment'.

Installing

The new installer is intended to be modular and very explicit about what is installed. It has far more steps than the Istio installer - but each step is smaller and focused on a specific feature, and can be performed by different people/teams at different times.

It is strongly recommended that different namespaces are used, with different service accounts. In particular access to the security-critical production components (root CA, policy, control) should be locked down and restricted. The new installer allows multiple instances of policy/control/telemetry - so testing/staging of new settings and versions can be performed by a different role than the prod version.

The intended users of this repo are users running Istio in production who want to select, tune and understand each binary that gets deployed, and select which combination to use.

Note: each component can be installed in parallel with an existing Istio 1.0 or 1.1 install in istio-system. The new components will not interfere with existing apps, but can interoperate and it is possible to gradually move apps from Istio 1.0/1.1 to the new environments and across environments ( for example canary -> prod )

Note: there are still some cluster roles that may need to be fixed, most likely cluster permissions will need to move to the security component.

Everything is Optional

Each component in the new installer is optional. Users can install the component defined in the new installer, use the equivalent component in istio-system, configured with the official installer, or use a different version or implementation.

For example you may use your own Prometheus and Grafana installs, or you may use a specialized/custom certificate provisioning tool, or use components that are centrally managed and running in a different cluster.

This is a work in progress - building on top of the multi-cluster installer.

As an extreme, the goal is to be possible to run Istio workloads in a cluster without installing any Istio component in that cluster. Currently the minimum we require is the security provider (node agent or citadel).

Namespaces

The new installer recommends isolating components in different namespaces with different service accounts and access.

Recommended mode:

Singleton:

  • istio-system: root CA and cert provisioning components.
  • istio-cni: optional CNI (avoids requiring root/netadmin from workload pods)

Multi-environment components:

  • istio-control: config, discovery, auto-inject. All impact the generated config including enforcement of policies and secure naming.
  • istio-telemetry: mixer, kiali, tracing providers, grafana, prometheus. Custom install of prometheus, grafana can be used instead in dedicated namespaces.
  • istio-policy
  • istio-gateways: production domains should be in a separate namespace, to restrict access. It is possible to segregate gateways by the team that control access to the domain. Access to the gateway namespace provides access to certificates and control over domain delegation. The optional egress gateway provides control over outbound traffic.

In addition, it is recommended to have a second set of the multi-environment components to use for canary/testing new versions. In this doc we will use an environment based on the istio-master namespace:

  • istio-master: config, discovery, etc
  • istio-telemetry-master
  • istio-gateway-master
  • istio-policy-master ...

For each component, there are 2 styles of installing, using 'helm + tiller' or 'helm template + kubectl apply --prune'.

Using kubectl --prune is recommended:

helm template --namespace $NAMESPACE --name-template $COMPONENT $CONFIGDIR -f global.yaml | \
   kubectl apply -n $NAMESPACE --prune -l release=$COMPONENT -f -

Using helm:

helm upgrade --namespace $NAMESPACE $COMPONENT $CONFIGDIR -f global.yaml

The doc will use the iop $NAMESPACE $COMPONENT $CONFIGDIR helper from env.sh - which is the equivalent to the commands above.

In the instructions below, $IBASE refers to the working tree of this repo.

Common options

TODO: replicas, cpu allocs, etc.

Install Istio CRDs

This is the first step of the install. Please do not remove or edit any CRD - config currently requires all CRDs to be present. On each upgrade it is recommended to reapply the file, to make sure you get all CRDs. CRDs are separated by release and by component type in the CRD directory.

Istio has strong integration with certmanager. Some operators may want to keep their current certmanager CRDs in place and not have Istio modify them. In this case, it is necessary to apply CRD files individually.

kubectl apply -k github.com/istio/installer/base

or

kubectl apply -f base/files

Install Security

Security should be installed in istio-system, since it needs access to the root CA. For upgrades from the official installer, it is recommended to install the security component in istio-system, install the other components in different namespaces, migrate all workloads - and at the end uninstall the official installer, and lock down istio-system.

This is currently required if any mTLS is used. In future other Spifee implementations can be used, and it is possible to use other tools that create the expected certificates for Istio.

# use the default HUB=gcr.io/istio-testing, TAG=latest defined in global.yaml
iop istio-system citadel $IBASE/security/citadel

If you want to use a different HUB and TAG, the following example shows how to install istio 1.3.0's citadel:

export HUB=docker.io/istio TAG=1.3.0
iop istio-system citadel $IBASE/security/citadel

Important options: the dnsCerts list allows associating DNS certs with specific service accounts. This should be used if you plan to use Galley or Sidecar injector in different namespaces. By default it supports istio-control, istio-master namespaces used in the examples.

Access to the security namespace and istio-system should be highly restricted.

Install Istio-CNI

This is an optional step - CNI must run in a dedicated namespace, it is a 'singleton' and extremely security sensitive. Access to the CNI namespace must be highly restricted.

NOTE: The environment variable ISTIO_CLUSTER_ISGKE is assumed to be set to true if the cluster is a GKE cluster.

ISTIO_CNI_ARGS=
# TODO: What k8s data can we use for this check for whether GKE?
if [[ "${ISTIO_CLUSTER_ISGKE}" == "true" ]]; then
    ISTIO_CNI_ARGS="--set cni.cniBinDir=/home/kubernetes/bin"
fi
iop kube-system istio-cni $IBASE/istio-cni/ ${ISTIO_CNI_ARGS}

TODO. It is possible to add Istio-CNI later, and gradually migrate.

Install Control plane

The control plane contains 3 components.

Config (Galley)

This can be run in any other cluster having the CRDs configured via CI/CD systems or other sync mechanisms. It should not be run in 'secondary' clusters, where the configs are not replicated.

Galley provides config access and validation. Only one environment should enable validation - it is not currently supported in multiple namespaces.

iop istio-control istio-config $IBASE/istio-control/istio-config --set configValidation=true

# Second Galley, using master version of istio
TAG=latest HUB=gcr.io/istio-testing iop istio-master istio-config-master $IBASE/istio-control/istio-config

Other MCP providers can be used - currently the address and credentials need to match what galley is using.

Discovery, Policy and Telemetry components will need to be configured with the address of the config server - either in the local cluster or in a central cluster.

Discovery (Pilot)

This can run in any cluster. A mesh should have at least one cluster should run Pilot or equivalent XDS server, and it is recommended to have Pilot running in each region and in multiple availability zones for multi cluster.

iop istio-control istio-discovery $IBASE/istio-control/istio-discovery \
            --set global.istioNamespace=istio-system \
            --set global.configNamespace=istio-control \
            --set global.telemetryNamespace=istio-telemetry \
            --set global.policyNamespace=istio-policy

# Second istio-discovery, using master version of istio
TAG=latest HUB=gcr.io/istio-testing iop istio-master istio-discovery-master $IBASE/istio-control/istio-discovery \
            --set policy.enable=false \
            --set global.istioNamespace=istio-master \
            --set global.configNamespace=istio-master \
            --set global.telemetryNamespace=istio-telemetry-master \
            --set global.policyNamespace=istio-policy-master

Auto-injection

This is optional - istioctl kube-inject can be used instead.

If installed, namespaces can select the injector by setting the istio-env label on the namespace.

Only one auto-injector environment should have enableNamespacesByDefault=true, which will apply that environment to any namespace without an explicit istio-env label.

If istio-system has set enableNamespaceByDefault you must set istio-inject: disabled label to prevent istio-system from taking over. In this case, it is recommended to first install istio-control autoinject with the default disabled, test it, and move the default from istio-system to istio-control.

# ENABLE_CNI is set to true if istio-cni is installed
iop istio-control istio-autoinject $IBASE/istio-control/istio-autoinject \
        --set sidecarInjectorWebhook.enableNamespacesByDefault=true \
        --set global.configNamespace=istio-control \
        --set global.telemetryNamespace=istio-telemetry \
        --set istio_cni.enabled=${ENABLE_CNI}

# Second auto-inject using master version of istio
# Notice the different options
TAG=latest HUB=gcr.io/istio-testing iop istio-master istio-autoinject-master $IBASE/istio-control/istio-autoinject \
        --set global.telemetryNamespace=istio-telemetry-master \
        --set global.configNamespace=istio-master

Gateways

A cluster may use multiple Gateways, each with a different load balancer IP, domains and certificates.

Since the domain certificates are stored in the gateway namespace, it is recommended to keep each gateway in a dedicated namespace and restrict access.

For large-scale gateways it is optionally possible to use a dedicated pilot in the gateway namespace.

K8S Ingress

To support K8S ingress we currently use a separate namespace. In Istio 1.1, this requires using a dedicated Pilot instance in the ingress namespace. This will be fixed in future releases.

Note that running a dedicated Pilot for ingress/gateways is supported and recommended for very large sites, but in the case of K8S ingress it is currently required.

iop istio-ingress istio-ingress $IBASE/gateways/istio-ingress --set global.configNamespace=istio-control
TAG=latest HUB=gcr.io/istio-testing iop istio-ingress-master istio-ingress $IBASE/gateways/istio-ingress \
        --set global.configNamespace=istio-master\

Telemetry

iop istio-telemetry istio-grafana $IBASE/istio-telemetry/grafana/ --set global.configNamespace=istio-control
iop istio-telemetry istio-mixer $IBASE/istio-telemetry/mixer-telemetry/ \
        --set global.configNamespace=istio-control \
        --set global.istioNamespace=istio-system \
        --set global.telemetryNamespace=istio-telemetry \
        --set global.policyNamespace=istio-policy
iop istio-telemetry istio-prometheus $IBASE/istio-telemetry/prometheus/ \
        --set global.configNamespace=istio-control \
        --set global.istioNamespace=istio-system \
        --set global.telemetryNamespace=istio-telemetry \
        --set global.policyNamespace=istio-policy

TAG=latest HUB=gcr.io/istio-testing iop istio-telemetry-master istio-grafana $IBASE/istio-telemetry/grafana/ \
        --set global.configNamespace=istio-master
TAG=latest HUB=gcr.io/istio-testing iop istio-telemetry-master istio-mixer $IBASE/istio-telemetry/mixer-telemetry/ \
        --set global.configNamespace=istio-master \
        --set global.istioNamespace=istio-master \
        --set global.telemetryNamespace=istio-telemetry-master \
        --set global.policyNamespace=istio-policy-master
TAG=latest HUB=gcr.io/istio-testing iop istio-telemetry-master istio-prometheus $IBASE/istio-telemetry/prometheus/ \
        --set global.configNamespace=istio-master \
        --set global.istioNamespace=istio-master \
        --set global.telemetryNamespace=istio-telemetry-master \
        --set global.policyNamespace=istio-policy-master

Kiali

iop istio-telemetry kiali $IBASE/istio-telemetry/kiali \
        --set global.configNamespace=istio-control \
        --set global.istioNamespace=istio-system \
        --set global.telemetryNamespace=istio-telemetry \
        --set global.policyNamespace=istio-policy \
        --set global.prometheusNamespace=istio-telemetry

Additional test templates

A number of helm test setups are general-purpose and should be installable in any cluster, to confirm Istio works properly and allow testing the specific install.

More Repositories

1

istio

Connect, secure, control, and observe services.
Go
33,248
star
2

community

Istio governance material.
Go
2,596
star
3

istio.io

Source for the istio.io site
HTML
762
star
4

proxy

The Istio proxy components.
C++
724
star
5

api

API definitions for the Istio project
Shell
413
star
6

tools

Tools shared by several Istio repositories
Go
311
star
7

client-go

Go client libraries for the Istio API.
Makefile
241
star
8

operator

Istio operator provides user friendly options to operate the Istio service mesh
Go
173
star
9

ztunnel

An experimental implementation of the `ztunnel` component of ambient mesh
Rust
156
star
10

cni

Istio CNI to setup kubernetes pod namespaces to redirect traffic to sidecar proxy.
Go
140
star
11

old_pilot_repo

Deprecated home of Istio's Pilot, now in istio/istio's pilot dir
Go
137
star
12

old_auth_repo

Deprecated home of Istio authentication components, now in istio/istio's security dir
Go
73
star
13

test-infra

Go
73
star
14

old_mixer_repo

Deprecated home of Istio's Mixer and its adapters, now in istio/istio's mixer dir
Go
67
star
15

pkg

Common packages used by other repos.
Go
47
star
16

release-builder

Builds istio releases in a hermetic and reproducible environment
Go
43
star
17

old_issues_repo

Deprecated issue-tracking repo, please post new issues or feature requests to istio/istio instead.
37
star
18

glog

A replacement for the glog package that directs logging to zap.
Go
23
star
19

old_mixerclient_repo

Deprecated home of the C++ client library for Mixer's API, now in istio/proxy
C++
15
star
20

fortio-deployment

Deployment files for Fortio.
Makefile
15
star
21

enhancements

Enhancement Tracking and Backlog Repo for Istio Releases
14
star
22

common-files

Contains build files shared by many Istio repos.
Shell
11
star
23

bots

Source code for bots used by the Istio project.
Go
10
star
24

old_broker_repo

Deprecated home of Istio's Service Broker, now in istio/istio's broker dir
Go
8
star
25

get-istioctl

JavaScript
7
star
26

gogo-genproto

Houses generated source code for imported libraries used within Istio.
Shell
7
star
27

cri

Istio CRI
Shell
5
star
28

klog

A replacement for the klog package that directs logging to zap.
Go
3
star
29

old_vendor-istio_repo

(former) Vendored dependencies for the main istio repo
Go
2
star
30

bottestrepo

Used for testing the Istio bots
HTML
1
star
31

.github

1
star
32

.allstar

1
star
33

old_manifest_repo

1
star