• Stars
    star
    565
  • Rank 76,881 (Top 2 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created about 6 years ago
  • Updated 11 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Container Storage Interface (CSI) Driver for DigitalOcean Block Storage

csi-digitalocean

A Container Storage Interface (CSI) Driver for DigitalOcean Block Storage. The CSI plugin allows you to use DigitalOcean Block Storage with your preferred Container Orchestrator.

The DigitalOcean CSI plugin is mostly tested on Kubernetes. Theoretically it should also work on other Container Orchestrators, such as Mesos or Cloud Foundry. Feel free to test it on other CO's and give us a feedback.

Releases

The DigitalOcean CSI plugin follows semantic versioning. The version will be bumped following the rules below:

  • Bug fixes will be released as a PATCH update.
  • New features (such as CSI spec bumps with no breaking changes) will be released as a MINOR update.
  • Significant breaking changes makes a MAJOR update.

Features

Below is a list of functionality implemented by the plugin. In general, CSI features implementing an aspect of the specification are available on any DigitalOcean Kubernetes version for which beta support for the feature is provided.

See also the project examples for use cases.

Volume Expansion

Volumes can be expanded by updating the storage request value of the corresponding PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc
  namespace: default
spec:
  [...]
  resources:
    requests:
      # The field below can be increased.
      storage: 10Gi
      [...]

After successful expansion, the status section of the PVC object will reflect the actual volume capacity.

Important notes:

  • Volumes can only be increased in size, not decreased; attempts to do so will lead to an error.
  • Expanding a volume that is larger than the target size will have no effect. The PVC object status section will continue to represent the actual volume capacity.
  • Resizing volumes other than through the PVC object (e.g., the DigitalOcean cloud control panel) is not recommended as this can potentially cause conflicts. Additionally, size updates will not be reflected in the PVC object status section immediately, and the section will eventually show the actual volume capacity.

Raw Block Volume

Volumes can be used in raw block device mode by setting the volumeMode on the corresponding PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc
  namespace: default
spec:
  [...]
  volumeMode: Block

Important notes:

  • If using volume expansion functionality, only expansion of the underlying persistent volume is guaranteed. We do not guarantee to automatically expand the filesystem if you have formatted the device.

Volume Snapshots

Snapshots can be created and restored through VolumeSnapshot objects.

Note:

Version 1 of the CSI driver supports v1alpha1 Volume Snapshots only.

Version 2 and 3 of the CSI driver supports v1beta1 Volume Snapshots only.

Version 4 and later of the CSI driver support v1 Volume Snapshots only, which is backwards compatible to v1beta1. However, version 3 renders snapshots unusable that had previously been marked as invalid. See the csi-snapshotter documentation on the validating webhook and v1beta1 to v1 upgrade notes.


See also the example.

Volume Statistics

Volume statistics are exposed through the CSI-conformant endpoints. Monitoring systems such as Prometheus can scrape metrics and provide insights into volume usage.

Volume Transfer

Volumes can be transferred across clusters. The exact steps are outlined in our example.

Installing to Kubernetes

Kubernetes Compatibility

The following table describes the required DigitalOcean CSI driver version per supported Kubernetes release.

Kubernetes Release DigitalOcean CSI Driver Version
1.19 v3
1.20 v3
1.21 v3
1.22 v4
1.23 v4.2.0+
1.24 v4.3.0+
1.25 v4.4.0+
1.26 v4.5.0+
1.27 v4.6.0+

Note:

The DigitalOcean Kubernetes product comes with the CSI driver pre-installed and no further steps are required.


Driver modes

By default, the driver supports both the controller and node mode. It can manage DigitalOcean Volumes via the cloud API and mount them on the required node. The actually used mode is determined by how the driver is deployed and configured. The suggested release manifests provide separate deployments for controller and node modes, respectively.

When running outside of DigitalOcean droplets, the driver can only function in controller mode. This requires to set the --region flag to a valid DigitalOcean region slug in addition to the other flags.

The --region flag must not be set when running the driver on DigitalOcean droplets.

Alternatively driver can be run in node only mode on DigitalOcean droplets. Driver would only handle node related requests like mount volume. Driver runs in node only mode when --token flag is not provided.

Skip secret creation (section 1. in following deployment instructions) when using node only mode as API token is not required.

Modes --token flag --region flag
Controller and Node mode in DigitalOcean
Controller only mode not in DigitalOcean
Node only mode in DigitalOcean

Requirements

  • --allow-privileged flag must be set to true for the API server
  • --allow-privileged flag must be set to true for the kubelet in Kubernetes 1.14 and below (flag does not exist in later releases)
  • --feature-gates=KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true feature gate flags must be set to true for both the API server and the kubelet
  • Mount Propagation needs to be enabled. If you use Docker, the Docker daemon of the cluster nodes must allow shared mounts.

1. Create a secret with your DigitalOcean API Access Token

Replace the placeholder string starting with a05... with your own secret and save it as secret.yml:

apiVersion: v1
kind: Secret
metadata:
  name: digitalocean
  namespace: kube-system
stringData:
  access-token: "a05dd2f26b9b9ac2asdas__REPLACE_ME____123cb5d1ec17513e06da"

and create the secret using kubectl:

$ kubectl create -f ./secret.yml
secret "digitalocean" created

You should now see the digitalocean secret in the kube-system namespace along with other secrets

$ kubectl -n kube-system get secrets
NAME                  TYPE                                  DATA      AGE
default-token-jskxx   kubernetes.io/service-account-token   3         18h
digitalocean          Opaque                                1         18h

2. Provide authentication data for the snapshot validation webhook

Snapshots are validated through a ValidatingWebhookConfiguration which requires proper CA, certificate, and key data. The manifests in snapshot-validation-webhook.yaml should provide sufficient scaffolding to inject the data accordingly. However, the details on how to create and manage them is up to the user and dependent on the exact environment the webhook runs in. See the XXX-marked comments in the manifests file for user-required injection points.

The official snapshot webhook example offers a non-production-ready solution suitable for testing. For full production readiness, something like cert-manager can be leveraged.

3. Deploy the CSI plugin and sidecars

Always use the latest release compatible with your Kubernetes release (see the compatibility information).

The releases directory holds manifests for all plugin releases. You can deploy a specific version by executing the command

# Do *not* add a blank space after -f
kubectl apply -fhttps://raw.githubusercontent.com/digitalocean/csi-digitalocean/master/deploy/kubernetes/releases/csi-digitalocean-vX.Y.Z/{crds.yaml,driver.yaml,snapshot-controller.yaml}

where vX.Y.Z is the plugin target version. (Note that for releases older than v2.0.0, the driver was contained in a single YAML file. If you'd like to deploy an older release you need to use kubectl apply -fhttps://raw.githubusercontent.com/digitalocean/csi-digitalocean/master/deploy/kubernetes/releases/csi-digitalocean-vX.Y.Z.yaml)

If you see any issues during the installation, this could be because the newly created CRDs haven't been established yet. If you call kubectl apply -f again on the same file, the missing resources will be applied again.

The above does not include the snapshot validating webhook which needs extra configuration as outlined above. You may append ,snapshot-validation-webhook.yaml to the {...} list if you want to install a (presumably configured) webhook as well.

4. Test and verify

Create a PersistentVolumeClaim. This makes sure a volume is created and provisioned on your behalf:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: do-block-storage

Check that a new PersistentVolume is created based on your claim:

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM             STORAGECLASS       REASON    AGE
pvc-0879b207-9558-11e8-b6b4-5218f75c62b9   5Gi        RWO            Delete           Bound     default/csi-pvc   do-block-storage             3m

The above output means that the CSI plugin successfully created (provisioned) a new Volume on behalf of you. You should be able to see this newly created volume under the Volumes tab in the DigitalOcean UI

The volume is not attached to any node yet. It'll only attached to a node if a workload (i.e: pod) is scheduled to a specific node. Now let us create a Pod that refers to the above volume. When the Pod is created, the volume will be attached, formatted and mounted to the specified Container:

kind: Pod
apiVersion: v1
metadata:
  name: my-csi-app
spec:
  containers:
    - name: my-frontend
      image: busybox
      volumeMounts:
      - mountPath: "/data"
        name: my-do-volume
      command: [ "sleep", "1000000" ]
  volumes:
    - name: my-do-volume
      persistentVolumeClaim:
        claimName: csi-pvc

Check if the pod is running successfully:

kubectl describe pods/my-csi-app

Write inside the app container:

$ kubectl exec -ti my-csi-app /bin/sh
/ # touch /data/hello-world
/ # exit
$ kubectl exec -ti my-csi-app /bin/sh
/ # ls /data
hello-world

Upgrading

When upgrading to a new Kubernetes minor version, you should upgrade the CSI driver to match. See the table above for which driver version is used with each Kubernetes version.

Special consideration is necessary when upgrading from Kubernetes 1.11 or earlier, which uses CSI driver version 0.2 or earlier. In these early releases, the driver name was com.digitalocean.csi.dobs, while in all subsequent releases it is dobs.csi.digitalocean.com. When upgrading, use the commandline flag --driver-name to force the new driver to use the old name. Failing to do so will cause any existing PVs to be unusable since the new driver will not manage them and the old driver is no longer running.

Configuration

Default volumes paging size

Some CSI driver operations require paging through the volumes returned from the DO Volumes API. By default, the page size is not defined and causes the DO API to choose a value as specified in the API reference. In the vast majority of cases, this should work fine. However, for accounts with a very large number of volumes, the API server-chosen default page size may be too small to return all volumes within the configured (sidecar-provided) timeout.

For that reason, the default page size can be customized by passing the --default-volumes-page-size flag a positive number.


Notes:

  1. The user is responsible for selecting a value below the maximum limit mandated by the DO API. Please see the API reference link above to see the current limit.
  2. The configured sidecar timeout values may need to be aligned with the chosen page size. In particular, csi-attacher invokes ListVolumes to periodically synchronize the API and cluster-local volume states; as such, its timeout must be large enough to account for the expected number of volumes in the given account and region.
  3. The default page size does not become effective if an explicit page size (more precisely, max entries in CSI spec speak) is passed to a given gRPC method.

API rate limiting

DO API usage is subject to certain rate limits. In order to protect against running out of quota for extremely heavy regular usage or pathological cases (e.g., bugs or API thrashing due to an interfering third-party controller), a custom rate limit can be configured via the --do-api-rate-limit flag. It accepts a float value, e.g., --do-api-rate-limit=3.5 to restrict API usage to 3.5 queries per second.

Flags

Name Description Default
--validate-attachment Validate if the attachment has fully completed before formatting/mounting the device false

The --validate-attachment options adds an additional validation which checks for the /sys/class/block/<device name>/device/state file content for the running status. When enabling this flag, it prevents a racing condition where the DOBS volumes aren't fully attached which can be misinterpreted by the CSI implementation causing a force format of the volume which results in data loss.


Development

Requirements:

  • Go at the version specified in .github/workflows/test.yaml
  • Docker (for building via the Makefile, post-unit testing, and publishing)

Dependencies are managed via Go modules.

PRs from the code-hosting repository are automatically unit- and end-to-end-tested in our CI (implemented by Github Actions). See the .github/workflows directory for details.

For every green build of the master branch, the container image digitalocean/do-csi-plugin:master is updated and pushed at the end of the CI run. This allows to test the latest commit easily.

Steps to run the tests manually are outlined below.

Unit Tests

To execute the unit tests locally, run:

make test

End-to-End Tests

To manually run the end-to-end tests, you need to build a container image for your change first and publish it to a registry. Repository owners can publish under digitalocean/do-csi-plugin:dev:

VERSION=dev make publish

If you do not have write permissions to digitalocean/do-csi-plugin on Docker Hub or are worried about conflicting usage of that tag, you can also publish under a different (presumably personal) organization:

DOCKER_REPO=johndoe VERSION=latest-feature make publish

This would yield the published container image johndoe/do-csi-plugin:latest-feature.

Assuming you have your DO API token assigned to the DIGITALOCEAN_ACCESS_TOKEN environment variable, you can then spin up a DOKS cluster on-the-fly and execute the upstream end-to-end tests for a given set of Kubernetes versions like this:

make test-e2e E2E_ARGS="-driver-image johndoe/do-csi-plugin:latest-feature 1.16 1.15 1.14"

See our documentation for an overview on how the end-to-end tests work as well as usage instructions.

Integration Tests

There is a set of custom integration tests which are mostly useful for Kubernetes pre-1.14 installations as these are not covered by the upstream end-to-end tests.

To run the integration tests on a DOKS cluster, follow the instructions.

Prepare CSI driver for a new Kubernetes minor version

  1. Review recently merged PRs and any in-progress / planned work to ensure any bugs scheduled for the release have been fixed and merged.
  2. Bump kubernetes dependency versions
  3. Support running e2e on new $MAJOR.$MINOR
    1. Since we only support three minor versions at a time. E2e tests for the oldest supported version can be removed.
  4. Verify e2e tests pass - see here about running tests locally
  5. Prepare for release
  6. Perform release

See e2e test README on how to run conformance tests locally.

Updating the Kubernetes dependencies

Run

make NEW_KUBERNETES_VERSION=X.Y.Z update-k8s

to update the Kubernetes dependencies to version X.Y.Z.

Note: Make sure to also add support to the e2e tests for the new kubernetes version, following these instructions.

Releasing

Releases may happen either for the latest minor version of the CSI driver maintained in the master branch, or an older minor version still maintained in one of the release-* branches. In this section, we will call that branch the release branch.

To release a new version vX.Y.Z, first check out the release branch and bump the version:

make NEW_VERSION=vX.Y.Z bump-version

This will create the set of files specific to a new release. Make sure everything looks good; in particular, ensure that the change log is up-to-date and is not missing any important, user-facing changes.

Create a new branch with all changes:

git checkout -b prepare-release-vX.Y.Z
git add .
git push origin

After it is merged to the release branch, wait for the release branch build to go green. (This will entail another run of the entire test suite.)

Finally, check out the release branch again, tag the release, and push it:

git checkout <release branch>
git pull
git tag vX.Y.Z
git push origin vX.Y.Z

(This works for non-master release branches as well since the checkout Github Action we use defaults to checking out the ref/SHA that triggered the workflow.)

The CI will publish the container image digitalocean/do-csi-plugin:vX.Y.Z and create a Github Release under the name vX.Y.Z automatically. Nothing else needs to be done.

Contributing

At DigitalOcean we value and love our community! If you have any issues or would like to contribute, feel free to open an issue or PR.

More Repositories

1

nginxconfig.io

⚙️ NGINX config generator on steroids 💉
JavaScript
27,244
star
2

doctl

The official command line interface for the DigitalOcean API.
Go
3,155
star
3

godo

DigitalOcean Go API client
Go
1,328
star
4

go-libvirt

Package libvirt provides a pure Go interface for interacting with Libvirt. Apache 2.0 Licensed.
Go
815
star
5

do_user_scripts

Shell
804
star
6

Kubernetes-Starter-Kit-Developers

Hands-on tutorial and Automation stack for an operations-ready DigitalOcean Kubernetes (DOKS) cluster.
HCL
705
star
7

firebolt

Golang framework for streaming ETL, observability data pipeline, and event processing apps
Go
688
star
8

go-qemu

Go packages to interact with QEMU using the QEMU Machine Protocol (QMP). Apache 2.0 Licensed.
Go
684
star
9

do-agent

Collects system metrics from DigitalOcean Droplets
Go
586
star
10

clusterlint

A best practices checker for Kubernetes clusters. 🤠
Go
536
star
11

vulcan

Vulcan extends Prometheus adding horizontal scalability and long-term storage
Go
531
star
12

digitalocean-cloud-controller-manager

Kubernetes cloud-controller-manager for DigitalOcean (beta)
Go
517
star
13

hacktoberfest

Hacktoberfest - App to manage the annual open-source challenge, used for the 2019 & 2020 seasons.
Ruby
510
star
14

droplet_kit

DropletKit is the official DigitalOcean API client for Ruby.
Ruby
507
star
15

terraform-provider-digitalocean

Terraform DigitalOcean provider
Go
487
star
16

action-doctl

GitHub Actions for DigitalOcean - doctl
JavaScript
454
star
17

ceph_exporter

Prometheus exporter that scrapes meta information about a ceph cluster.
Go
396
star
18

engineering-code-of-conduct

Code of Conduct for DigitalOcean's Engineering Team
289
star
19

go-openvswitch

Go packages which enable interacting with Open vSwitch and related tools. Apache 2.0 Licensed.
Go
282
star
20

kubernetes-sample-apps

Example DigitalOcean Kubernetes workload with service exposed through a DO load-balancer.
Python
252
star
21

marketplace-partners

Image validation, automation, and other tools for DigitalOcean Marketplace Vendors and Custom Image users
Shell
190
star
22

gta

gta: do transitive analysis to find packages whose dependencies have changed
Go
182
star
23

heartbot

A shot of love for your favorite chat client.
CoffeeScript
178
star
24

prometheus-client-c

A Prometheus Client in C
C
154
star
25

marketplace-kubernetes

This repository contains the source code and deployment scripts for Kubernetes-based applications listed in the DigitalOcean Marketplace.
Shell
154
star
26

go-smbios

Package smbios provides detection and access to System Management BIOS (SMBIOS) and Desktop Management Interface (DMI) data and structures. Apache 2.0 Licensed.
Go
152
star
27

kartograph

Kartograph makes it easy to generate and convert JSON. It's intention is to be used for API clients.
Ruby
147
star
28

OpenVPN-Pihole

https://marketplace.digitalocean.com/apps/openvpn-pihole
Shell
146
star
29

captainslog

A Syslog Protocol Parser
Go
136
star
30

resource_kit

Resource Kit provides tools to aid in making API Clients. Such as URL resolving, Request / Response layer, and more.
Ruby
134
star
31

go-workers2

better-go-workers
Go
121
star
32

doks-debug

A Docker image with Kubernetes manifests for investigation and troubleshooting.
Dockerfile
109
star
33

droplet-1-clicks

Packer build scripts for DigitalOcean Marketplace 1-clicks.
Shell
105
star
34

supabase-on-do

HCL
98
star
35

openapi

The OpenAPI v3 specification for DigitalOcean's public API.
JavaScript
97
star
36

container-blueprints

DigitalOcean Kubernetes(DOKS) Solution Blueprints
HCL
92
star
37

sample-dockerfile

⛵ App Platform sample Docker application.
Go
90
star
38

DOKS

Managed Kubernetes designed for simple and cost effective container orchestration.
80
star
39

app_action

Deploy to DigitalOcean Container Registry and App Platform
Go
78
star
40

navigators-guide

Book and code examples that help to build infrastructure on DigitalOcean
Shell
76
star
41

do-operator

The Kubernetes Operator for DigitalOcean
Go
76
star
42

pydo

Official DigitalOcean Python Client based on the DO OpenAPIv3 specification
Python
75
star
43

sample-django

Django sample app for DigitalOcean App Platform
Python
74
star
44

logtalez

logtalez is a minimal command line client (and API) for retrieving log streams from the rsyslog logging daemon over zeromq.
Go
73
star
45

do-markdownit

Markdown-It plugin for the DigitalOcean Community.
JavaScript
71
star
46

databases

66
star
47

sample-nodejs

⛵ App Platform sample Node.js application.
JavaScript
60
star
48

debian-sys-maint-roll-passwd

Script to update password for MySQL user "debian-sys-maint"
Shell
58
star
49

sample-nextjs

⛵ App Platform sample Next.js application.
JavaScript
57
star
50

sample-python

⛵ App Platform sample Python application.
Python
52
star
51

vmtop

Real-time monitoring of KVM/Qemu VMs
Python
52
star
52

kubecon-2022-doks-workshop

HCL
48
star
53

sample-flask

Sample Flask Application to be deployed on DigitalOcean's App Platform
HTML
45
star
54

sample-laravel

⛵ App Platform sample Laravel application.
PHP
43
star
55

pgremapper

CLI tool for manipulating Ceph's upmap exception table.
Go
43
star
56

k8s-staticroute-operator

Create static routes for your k8s nodes using CRDs.
Python
42
star
57

sample-functions-nodejs-qrcode

HTML
39
star
58

tos

DigitalOcean's Terms of Service agreement
37
star
59

sample-monorepo

Sample mono repo app (with multiple components) on the DigitalOcean App Platform.
Go
36
star
60

sample-golang

⛵ App Platform sample Golang application.
Go
36
star
61

droplet-agent

Droplet Agent is the daemon that runs on customer droplets to enable some features such as web console access.
Go
36
star
62

openvswitch_exporter

Command openvswitch_exporter implements a Prometheus exporter for Open vSwitch.
Go
32
star
63

sample-php

⛵ App Platform sample PHP application.
PHP
32
star
64

mastodon-on-kubernetes

Setting up Mastodon on DigitalOcean Kubernetes
HCL
30
star
65

sample-html

⛵ App Platform sample HTML application.
HTML
30
star
66

sample-functions-nodejs-helloworld

JavaScript
30
star
67

sample-functions-python-jokes

Python
30
star
68

flipop

Floating IP Controller for Kubernetes
Go
29
star
69

ansible-collection

DigitalOcean Ansible Collection
Python
28
star
70

sample-functions-golang-helloworld

Go
28
star
71

go-metadata

Go client for the metadata API.
Go
27
star
72

sample-react

⛵ App Platform sample React application.
JavaScript
27
star
73

marketplace-pi-hole-vpn

Pi-hole VPN image for Marketplace with Unbound & Wireguard
Shell
26
star
74

omniauth-digitalocean

DigitalOcean OAuth2 Strategy for OmniAuth
Ruby
26
star
75

github-changelog-generator

A tool to generate changelog entries from GitHub repositories.
Go
25
star
76

sample-functions-python-helloworld

Python
22
star
77

terraform-vault-github-oidc

Terraform module to configure Vault for GitHub OIDC authentication from Action runners.
HCL
22
star
78

sample-push-to-deploy-doks

Push-to-deploy example using DOCR and DOKS
Python
21
star
79

netbox-ip-controller

A Kubernetes controller to import the IP addresses and metadata of pods and services into NetBox.
Go
20
star
80

sample-expressjs

⛵ App Platform sample Express.js application.
19
star
81

terraform-provider-sendgrid

Sendgrid Terraform Provider
Go
19
star
82

sample-nuxtjs

⛵ App Platform sample Nuxt.js application.
Vue
19
star
83

sample-vuejs

⛵ App Platform sample Vue.js application.
Vue
17
star
84

production-ready-kubernetes-workshop

The repository for DigitalOcean's Production Ready Kubernetes Workshop
Python
16
star
85

sample-functions-python-twilio-sms

Sending sms via Twilio
Python
16
star
86

sample-rails

⛵ App Platform sample Ruby on Rails application.
Ruby
15
star
87

sample-functions-php-numberstowords

PHP
15
star
88

sample-functions-php-helloworld

A PHP helloworld sample function for Cloud Functions
PHP
14
star
89

sample-hugo

⛵ App Platform sample Hugo application.
14
star
90

github-pr-resource

Github pull request resource for Concourse
Go
13
star
91

sample-functions-python-sendgrid-email

Sending emails via Sendgrid API
Python
13
star
92

icingaweb2-module-netboximport

Icinga2 Director integration for Netbox
PHP
12
star
93

docker-shipit-engine

Docker image for https://github.com/Shopify/shipit-engine
Ruby
11
star
94

sample-functions-golang-presigned-url

Creating a presigned url for DO's Spaces
Go
10
star
95

digitalocean-ceph-lab

Terraform and Ansible automation to provision and configure a Ceph test environment on DigitalOcean.
HCL
10
star
96

k8s-adoption-journey

Hands-on tutorial for going from Day-1 to production on DigitalOcean Kubernetes. Goes with "Kubernetes Adoption Journey" document.
Python
9
star
97

sample-laravel-api

⛵ App Platform sample Laravel API application.
PHP
9
star
98

gnulib

A mirror of the gnulib portability and testing suite for internal builds that use it as a submodule
C
8
star
99

serverless-jamstack

Contains sample code for a serverless Jamstack tutorial published on docs.digitalocean.com
JavaScript
8
star
100

sample-gatsby

⛵ App Platform sample Gatsby application.
CSS
8
star