• Stars
    star
    1,236
  • Rank 37,991 (Top 0.8 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created almost 7 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

CSI driver for Ceph

Ceph CSI

GitHub release Mergify Status Go Report Card TODOs CII Best Practices

This repo contains the Ceph Container Storage Interface (CSI) driver for RBD, CephFS and Kubernetes sidecar deployment YAMLs to support CSI functionality: provisioner, attacher, resizer, driver-registrar and snapshotter.

Overview

Ceph CSI plugins implement an interface between a CSI-enabled Container Orchestrator (CO) and Ceph clusters. They enable dynamically provisioning Ceph volumes and attaching them to workloads.

Independent CSI plugins are provided to support RBD and CephFS backed volumes,

  • For details about configuration and deployment of RBD plugin, please refer rbd doc and for CephFS plugin configuration and deployment please refer cephFS doc.
  • For example usage of the RBD and CephFS CSI plugins, see examples in examples/.
  • Stale resource cleanup, please refer cleanup doc.

NOTE:

  • Ceph CSI Arm64 support is experimental.

Project status

Status: GA

Known to work CO platforms

Ceph CSI drivers are currently developed and tested exclusively in Kubernetes environments.

Ceph CSI Version Container Orchestrator Name Version Tested
v3.9.0 Kubernetes v1.25, v1.26, v1.27
v3.8.1 Kubernetes v1.25, v1.26, v1.27
v3.8.0 Kubernetes v1.24, v1.25, v1.26, v1.27

There is work in progress to make this CO-independent and thus support other orchestration environments (Nomad, Mesos..etc).

NOTE:

The supported window of Ceph CSI versions is "N.(x-1)": (N (Latest major release) . (x (Latest minor release) - 1)).

For example, if the Ceph CSI latest major version is 3.9.0 today, support is provided for the versions above 3.8.0. If users are running an unsupported Ceph CSI version, they will be asked to upgrade when requesting support.

Support Matrix

Ceph-CSI features and available versions

Please refer rbd nbd mounter for its support details.

Plugin Features Feature Status CSI Driver Version CSI Spec Version Ceph Cluster Version Kubernetes Version
RBD Dynamically provision, de-provision Block mode RWO volume GA >= v1.0.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.14.0
Dynamically provision, de-provision Block mode RWX volume GA >= v1.0.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.14.0
Dynamically provision, de-provision Block mode RWOP volume Alpha >= v3.5.0 >= v1.5.0 Octopus (>=15.0.0) >= v1.22.0
Dynamically provision, de-provision File mode RWO volume GA >= v1.0.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.14.0
Dynamically provision, de-provision File mode RWOP volume Alpha >= v3.5.0 >= v1.5.0 Octopus (>=15.0.0) >= v1.22.0
Provision File Mode ROX volume from snapshot Alpha >= v3.0.0 >= v1.0.0 Octopus (>=v15.0.0) >= v1.17.0
Provision File Mode ROX volume from another volume Alpha >= v3.0.0 >= v1.0.0 Octopus (>=v15.0.0) >= v1.16.0
Provision Block Mode ROX volume from snapshot Alpha >= v3.0.0 >= v1.0.0 Octopus (>=v15.0.0) >= v1.17.0
Provision Block Mode ROX volume from another volume Alpha >= v3.0.0 >= v1.0.0 Octopus (>=v15.0.0) >= v1.16.0
Creating and deleting snapshot GA >= v1.0.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.17.0
Provision volume from snapshot GA >= v1.0.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.17.0
Provision volume from another volume GA >= v1.0.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.16.0
Expand volume Beta >= v2.0.0 >= v1.1.0 Octopus (>=15.0.0) >= v1.15.0
Volume/PV Metrics of File Mode Volume GA >= v1.2.0 >= v1.1.0 Octopus (>=15.0.0) >= v1.15.0
Volume/PV Metrics of Block Mode Volume GA >= v1.2.0 >= v1.1.0 Octopus (>=15.0.0) >= v1.21.0
Topology Aware Provisioning Support Alpha >= v2.1.0 >= v1.1.0 Octopus (>=15.0.0) >= v1.14.0
CephFS Dynamically provision, de-provision File mode RWO volume GA >= v1.1.0 >= v1.0.0 Octopus (>=15.0.0) >= v1.14.0
Dynamically provision, de-provision File mode RWX volume GA >= v1.1.0 >= v1.0.0 Octopus (>=v15.0.0) >= v1.14.0
Dynamically provision, de-provision File mode ROX volume Alpha >= v3.0.0 >= v1.0.0 Octopus (>=v15.0.0) >= v1.14.0
Dynamically provision, de-provision File mode RWOP volume Alpha >= v3.5.0 >= v1.5.0 Octopus (>=15.0.0) >= v1.22.0
Creating and deleting snapshot GA >= v3.1.0 >= v1.0.0 Octopus (>=v15.2.4) >= v1.17.0
Provision volume from snapshot GA >= v3.1.0 >= v1.0.0 Octopus (>=v15.2.4) >= v1.17.0
Provision volume from another volume GA >= v3.1.0 >= v1.0.0 Octopus (>=v15.2.4) >= v1.16.0
Expand volume Beta >= v2.0.0 >= v1.1.0 Octopus (>=v15.0.0) >= v1.15.0
Volume/PV Metrics of File Mode Volume GA >= v1.2.0 >= v1.1.0 Octopus (>=v15.0.0) >= v1.15.0
NFS Dynamically provision, de-provision File mode RWO volume Alpha >= v3.6.0 >= v1.0.0 Pacific (>=16.2.0) >= v1.14.0
Dynamically provision, de-provision File mode RWX volume Alpha >= v3.6.0 >= v1.0.0 Pacific (>=16.2.0) >= v1.14.0
Dynamically provision, de-provision File mode ROX volume Alpha >= v3.6.0 >= v1.0.0 Pacific (>=16.2.0) >= v1.14.0
Dynamically provision, de-provision File mode RWOP volume Alpha >= v3.6.0 >= v1.5.0 Pacific (>=16.2.0) >= v1.22.0
Expand volume Alpha >= v3.7.0 >= v1.1.0 Pacific (>=16.2.0) >= v1.15.0
Creating and deleting snapshot Alpha >= v3.7.0 >= v1.1.0 Pacific (>=16.2.0) >= v1.17.0
Provision volume from snapshot Alpha >= v3.7.0 >= v1.1.0 Pacific (>=16.2.0) >= v1.17.0
Provision volume from another volume Alpha >= v3.7.0 >= v1.1.0 Pacific (>=16.2.0) >= v1.16.0

NOTE: The Alpha status reflects possible non-backward compatible changes in the future, and is thus not recommended for production use.

CSI spec and Kubernetes version compatibility

Please refer to the matrix in the Kubernetes documentation.

Ceph CSI Container images and release compatibility

Ceph CSI Release/Branch Container image name Image Tag
devel (Branch) quay.io/cephcsi/cephcsi canary
v3.9.0 (Release) quay.io/cephcsi/cephcsi v3.9.0
v3.8.1 (Release) quay.io/cephcsi/cephcsi v3.8.1
v3.8.0 (Release) quay.io/cephcsi/cephcsi v3.8.0
Deprecated Ceph CSI Release/Branch Container image name Image Tag
v3.7.2 (Release) quay.io/cephcsi/cephcsi v3.7.2
v3.7.1 (Release) quay.io/cephcsi/cephcsi v3.7.1
v3.7.0 (Release) quay.io/cephcsi/cephcsi v3.7.0
v3.6.1 (Release) quay.io/cephcsi/cephcsi v3.6.1
v3.6.0 (Release) quay.io/cephcsi/cephcsi v3.6.0
v3.5.1 (Release) quay.io/cephcsi/cephcsi v3.5.1
v3.5.0 (Release) quay.io/cephcsi/cephcsi v3.5.0
v3.4.0 (Release) quay.io/cephcsi/cephcsi v3.4.0
v3.3.1 (Release) quay.io/cephcsi/cephcsi v3.3.1
v3.3.0 (Release) quay.io/cephcsi/cephcsi v3.3.0
v3.2.2 (Release) quay.io/cephcsi/cephcsi v3.2.2
v3.2.1 (Release) quay.io/cephcsi/cephcsi v3.2.1
v3.2.0 (Release) quay.io/cephcsi/cephcsi v3.2.0
v3.1.2 (Release) quay.io/cephcsi/cephcsi v3.1.2
v3.1.1 (Release) quay.io/cephcsi/cephcsi v3.1.1
v3.1.0 (Release) quay.io/cephcsi/cephcsi v3.1.0
v3.0.0 (Release) quay.io/cephcsi/cephcsi v3.0.0
v2.1.2 (Release) quay.io/cephcsi/cephcsi v2.1.2
v2.1.1 (Release) quay.io/cephcsi/cephcsi v2.1.1
v2.1.0 (Release) quay.io/cephcsi/cephcsi v2.1.0
v2.0.1 (Release) quay.io/cephcsi/cephcsi v2.0.1
v2.0.0 (Release) quay.io/cephcsi/cephcsi v2.0.0
v1.2.2 (Release) quay.io/cephcsi/cephcsi v1.2.2
v1.2.1 (Release) quay.io/cephcsi/cephcsi v1.2.1
v1.2.0 (Release) quay.io/cephcsi/cephcsi v1.2.0
v1.1.0 (Release) quay.io/cephcsi/cephcsi v1.1.0
v1.0.0 (Branch) quay.io/cephcsi/cephfsplugin v1.0.0
v1.0.0 (Branch) quay.io/cephcsi/rbdplugin v1.0.0

Contributing to this repo

Please follow development-guide and coding style guidelines if you are interested to contribute to this repo.

Troubleshooting

Please submit an issue at: Issues

Weekly Bug Triage call

We conduct weekly bug triage calls at our slack channel on Tuesdays. More details are available here

Dev standup

A regular dev standup takes place every Tuesday at 12:00 PM UTC Convert to your local timezone by executing command date -d "12:00 UTC" on terminal

Any changes to the meeting schedule will be added to the agenda doc.

Anyone who wants to discuss the direction of the project, design and implementation reviews, or general questions with the broader community is welcome and encouraged to join.

Contact

Please use the following to reach members of the community:

More Repositories

1

ceph

Ceph is a distributed object, block, and file storage platform
C++
13,799
star
2

ceph-ansible

Ansible playbooks to deploy Ceph, the distributed filesystem.
Python
1,685
star
3

ceph-container

Docker files and images to run Ceph in containers
Shell
1,316
star
4

go-ceph

Go bindings for Ceph πŸ™ πŸ™ πŸ™
Go
610
star
5

ceph-deploy

Deploy Ceph with minimal infrastructure, using just SSH access
Python
421
star
6

calamari

Web-based monitoring and management for Ceph
Python
349
star
7

cbt

The Ceph Benchmarking Tool
Python
266
star
8

s3-tests

Compatibility tests for S3 clones
Python
261
star
9

cn

Ceph Nano - One step S3 in container with Ceph.
Go
235
star
10

ceph-client

Ceph kernel client (kernel modules)
C
187
star
11

teuthology

Ceph test suite
Python
153
star
12

cephadm-ansible

ansible playbooks to be used with cephadm
Python
112
star
13

calamari-clients

Ceph Manager API Client Code
JavaScript
102
star
14

ceph-cookbook

Chef cookbooks for Ceph
Ruby
101
star
15

ceph-rust

Rust-lang interface to Ceph.
Rust
92
star
16

dmclock

Code that implements the dmclock distributed quality of service algorithm. See "mClock: Handling Throughput Variability for Hypervisor IO Scheduling" by Gulati, Merchant, and Varman.
C++
88
star
17

ceph-nagios-plugins

Nagios plugins for Ceph
Python
82
star
18

ceph-nvmeof

Service to provide Ceph storage over NVMe-oF/TCP protocol
Python
76
star
19

cephmetrics

ceph metric collectors with collectd integration
Python
64
star
20

gf-complete

this repository is a read only mirror, the upstream is
C
62
star
21

ceph-iscsi

Ceph iSCSI tools
Python
61
star
22

cephfs-hadoop

cephfs-hadoop
Java
57
star
23

romana

JavaScript
51
star
24

ceph-build

Helper scripts for building the official Ceph packages
Shell
44
star
25

qemu-kvm

Ceph RBD support for Qemu/KVM
C
40
star
26

phprados

PHP bindings for the RADOS client library
C
37
star
27

ceph-cosi

COSI driver for Ceph Object Store aka RGW
Go
34
star
28

jerasure

this repository is a read only mirror, the upstream is
C
33
star
29

ceph-tools

Misc ceph tools
Python
33
star
30

ceph-qa-suite

[DEPRECATED; see ceph.git/qa] Suite of Ceph QA tests to run with Teuthology
Python
33
star
31

libs3

Fork of http://git.ischo.com/libs3.git
C
32
star
32

ceph-salt

Deploy Ceph clusters using cephadm
Python
31
star
33

ceph-chef

Chef cookbooks for managing a Ceph cluster
Ruby
29
star
34

downburst

Fast Ubuntu Cloud Image creation on libvirt
Python
27
star
35

ceph-iscsi-cli

NOTICE: moved to https://github.com/ceph/ceph-iscsi
Python
25
star
36

ceph-medic

find common issues in ceph clusters
Python
22
star
37

pulpito

A dashboard for Ceph tests
JavaScript
22
star
38

ceph-iscsi-config

NOTICE: moved to https://github.com/ceph/ceph-iscsi
Python
22
star
39

radosgw-agent

radosgw sync agent
Python
22
star
40

ceph-cm-ansible

Ansible configurations for Ceph.com infrastructure
Shell
21
star
41

python-crush

C++
20
star
42

ceph-mixins

A set of Grafana dashboards and Prometheus alerts for Ceph.
Jsonnet
20
star
43

ceph-ci

ceph.git clone as source for CI
C++
20
star
44

dpdk

DPDK
C
19
star
45

puppet-ceph

Mirror of stackforge/puppet-ceph
Ruby
17
star
46

ceph.io

This repo contains static site content for www.ceph.io
HTML
16
star
47

libcrush

C
16
star
48

ceph-installer

A service to provision Ceph clusters
Python
15
star
49

paddles

RESTful API to store (and report) on Ceph tests
Python
14
star
50

persistent-volume-migrator

A collection of tools to migrate an ancient Kubernetes Ceph storage driver (in-tree, Flex) to Ceph-CSI
Go
14
star
51

propernoun

Update PowerDNS from DHCP leases and libvirt virtual machines
Python
13
star
52

rgw-pubsub-api

RGW PubSub API Clients
Go
13
star
53

ceph-csi-operator

Kubernetes operator for managing the CephCSI plugins
Go
13
star
54

obsync

rsync-like utility for syncing bucket data between object storage APIs like S3, Swift
Python
12
star
55

ceph-iscsi-tools

Useful tools for a ceph/iscsi gateway environment
Python
11
star
56

autobuild-ceph

Setup for running gitbuilder for the Ceph project
Shell
11
star
57

ceph-openstack-tools

Tools to develop Ceph/OpenStack integration
Shell
11
star
58

simplegpt

Simple Python library to parse GPT (GUID Partition Table) header and entries, useful as a learning tool
Python
11
star
59

ceph-ruby

Easy management of Ceph Distributed Storage System (rbd, images, rados objects) using ruby.
Ruby
11
star
60

python-jenkins

fork of python-jenkins for https://review.openstack.org/460363
Python
11
star
61

mod_fastcgi

Bugfixes and improves to mod_fastcgi, for use with RADOS Gateway
C
10
star
62

samba

Clone of the main samba repo: git://git.samba.org/samba.git
C
10
star
63

csi-charts

csi-charts
9
star
64

chacra

A binary/file REST API to aid in multi-distro|arch|release management
Python
9
star
65

ceph-erasure-code-corpus

Objects erasure encoded by Ceph
Shell
9
star
66

ceph-client-standalone

Standalone Ceph kernel client -- you probably want https://github.com/NewDreamNetwork/ceph-client instead
C
9
star
67

barclamp-ceph

Crowbar Barclamp for installing Ceph clusters
Ruby
8
star
68

blkin

C++
8
star
69

shaman

source of truth for the state of repositories on Chacra nodes
Python
8
star
70

apache2

A version of Apache HTTP Server with fixes for use with RADOS Gateway
C
7
star
71

ceph-qa-chef

Chef cookbooks used in Ceph QA jobs. (This is deprecated; please see ceph-cm-ansible instead.)
Ruby
7
star
72

ceph-kmod-rpm

kabi-tracking kmod RPMs for libceph, CephFS, and RDB for RHEL 7
7
star
73

mod-proxy-fcgi

mod_proxy_fcgi for apache 2.2
C
6
star
74

ceph-devstack

DevStack files
Shell
6
star
75

spawn

C++
6
star
76

leveldb

Fork of the LevelDB project
C++
5
star
77

keys

SSH and other keys used by the project, mostly in the Sepia lab
Shell
5
star
78

gmock

C++
5
star
79

cn-core

Bootstrap Ceph AIO - source of cn project
Go
5
star
80

qemu-iotests

Shell
5
star
81

ceph-autotests

HISTORICAL value only: Autotest helper for Ceph QA (obsolete)
Python
4
star
82

mita

Jenkins Slave orchestration service
Python
4
star
83

asphyxiate

Grab source code documentation via Doxygen into a Sphinx document
Python
4
star
84

collectd-4.10.1

A version of collectd that supports monitoring Ceph clusters (on top of the Debian 4.10.1-1+squeeze2 package)
C
4
star
85

ceph-nagios-plugin

A Nagios plugin that checks the health of a ceph cluster.
Perl
4
star
86

cookbook-vercoi

Chef Solo recipes used to bring up KVM hypervisors in the Sepia lab
Ruby
4
star
87

prado

Prado is a webservice that provides a single script to run Ansible playbooks
Python
4
star
88

handle_core

A userspace core file handler for Linux
C
4
star
89

ceph-object-corpus

corpus of encoded ceph structures
Shell
4
star
90

ceph-telemetry

Python
4
star
91

jenkins-slave-chef

Chef to setup jenkins slaves (pbuilder/regular).
Ruby
3
star
92

merfi

Finds and signs files with different signing tools (gpg, rpm-sign)
Python
3
star
93

ceph-kdump-copy

ceph kdump handler
Shell
3
star
94

run-crowbar-on-sepia

Quick & dirty script to run Crowbar on Sepia
Python
3
star
95

ceph-notes

3
star
96

libkmip

C
3
star
97

cephmetrics-osp

Shell
2
star
98

ceph-brag

Ceph performance testing results repository.
Python
2
star
99

munging_http_proxy

Munging HTTP proxy, for developing and debugging
Python
2
star
100

cookbook-vm-general

Python
2
star