• Stars
    star
    231
  • Rank 166,944 (Top 4 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 6 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Ceph Nano - One step S3 in container with Ceph.

cn (ceph-nano)

Ceph, the future of storage

The project

cn is a little program written in Go that helps you interact with the S3 API by providing a REST S3 compatible gateway. The target audience is developers building their applications on Amazon S3. It is also an exciting tool to showcase Ceph Rados Gateway S3 compatibility. This is brought to you by the power of Ceph and Containers. Under the hood, cn runs a Ceph container and exposes a Rados Gateway. For convenience, cn also comes with a set of commands to work with the S3 gateway. Before you ask "why not using s3cmd instead?", then you will be happy to read that internally cn uses s3cmd and act as a wrapper around the most commonly used commands. Also, keep in mind that the CLI is just for convenience, and the primary use case is you developing your application directly on the S3 API.

Table of contents

Build

You can build cn by using make. Be sure dep is installed:

$ go get github.com/golang/dep/cmd/dep

Then, add ~/go/bin to your $PATH:

$ export PATH=$PATH:~/go/bin

Build cn:

$ make
rm -f cn cn &>/dev/null || true
dep ensure
GOOS=linux GOARCH=amd64 go build -i -ldflags="-X main.version=cea247c-dirty -X main.tag=devel -X main.branch=guits-doc_build" -o cn-devel-cea247c-dirty-linux-amd64 main.go
ln -sf "cn-devel-cea247c-dirty-linux-amd64" cn

Once the build is done, you should have a symlink cn pointing to the binary that just got built:

$ ls -l
total 10692
-rw-rw-r--. 1 guits guits    15292 20 nov.  22:03 ceph-nano-logo-vertical.jpg
drwxrwxr-x. 2 guits guits     4096 20 nov.  22:03 cmd
lrwxrwxrwx. 1 guits guits       34 20 nov.  22:27 cn -> cn-devel-cea247c-dirty-linux-amd64
-rwxrwxr-x. 1 guits guits 10881196 20 nov.  22:27 cn-devel-cea247c-dirty-linux-amd64

Installation

cn relies on Docker so it must be installed on your machine. If you're not running a Linux workstation you can install Docker for Mac.

Once Docker is installed you're ready to start. Open your terminal and download the cn binary.

macOS:

curl -L https://github.com/ceph/cn/releases/download/v2.3.1/cn-v2.3.1-darwin-amd64 -o cn && chmod +x cn

Linux amd64:

curl -L https://github.com/ceph/cn/releases/download/v2.3.1/cn-v2.3.1-linux-amd64 -o cn && chmod +x cn

Linux arm64:

curl -L https://github.com/ceph/cn/releases/download/v2.3.1/cn-v2.3.1-linux-arm64 -o cn && chmod +x cn

Test it out

$ ./cn
Ceph Nano - One step S3 in container with Ceph.

                  *(((((((((((((
                (((((((((((((((((((
              ((((((((*     ,(((((((*
             ((((((             ((((((
            *((((,               ,((((/
            ((((,     ((((((/     *((((
            ((((     (((((((((     ((((
            /(((     (((((((((     ((((
             (((.     (((((((     /(((/
              (((                *((((
              .(((              (((((
         ,(((((((*             /(((
          .(((((  ((( (/  //   (((
                 /(((.  /(((((  /(((((
                        .((((/ (/

Usage:
  cn [command]

Available Commands:
  cluster      Interact with a particular Ceph cluster
  s3           Interact with a particular S3 object server
  image        Interact with cn's container image(s)
  version      Print the version of cn
  kube         Outputs cn kubernetes template (cn kube > kube-cn.yml)
  update-check Print cn current and latest version number
  flavors      Interact with flavors
  completion   Generates bash completion scripts

Flags:
  -h, --help   help for cn

Use "cn [command] --help" for more information about a command.

Get started

Start the program with a working directory /tmp, the initial start might take a few minutes since we need to download the container image:

$ ./cn cluster start -d /tmp my-first-cluster
Running ceph-nano...
The container image is not present, pulling it.
This operation can take a few minutes......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Endpoint: http://10.36.116.164:8000
Dashboard: http://10.36.116.164:5001
Access key is: 9ZU1QBYX13KPLXXDDCY2
Secret key is: nthNG1xb7ta5IDKiJKM8626pQitqsalEo0ta7B9E
Working directory: /usr/share/ceph-nano

Selecting the cluster flavor

Is it possible to select the cluster flavor by using the -f option on the command line.

$ ./cn cluster start mycluster -f huge

The full documentation of flavors can be found here

Your first S3 bucket

Create a bucket with cn:

$ ./cn s3 mb my-first-cluster my-buc
Bucket 's3://my-buc/' created

$ ./cn s3 put my-first-cluster /etc/passwd my-buc
upload: '/tmp/passwd' -> 's3://my-buc/passwd'  [1 of 1]
 5925 of 5925   100% in    1s     4.57 kB/s  done

Multi-cluster support

cn can manage any number of clusters on your local machine:

$ ./cn cluster ls
+------+---------+-------------------------------------------------------------------------------------+----------------+--------------------------------+---------+
| NAME | STATUS  | IMAGE                                                                               | IMAGE RELEASE  | IMAGE CREATION TIME            | FLAVOR  |
+------+---------+-------------------------------------------------------------------------------------+----------------+--------------------------------+---------+
| d    | running | ceph/daemon:latest                                                                  | master-77e3d8d | 2018-04-05T15:01:40.323603472Z | default |
| b    | running | ceph/daemon@sha256:369867e450ccdea9bcea7f54e97ed8b2cb1a0437fbef658d2d01fce2b8a2c648 | master-5f44af9 | 2018-03-30T21:08:31.117367166Z | medium  |
+------+---------+-------------------------------------------------------------------------------------+----------------+--------------------------------+---------+

List Ceph container images available

cn can list the available Ceph container images, the default output shows the 100 first images:

$ ./cn image ls
ceph/daemon:latest-mimic
ceph/daemon:latest-luminous
ceph/daemon:latest-master
ceph/daemon:master-0b3eb04-mimic-centos-7
ceph/daemon:master-0b3eb04-luminous-centos-7
ceph/daemon:master-0b3eb04-luminous-opensuse-42.3-x86_64
ceph/daemon:master-0b3eb04-master-centos-7-x86_64
ceph/daemon:master-0b3eb04-luminous-centos-7-x86_64
ceph/daemon:master-0b3eb04-mimic-centos-7-x86_64
[...]

Using images aliases

The image option (-i) support aliases to simply the command line. It is possible to list the aliases by running the image show-aliases command as per below :

$ ./cn image show-aliases
+----------+--------------------------------------------------+
| ALIAS    | IMAGE_NAME                                       |
+----------+--------------------------------------------------+
| mimic    | ceph/daemon:latest-mimic                         |
| luminous | ceph/daemon:latest-luminous                      |
| redhat   | registry.access.redhat.com/rhceph/rhceph-3-rhel7 |
+----------+--------------------------------------------------+

Aliases can be use in place of the traditional image name as per the following example:

$ ./cn cluster start mycluster -i mimic

It is also possible to create new aliases as detailed here

Enable mgr dashboard

TODO: This is a temporary hack to enable the manager dashboard

Currently cn does not expose a port for the mgr dashboard. It only exposes port 8000 for S3 API, and port 5000 for Sree - S3 web client. To expose also the mgr dashboard port we currently have to do some hacks.

This section will guide you how to manually commit a new image and then run a new container with the desired expose ports.

Commit a copy of the docker image:

./cn cluster start temp -d /tmp
docker commit ceph-nano-temp ceph-nano
./cn cluster purge temp --yes-i-am-sure

Run the container:

docker run -dt --name cn -p 8080:8080 -p 5000:5000 -p 8000:8000 ceph-nano

Enable dashboard:

(Note: 'enable dashboard' command will cause the container to exit, so need to start it after)

docker exec cn ceph config set mgr mgr/dashboard/ssl false
docker exec cn ceph config set mgr mgr/dashboard/server_addr 0.0.0.0
docker exec cn ceph config set mgr mgr/dashboard/server_port 8080
docker exec cn ceph mgr module enable dashboard
until docker exec cn ceph; do docker start cn; sleep 1; done # wait for the services to start
docker exec cn ceph dashboard set-login-credentials nano nano

Note that the Object Gateway tab in the dashboard is not enabled yet, so run the following to enable RGW dashboard:

RGW_USER=$(docker exec cn radosgw-admin user create --uid=rgw --display-name=rgw --system)
RGW_ACCESS=$(echo $RGW_USER | awk '{ for (i=1;i<=NF;++i) if ($i ~ /access_key/) { split($(i+1),a,"\""); print a[2] } }')
RGW_SECRET=$(echo $RGW_USER | awk '{ for (i=1;i<=NF;++i) if ($i ~ /secret_key/) { split($(i+1),a,"\""); print a[2] } }')
docker exec cn ceph dashboard set-rgw-api-access-key "$RGW_ACCESS"
docker exec cn ceph dashboard set-rgw-api-secret-key "$RGW_SECRET"
docker exec cn ceph dashboard set-rgw-api-host 127.0.0.1
docker exec cn ceph dashboard set-rgw-api-port 8000
docker exec cn ceph dashboard set-rgw-api-scheme http
docker exec cn ceph dashboard set-rgw-api-user-id rgw

The dashboard should now be accessible:

Troubleshooting - verify that your config dump should look like this:

$ docker exec cn ceph config dump
WHO   MASK LEVEL   OPTION                           VALUE                                                        RO 
  mgr      unknown mgr/dashboard/RGW_API_ACCESS_KEY ********************                                         *  
  mgr      unknown mgr/dashboard/RGW_API_HOST       127.0.0.1                                                    *  
  mgr      unknown mgr/dashboard/RGW_API_PORT       8000                                                         *  
  mgr      unknown mgr/dashboard/RGW_API_SCHEME     http                                                         *  
  mgr      unknown mgr/dashboard/RGW_API_SECRET_KEY ****************************************                     *  
  mgr      unknown mgr/dashboard/RGW_API_USER_ID    rgw                                                          *  
  mgr      unknown mgr/dashboard/password           ************************************************************ *  
  mgr      unknown mgr/dashboard/server_addr        0.0.0.0                                                      *  
  mgr      unknown mgr/dashboard/server_port        8080                                                         *  
  mgr      unknown mgr/dashboard/ssl                false                                                        *  
  mgr      unknown mgr/dashboard/username           nano                                                         *  

More Repositories

1

ceph

Ceph is a distributed object, block, and file storage platform
C++
13,026
star
2

ceph-ansible

Ansible playbooks to deploy Ceph, the distributed filesystem.
Python
1,631
star
3

ceph-container

Docker files and images to run Ceph in containers
Shell
1,297
star
4

ceph-csi

CSI driver for Ceph
Go
1,123
star
5

go-ceph

Go bindings for Ceph 🐙 🐙 🐙
Go
585
star
6

ceph-deploy

Deploy Ceph with minimal infrastructure, using just SSH access
Python
417
star
7

calamari

Web-based monitoring and management for Ceph
Python
349
star
8

s3-tests

Compatibility tests for S3 clones
Python
261
star
9

cbt

The Ceph Benchmarking Tool
Python
250
star
10

ceph-client

Ceph kernel client (kernel modules)
C
187
star
11

teuthology

Ceph test suite
Python
153
star
12

calamari-clients

Ceph Manager API Client Code
JavaScript
103
star
13

ceph-cookbook

Chef cookbooks for Ceph
Ruby
101
star
14

dmclock

Code that implements the dmclock distributed quality of service algorithm. See "mClock: Handling Throughput Variability for Hypervisor IO Scheduling" by Gulati, Merchant, and Varman.
C++
88
star
15

cephadm-ansible

ansible playbooks to be used with cephadm
Python
86
star
16

ceph-nagios-plugins

Nagios plugins for Ceph
Python
81
star
17

ceph-nvmeof

Service to provide Ceph storage over NVMe-oF protocol
Python
66
star
18

cephmetrics

ceph metric collectors with collectd integration
Python
64
star
19

gf-complete

this repository is a read only mirror, the upstream is
C
60
star
20

ceph-iscsi

Ceph iSCSI tools
Python
58
star
21

cephfs-hadoop

cephfs-hadoop
Java
57
star
22

romana

JavaScript
51
star
23

ceph-build

Helper scripts for building the official Ceph packages
Shell
41
star
24

qemu-kvm

Ceph RBD support for Qemu/KVM
C
40
star
25

phprados

PHP bindings for the RADOS client library
C
37
star
26

jerasure

this repository is a read only mirror, the upstream is
C
33
star
27

ceph-tools

Misc ceph tools
Python
33
star
28

libs3

Fork of http://git.ischo.com/libs3.git
C
32
star
29

ceph-qa-suite

[DEPRECATED; see ceph.git/qa] Suite of Ceph QA tests to run with Teuthology
Python
32
star
30

ceph-cosi

COSI driver for Ceph Object Store aka RGW
Go
29
star
31

ceph-chef

Chef cookbooks for managing a Ceph cluster
Ruby
29
star
32

ceph-salt

Deploy Ceph clusters using cephadm
Python
29
star
33

downburst

Fast Ubuntu Cloud Image creation on libvirt
Python
27
star
34

ceph-iscsi-cli

NOTICE: moved to https://github.com/ceph/ceph-iscsi
Python
25
star
35

ceph-medic

find common issues in ceph clusters
Python
22
star
36

pulpito

A dashboard for Ceph tests
JavaScript
22
star
37

ceph-iscsi-config

NOTICE: moved to https://github.com/ceph/ceph-iscsi
Python
22
star
38

radosgw-agent

radosgw sync agent
Python
22
star
39

python-crush

C++
20
star
40

ceph-cm-ansible

Ansible configurations for Ceph.com infrastructure
Shell
20
star
41

ceph-mixins

A set of Grafana dashboards and Prometheus alerts for Ceph.
Jsonnet
19
star
42

puppet-ceph

Mirror of stackforge/puppet-ceph
Ruby
18
star
43

ceph-ci

ceph.git clone as source for CI
C++
18
star
44

dpdk

DPDK
C
18
star
45

libcrush

C
15
star
46

ceph-installer

A service to provision Ceph clusters
Python
15
star
47

paddles

RESTful API to store (and report) on Ceph tests
Python
14
star
48

propernoun

Update PowerDNS from DHCP leases and libvirt virtual machines
Python
13
star
49

ceph.io

This repo contains static site content for www.ceph.io
HTML
13
star
50

persistent-volume-migrator

A collection of tools to migrate an ancient Kubernetes Ceph storage driver (in-tree, Flex) to Ceph-CSI
Go
13
star
51

rgw-pubsub-api

RGW PubSub API Clients
Go
13
star
52

obsync

rsync-like utility for syncing bucket data between object storage APIs like S3, Swift
Python
12
star
53

ceph-iscsi-tools

Useful tools for a ceph/iscsi gateway environment
Python
11
star
54

autobuild-ceph

Setup for running gitbuilder for the Ceph project
Shell
11
star
55

simplegpt

Simple Python library to parse GPT (GUID Partition Table) header and entries, useful as a learning tool
Python
11
star
56

ceph-openstack-tools

Tools to develop Ceph/OpenStack integration
Shell
11
star
57

ceph-ruby

Easy management of Ceph Distributed Storage System (rbd, images, rados objects) using ruby.
Ruby
11
star
58

python-jenkins

fork of python-jenkins for https://review.openstack.org/460363
Python
11
star
59

mod_fastcgi

Bugfixes and improves to mod_fastcgi, for use with RADOS Gateway
C
10
star
60

samba

Clone of the main samba repo: git://git.samba.org/samba.git
C
10
star
61

chacra

A binary/file REST API to aid in multi-distro|arch|release management
Python
9
star
62

ceph-client-standalone

Standalone Ceph kernel client -- you probably want https://github.com/NewDreamNetwork/ceph-client instead
C
9
star
63

barclamp-ceph

Crowbar Barclamp for installing Ceph clusters
Ruby
8
star
64

blkin

C++
8
star
65

shaman

source of truth for the state of repositories on Chacra nodes
Python
8
star
66

csi-charts

csi-charts
8
star
67

ceph-erasure-code-corpus

Objects erasure encoded by Ceph
Shell
8
star
68

apache2

A version of Apache HTTP Server with fixes for use with RADOS Gateway
C
7
star
69

ceph-qa-chef

Chef cookbooks used in Ceph QA jobs. (This is deprecated; please see ceph-cm-ansible instead.)
Ruby
7
star
70

ceph-kmod-rpm

kabi-tracking kmod RPMs for libceph, CephFS, and RDB for RHEL 7
7
star
71

mod-proxy-fcgi

mod_proxy_fcgi for apache 2.2
C
6
star
72

ceph-devstack

DevStack files
Shell
6
star
73

spawn

C++
6
star
74

leveldb

Fork of the LevelDB project
C++
5
star
75

gmock

C++
5
star
76

cn-core

Bootstrap Ceph AIO - source of cn project
Go
5
star
77

keys

SSH and other keys used by the project, mostly in the Sepia lab
Shell
5
star
78

qemu-iotests

Shell
5
star
79

ceph-autotests

HISTORICAL value only: Autotest helper for Ceph QA (obsolete)
Python
4
star
80

mita

Jenkins Slave orchestration service
Python
4
star
81

collectd-4.10.1

A version of collectd that supports monitoring Ceph clusters (on top of the Debian 4.10.1-1+squeeze2 package)
C
4
star
82

asphyxiate

Grab source code documentation via Doxygen into a Sphinx document
Python
4
star
83

ceph-nagios-plugin

A Nagios plugin that checks the health of a ceph cluster.
Perl
4
star
84

cookbook-vercoi

Chef Solo recipes used to bring up KVM hypervisors in the Sepia lab
Ruby
4
star
85

prado

Prado is a webservice that provides a single script to run Ansible playbooks
Python
4
star
86

handle_core

A userspace core file handler for Linux
C
4
star
87

ceph-telemetry

Python
4
star
88

rook-client-python

Python bindings for Rook-Ceph CRDs
Python
3
star
89

ceph-kdump-copy

ceph kdump handler
Shell
3
star
90

jenkins-slave-chef

Chef to setup jenkins slaves (pbuilder/regular).
Ruby
3
star
91

merfi

Finds and signs files with different signing tools (gpg, rpm-sign)
Python
3
star
92

run-crowbar-on-sepia

Quick & dirty script to run Crowbar on Sepia
Python
3
star
93

ceph-notes

3
star
94

ceph-object-corpus

corpus of encoded ceph structures
Shell
3
star
95

libkmip

C
3
star
96

cookbook-vm-general

Python
2
star
97

munging_http_proxy

Munging HTTP proxy, for developing and debugging
Python
2
star
98

javaws-kludge

Kludges to make working with Dell iDRAC remote consoles nicer
Python
2
star
99

sepia

Notes on the test lab for the Ceph project
Shell
2
star
100

bubbles

Bubbles is an experiment on a simplified management UI for Ceph.
TypeScript
2
star