• Stars
    star
    1,675
  • Rank 27,886 (Top 0.6 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created over 8 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Zenko CloudServer, an open-source Node.js implementation of the Amazon S3 protocol on the front-end and backend storage capabilities to multiple clouds, including Azure and Google.

Zenko CloudServer

Zenko CloudServer logo

Docker Pulls Docker Pulls

Overview

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller.

CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.

CloudServer is useful for Developers, either to run as part of a continous integration test environment to emulate the AWS S3 service locally or as an abstraction layer to develop object storage enabled application on the go.

Learn more at www.zenko.io/cloudserver

May I offer you some lovely documentation?

Docker

Run your Zenko CloudServer with Docker

Contributing

In order to contribute, please follow the Contributing Guidelines.

Installation

Dependencies

Building and running the Zenko CloudServer requires node.js 10.x and yarn v1.17.x . Up-to-date versions can be found at Nodesource.

Clone source code

git clone https://github.com/scality/S3.git

Install js dependencies

Go to the ./S3 folder,

yarn install --frozen-lockfile

If you get an error regarding installation of the diskUsage module, please install g++.

If you get an error regarding level-down bindings, try clearing your yarn cache:

yarn cache clean

Run it with a file backend

yarn start

This starts a Zenko CloudServer on port 8000. Two additional ports 9990 and 9991 are also open locally for internal transfer of metadata and data, respectively.

The default access key is accessKey1 with a secret key of verySecretKey1.

By default the metadata files will be saved in the localMetadata directory and the data files will be saved in the localData directory within the ./S3 directory on your machine. These directories have been pre-created within the repository. If you would like to save the data or metadata in different locations of your choice, you must specify them with absolute paths. So, when starting the server:

mkdir -m 700 $(pwd)/myFavoriteDataPath
mkdir -m 700 $(pwd)/myFavoriteMetadataPath
export S3DATAPATH="$(pwd)/myFavoriteDataPath"
export S3METADATAPATH="$(pwd)/myFavoriteMetadataPath"
yarn start

Run it with multiple data backends

export S3DATA='multiple'
yarn start

This starts a Zenko CloudServer on port 8000. The default access key is accessKey1 with a secret key of verySecretKey1.

With multiple backends, you have the ability to choose where each object will be saved by setting the following header with a locationConstraint on a PUT request:

'x-amz-meta-scal-location-constraint':'myLocationConstraint'

If no header is sent with a PUT object request, the location constraint of the bucket will determine where the data is saved. If the bucket has no location constraint, the endpoint of the PUT request will be used to determine location.

See the Configuration section in our documentation here to learn how to set location constraints.

Run it with an in-memory backend

yarn run mem_backend

This starts a Zenko CloudServer on port 8000. The default access key is accessKey1 with a secret key of verySecretKey1.

Run it with Vault user management

Note: Vault is proprietary and must be accessed separately.

export S3VAULT=vault
yarn start

This starts a Zenko CloudServer using Vault for user management.

More Repositories

1

Zenko

Zenko is the open source multi-cloud data controller: own and keep control of your data on any cloud.
Gherkin
565
star
2

metalk8s

An opinionated Kubernetes distribution with a focus on long-term on-prem deployments
SaltStack
353
star
3

elmerfs

Rust
94
star
4

Droplet

Cloud storage client library
C
54
star
5

backbeat

Zenko Backbeat is the core engine for asynchronous replication, optimized for queuing metadata updates and dispatching work to long-running tasks in the background.
JavaScript
53
star
6

quadiron

Fast Erasure Coding Library for Large Number of Data and Parities
C++
27
star
7

core-ui

Shared React components for UI
TypeScript
20
star
8

werelogs

A logging library providing efficient raw logging in the form of JSON data.
JavaScript
16
star
9

Cloud-Migration-Tool

A Cloud storage migration command-line tool that uses the scality droplet library
C
15
star
10

node-nfsc

NFS client bindings for NodeJS
C++
14
star
11

Arsenal

Common utilities for the open-source Scality S3 project components
JavaScript
14
star
12

RestBlockDriver

RestBlockDriver
C
13
star
13

utapi

API for tracking resource utilization and reporting metrics
JavaScript
12
star
14

Droplet-sh

Shell to access cloud storage via command line
C
11
star
15

s3blaster

S3 Blaster
JavaScript
10
star
16

zenko-ui

Zenko UI
TypeScript
10
star
17

vaultclient

Client library for the Proprietary Scality Auth server backing the FOSS S3 server
JavaScript
8
star
18

runner-manager

Service to manage github Actions self-hosted runners
Python
7
star
19

node-livy-client

NodeJS Livy Client
JavaScript
6
star
20

zenko-cosbench

Docker image and Helm Chart for Cosbench, as deployed in Zenko https://zenko.io
Smarty
5
star
21

ScalitySproxydSwift

Scality sproxyd driver for OpenStack Swift
Python
5
star
22

s3utils

S3 Connector and Zenko Utilities
JavaScript
4
star
23

ballot

Go
4
star
24

eclib

JavaScript
4
star
25

lustre-droplet-copytool

A copytool for the Lustre HSM to tier the data towards Scality's storage system.
C
4
star
26

osis

Java
3
star
27

centos-kernel

CentOS kernel packages with various patches
Shell
3
star
28

Guidelines

Provide the basic common guidelines for Scality JS projects
JavaScript
3
star
29

bucketclient

Client library for the Proprietary Scality Bucket server backing the FOSS S3 server
JavaScript
3
star
30

gh-actions-exporter

GitHub Actions Exporter
Python
3
star
31

CaDMIum

CaDMIum is a CDMI client java library
Java
3
star
32

zookeeper-mock

NodeJS Zookeeper Mock
JavaScript
3
star
33

scality-sproxyd-client

Python client for Scality sproxyd daemon
Python
3
star
34

spark

Python
3
star
35

dropletfs

A fuse file system for cloud storage, using the Scality Droplet library (http://github.com/scality/Droplet)
C
3
star
36

clueso

Search all the things!
Scala
3
star
37

hdclient

HyperDrive Client Library
TypeScript
2
star
38

module-federation

TypeScript
2
star
39

sproxydclient

Node.js client library for the Proprietary Scality sproxyd software
JavaScript
2
star
40

react-chained-query

A wrapper of react-query useQuery hook allowing chaining queries.
TypeScript
2
star
41

changelog-binder

Tooling to generate changelogs and release notes from snippets stored in a project Git repository, compatible with the GitWaterFlow branching model.
Python
2
star
42

ecstream

Erasure Coding over streams
JavaScript
2
star
43

httpagent

JavaScript
1
star
44

action-prom-render-test

1
star
45

zenko-demo-photobooth

A demo of Zenko multi-cloud replication and metadata search capabilities.
Python
1
star
46

zenkoclient

Zenko Client SDK
JavaScript
1
star
47

Droplet-tests

C
1
star
48

actions

Shell
1
star
49

jsmn

C
1
star
50

ghaudit

Github organisation security auditing tool
Python
1
star
51

kineticlib

Opensource IPDrive Kinetic Protocol (defined by openkinetic.org) Library for NodeJS
JavaScript
1
star
52

node-uks

JavaScript
1
star
53

vaultclient-go

Golang Vaultclient Library
Go
1
star
54

breakbeat

TypeScript
1
star
55

sphinx-tools

Scality theme and other tools for Sphinx documentation
JavaScript
1
star
56

bert-e

Python
1
star
57

vaultclient-java

Java
1
star
58

Droplet-tools

Tools related to Droplet
C
1
star
59

cosi-driver-sample

A sample provisioner for COSI, the Container Object Storage Interface
Go
1
star
60

scalgrafanalib

Python
1
star
61

workflow-engine-defs

Common Workflow Engine Definitions and Models for UI and Backbeat
JavaScript
1
star
62

action-artifacts

TypeScript
1
star
63

workflows

GitHub Actions reusable workflows
Dockerfile
1
star