• This repository has been archived on 22/Sep/2020
  • Stars
    star
    1,776
  • Rank 26,205 (Top 0.6 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created about 9 years ago
  • Updated almost 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Torus Distributed Storage

Torus

Build Status Go Report Card GoDoc

Torus Overview

Torus is an open source project for distributed storage coordinated through etcd.

Torus provides a resource pool and basic file primitives from a set of daemons running atop multiple nodes. These primitives are made consistent by being append-only and coordinated by etcd. From these primitives, a Torus server can support multiple types of volumes, the semantics of which can be broken into subprojects. It ships with a simple block-device volume plugin, but is extensible to more.

Quick-glance overview

Sharding is done via a consistent hash function, controlled in the simple case by a hash ring algorithm, but fully extensible to arbitrary maps, rack-awareness, and other nice features. The project name comes from this: a hash 'ring' plus a 'volume' is a torus.

Project Status

Development on Torus at CoreOS stopped as of Feb 2017. We started Torus as a prototype in June 2016 to build a storage system that could be easily operated on top of Kubernetes. We have proven out that model with this project. But, we didn't achieve the development velocity over the 8 months that we had hoped for when we started out, and as such we didn't achieve the depth of community engagement we had hoped for either.

If you have immediate storage needs Kubernetes can plugin to dozens of other storage options including AWS/Azure/Google/OpenStack/etc block storage, Ceph, Gluster, NFS, etc that are external to Kubernetes.

We are also seeing the emergence of projects, like rook, which creates a storage system that is ran on top of Kubernetes, as an Operator. We expect to see more systems like this in the future, because Kubernetes is a perfect platform for running distributed storage systems.

If you are interested in continuing the project feel free to fork and continue; we can update this README if a particular fork gets solid traction.

For further questions email [email protected].

Trying out Torus

To get started quicky using Torus for the first time, start with the guide to running your first Torus cluster, learn more about setting up Torus on Kubernetes using FlexVolumes in contrib, or create a Torus cluster on bare metal.

Contributing to Torus

Torus is an open source project and contributors are welcome! Join us on IRC at #coreos on freenode.net, file an issue here on Github, check out bigger plans on the kind/design tag, contribute on bugs that are low hanging fruit for issue ideas and check the project layout for a guide to the sections that might interest you.

Licensing

Unless otherwise noted, all code in the Torus repository is licensed under the Apache 2.0 license. Some portions of the codebase are derived from other projects under different licenses; the appropriate information can be found in the header of those source files, as applicable.

More Repositories

1

fleet

fleet ties together systemd and etcd into a distributed init system
Go
2,426
star
2

go-systemd

Go bindings to systemd socket activation, journal, D-Bus, and unit files
Go
2,232
star
3

etcd-operator

etcd operator creates/configures/manages etcd clusters atop Kubernetes
Go
1,751
star
4

coreos-vagrant

Minimal Vagrantfile for Container Linux
1,654
star
5

go-oidc

A Go OpenID Connect client.
Go
1,633
star
6

coreos-kubernetes

CoreOS Container Linux+Kubernetes documentation & Vagrant installers
Shell
1,105
star
7

go-iptables

Go wrapper around iptables utility
Go
1,096
star
8

rpm-ostree

⚛📦 Hybrid image/package system with atomic upgrades and package layering
C
859
star
9

vault-operator

Run and manage Vault on Kubernetes simply and securely
Go
759
star
10

ignition

First boot installer and configuration tool
Go
747
star
11

tectonic-installer

Install a Kubernetes cluster the CoreOS Tectonic Way: HA, self-hosted, RBAC, etcd Operator, and more
HCL
597
star
12

toolbox

bring your tools with you
Shell
395
star
13

go-etcd

DEPRECATED - please use the official client at https://github.com/coreos/etcd/tree/master/client
Go
365
star
14

coreos-cloudinit

[DEPRECATED] - Simple configuration tool for Container Linux
Go
344
star
15

grub

GRand Unified Bootloader http://www.gnu.org/software/grub/grub.html
C
333
star
16

coreos-xhyve

Container Linux running on xhyve hypervisor
Shell
332
star
17

coreos-assembler

Tooling container to assemble CoreOS-like systems
Go
327
star
18

go-semver

semver library in Go
Go
317
star
19

locksmith

Reboot manager for Container Linux
Go
273
star
20

fedora-coreos-tracker

Issue tracker for Fedora CoreOS
262
star
21

coreos-overlay

Custom ebuilds for Container Linux
Shell
249
star
22

coreos-installer

Installer for CoreOS disk images
Rust
218
star
23

container-linux-update-operator

A Kubernetes operator to manage updates of Container Linux by CoreOS
Go
211
star
24

fero

YubiHSM2-backed signing server
Rust
208
star
25

etcd-ca

Go
199
star
26

butane

Butane translates human-readable Butane Configs into machine-readable Ignition Configs.
Go
199
star
27

awesome-kubernetes-extensions

A resource tracking a number of Kubernetes extensions built on TPRs, CRDs, and API Aggregation
199
star
28

container-linux-config-transpiler

Convert a Container Linux Config into Ignition
Go
190
star
29

pkg

a collection of go utility packages
Go
186
star
30

afterburn

A one-shot cloud provider agent
Rust
183
star
31

etcdctl

DEPRECATED - see https://github.com/coreos/etcd/tree/master/etcdctl instead
181
star
32

discovery.etcd.io

etcd discovery service
Go
168
star
33

scripts

Build and maintenance scripts for Container Linux
Shell
155
star
34

zincati

Agent for Fedora CoreOS auto-updates
Rust
155
star
35

fedora-coreos-config

Base configuration for Fedora CoreOS
Shell
152
star
36

bugs

Issue tracker for CoreOS Container Linux
148
star
37

kpm

KPM is a tool to deploy and manage application stacks on Kubernetes.
Python
124
star
38

issue-sync

A tool for synchronizing issue tracking between GitHub and JIRA
Go
123
star
39

terraform-aws-kubernetes

Install a Kubernetes cluster the CoreOS Tectonic Way: HA, self-hosted, RBAC, etcd Operator, and more
HCL
117
star
40

bootupd

Bootloader updater
Rust
112
star
41

manifest

repo tool manifest for Container Linux sdk
99
star
42

go-omaha

omaha protocol for go
Go
96
star
43

init

init system units and configuration for Container Linux
Shell
95
star
44

mantle

Mantle: Gluing Container Linux together
Go
93
star
45

quartermaster

A framework for managing containerized storage systems on top of Kubernetes
Go
93
star
46

aws-auth-proxy

HTTP proxy that signs requests for upstream AWS endpoints
Go
86
star
47

mayday

A diagnostics tool for capturing system state.
Go
80
star
48

go-webrtc-datachannel

Go
77
star
49

go-gitreceive

A gitreceive implementation in Go
Go
75
star
50

torcx

torcx is a boot-time addon manager for immutable systems
Go
73
star
51

elb-presence

Python
72
star
52

layering-examples

Dockerfile
72
star
53

unit-examples

A collection of systemd units designed to run on CoreOS/fleet
65
star
54

go-workflow

Go
60
star
55

corelb

a loadbalancer built on coreinit and nginx
Lua
55
star
56

go-tcmu

Go SCSI emulation via the Linux TCM in Userspace module
Go
54
star
57

go-namespaces

DEPRECATED: Golang implementations of Linux Namespaces
Go
52
star
58

fedora-coreos-docs

Documentation for Fedora CoreOS
Shell
50
star
59

third_party.go

third_party.go - self contained GOPATH helper
Go
49
star
60

fedora-coreos-pipeline

Build pipeline for Fedora CoreOS
Groovy
49
star
61

updateservicectl

CoreUpdate Command Line Interface
Go
48
star
62

coreos-web

CSS
45
star
63

depot_tools

Python
43
star
64

jenkins-os

Groovy pipeline jobs that build and test Container Linux with Jenkins
Groovy
43
star
65

fabric-kubernetes-nodes

A fabric Fabfile for SSHing into Kubernetes nodes by label query
43
star
66

gocat

Socket activated transparent SSL proxy written in Go
Go
42
star
67

ssh-key-dir

sshd AuthorizedKeysCommand to read ~/.ssh/authorized_keys.d
Rust
42
star
68

tectonic-docs

Tectonic documentation - https://coreos.com/tectonic/docs/latest/
42
star
69

flannel-cni

Image for sidecar container that installs cni related assets for flannel
Shell
41
star
70

grafiti

Tag and remove AWS Resources with Automation
Go
39
star
71

go-log

Go logging library with systemd journal support
Go
38
star
72

update-ssh-keys

Deprecated tool for managing authorized ssh keys
Rust
38
star
73

airlock

Minimal update/reboot orchestrator for Fedora CoreOS clusters
Go
38
star
74

nsproxy

Linux namespaces tcp proxy
Go
36
star
75

ksched

Experimental flow-based Kubernetes scheduler
Go
36
star
76

cargo-vendor-filterer

Tool to `cargo vendor` with filtering
Rust
35
star
77

fedora-coreos-streams

Stream metadata and overrides for Fedora CoreOS
Python
35
star
78

update_engine

update daemon for Container Linux
C++
35
star
79

openssh-keys

A pure-Rust library to read and write OpenSSH public keys
Rust
35
star
80

bootengine

Initramfs for Container Linux
Shell
31
star
81

krud

kubernetes rolling update webhook server
Go
30
star
82

tectonic-forum

29
star
83

gzran

gzip indexer for random access into compressed files
Go
28
star
84

bcrypt-tool

Go
27
star
85

portage-stable

unmodified ebuilds mirrored from the portage tree
Shell
27
star
86

minikube-iso

An alternative bootable ISO image for minikube
27
star
87

khealth

basic kubernetes health monitoring
Go
26
star
88

kscale

Scripts for k8s scalability testing and analysis
Go
24
star
89

terraform-azurerm-kubernetes

Install a Kubernetes cluster the CoreOS Tectonic Way: HA, self-hosted, RBAC, etcd Operator, and more
HCL
22
star
90

coreos.fedoraproject.org

Old coreos.fedoraproject.org website (deprecated)
HTML
21
star
91

awscli

AWS CLI container image
21
star
92

baselayout

Basic Container Linux filesystem layout and configs
Shell
20
star
93

subgun

Subscribe to a mailing list on mailgun via a web interface
Go
20
star
94

etcd-manager

An etcd cluster management tool
Go
19
star
95

license-bill-of-materials

Fork of https://github.com/pmezard/licenses
Go
19
star
96

systemd-rest

Go
19
star
97

kapprover

A kubelet CSR auto-approver
Go
18
star
98

enhancements

Enhancement tracking repo for CoreOS-based systems
17
star
99

docker-nginx-https-redirect

Docker container which redirects any http request on 80 to https on 443
17
star
100

envsubst-rs

Simple Rust library for variables substitution
Rust
17
star