• Stars
    star
    7,604
  • Rank 4,755 (Top 0.1 %)
  • Language
    C++
  • License
    Apache License 2.0
  • Created over 9 years ago
  • Updated 7 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

High performance server-side application framework

Seastar

CircleCI Version License: Apache2 n00b issues

Introduction

SeaStar is an event-driven framework allowing you to write non-blocking, asynchronous code in a relatively straightforward manner (once understood). It is based on futures.

Building Seastar

For more details and alternative work-flows, read HACKING.md.

Assuming that you would like to use system packages (RPMs or DEBs) for Seastar's dependencies, first install them:

$ sudo ./install-dependencies.sh

then configure (in "release" mode):

$ ./configure.py --mode=release

then compile:

$ ninja -C build/release

In case there are compilation issues, especially like g++: internal compiler error: Killed (program cc1plus) try giving more memory to gcc, either by limiting the amount of threads ( -j1 ) and/or allowing at least 4g ram to your machine.

If you're missing a dependency of Seastar, then it is possible to have the configuration process fetch a version of the dependency locally for development.

For example, to fetch fmt locally, configure Seastar like this:

$ ./configure.py --mode=dev --cook fmt

--cook can be repeated many times for selecting multiple dependencies.

Build modes

The configure.py script is a wrapper around cmake. The --mode argument maps to CMAKE_BUILD_TYPE, and supports the following modes

CMake mode Debug info Optimi­zations Sanitizers Allocator Checks Use for
debug Debug Yes -O0 ASAN, UBSAN System All gdb
release RelWithDebInfo Yes -O3 None Seastar Asserts production
dev Dev (Custom) No -O1 None Seastar Asserts build and test cycle
sanitize Sanitize (Custom) Yes -Os ASAN, UBSAN System All second level of tests, track down bugs

Note that seastar is more sensitive to allocators and optimizations than usual. A quick rule of the thumb of the relative performances is that release is 2 times faster than dev, 150 times faster than sanitize and 300 times faster than debug.

Using Seastar from its build directory (without installation)

It's possible to consume Seastar directly from its build directory with CMake or pkg-config.

We'll assume that the Seastar repository is located in a directory at $seastar_dir.

Via pkg-config:

$ g++ my_app.cc $(pkg-config --libs --cflags --static $seastar_dir/build/release/seastar.pc) -o my_app

and with CMake using the Seastar package:

CMakeLists.txt for my_app:

find_package (Seastar REQUIRED)

add_executable (my_app
  my_app.cc)
  
target_link_libraries (my_app
  Seastar::seastar)
$ mkdir $my_app_dir/build
$ cd $my_app_dir/build
$ cmake -DCMAKE_PREFIX_PATH="$seastar_dir/build/release;$seastar_dir/build/release/_cooking/installed" -DCMAKE_MODULE_PATH=$seastar_dir/cmake $my_app_dir

The CMAKE_PREFIX_PATH values ensure that CMake can locate Seastar and its compiled submodules. The CMAKE_MODULE_PATH value ensures that CMake can uses Seastar's CMake scripts for locating its dependencies.

Using an installed Seastar

You can also consume Seastar after it has been installed to the file-system.

Important:

  • Seastar works with a customized version of DPDK, so by default builds and installs the DPDK submodule to $build_dir/_cooking/installed

First, configure the installation path:

$ ./configure.py --mode=release --prefix=/usr/local

then run the install target:

$ ninja -C build/release install

then consume it from pkg-config:

$ g++ my_app.cc $(pkg-config --libs --cflags --static seastar) -o my_app

or consume it with the same CMakeLists.txt as before but with a simpler CMake invocation:

$ cmake ..

(If Seastar has not been installed to a "standard" location like /usr or /usr/local, then you can invoke CMake with -DCMAKE_PREFIX_PATH=$my_install_root.)

There are also instructions for building on any host that supports Docker.

Use of the DPDK is optional.

Seastar's C++ standard: C++17 or C++20

Seastar supports both C++17, and C++20. The build defaults to the latest standard supported by your compiler, but can be explicitly selected with the --c++-standard configure option, e.g., --c++-standard=17, or if using CMake directly, by setting on the CMAKE_CXX_STANDARD CMake variable.

See the compatibity statement for more information.

Getting started

There is a mini tutorial and a more comprehensive one.

The documentation is available on the web.

Resources

Ask questions and post patches on the development mailing list. Subscription information and archives are available here, or just send an email to [email protected].

Information can be found on the main project website.

File bug reports on the project issue tracker.

The Native TCP/IP Stack

Seastar comes with its own userspace TCP/IP stack for better performance.

Recommended hardware configuration for SeaStar

  • CPUs - As much as you need. SeaStar is highly friendly for multi-core and NUMA
  • NICs - As fast as possible, we recommend 10G or 40G cards. It's possible to use 1G too but you may be limited by their capacity. In addition, the more hardware queue per cpu the better for SeaStar. Otherwise we have to emulate that in software.
  • Disks - Fast SSDs with high number of IOPS.
  • Client machines - Usually a single client machine can't load our servers. Both memaslap (memcached) and WRK (httpd) cannot over load their matching server counter parts. We recommend running the client on different machine than the servers and use several of them.

Projects using Seastar

  • cpv-cql-driver: C++ driver for Cassandra/Scylla based on seastar framework
  • cpv-framework: A web framework written in c++ based on seastar framework
  • redpanda: A Kafka replacement for mission critical systems
  • Scylla: A fast and reliable NoSQL data store compatible with Cassandra and DynamoDB
  • smf: The fastest RPC in the West

More Repositories

1

scylladb

NoSQL data store using the seastar framework, compatible with Apache Cassandra
C++
12,344
star
2

gocqlx

All-In-One: CQL query builder, ORM and migration tool
Go
868
star
3

go-set

Type-safe, zero-allocation sets for Go
Go
804
star
4

scylla-rust-driver

Async CQL driver for Rust, optimized for Scylla!
Rust
517
star
5

scylla-operator

The Kubernetes Operator for ScyllaDB
Go
312
star
6

charybdefs

ScyllaDB fault injection filesystem
C++
241
star
7

scylla-code-samples

Code samples for working with ScyllaDB
Python
222
star
8

diskplorer

Python
210
star
9

scylla-monitoring

Simple monitoring of Scylla with Grafana
Shell
203
star
10

dpdk

Mirror of Data Plane Development Kit, git://dpdk.org/dpdk (http://dpdk.org)
C++
173
star
11

scylla-go-driver

Experimental, high performance Scylla Driver, University of Warsaw students' project
Go
146
star
12

termtables

Fork of github.com/apcera/termtables
Go
55
star
13

scylla-migrator

Migrate data extract using Spark to Scylla, normally from Cassandra
Scala
50
star
14

scylla-cluster-tests

Tests for Scylla Clusters
Python
50
star
15

scylla-tools-java

Apache Cassandra, supplying tools for Scylla
Java
49
star
16

gaming-leaderboard-demo

Learn how to use ScyllaDB to generate a monstrously fast leadearboard for your application!
TypeScript
48
star
17

scylla-manager

The Scylla Manager
Go
42
star
18

kafka-connect-scylladb

Kafka Connect Scylladb Sink
Java
42
star
19

scylla-ansible-roles

Ansible roles for deploying and managing Scylla, Scylla-Manager and Scylla-Monitoring
Python
39
star
20

scylla-bench

Go
39
star
21

scylla-cdc-source-connector

A Kafka source connector capturing Scylla CDC changes
Java
38
star
22

scylla-cloud-getting-started

Rust
38
star
23

scylla-cdc-go

Go
37
star
24

care-pet

Care Pet IoT ScyllaDB example
JavaScript
37
star
25

cassandra-test-and-deploy

Python
30
star
26

scylla-cdc-rust

Rust
27
star
27

gemini

Test data integrity by comparing against an Oracle running in parallel
Go
27
star
28

scylla-seastar

Stable-branch fork of seatar, for scylla use
C++
26
star
29

scylla-jmx

Scylla JMX proxy
Java
25
star
30

go-reflectx

Go reflection library to find struct field by its tag
Go
24
star
31

scylla-graphite-monitoring

Python
20
star
32

scylla-cdc-java

Java
20
star
33

scylla-ccm

Cassandra Cluster Manager, modified for Scylla
Python
18
star
34

scylla-machine-image

Python
17
star
35

scylla-grafana-datasource

A grafana backend plugin for ScyllaDB
TypeScript
17
star
36

scylladb-feature-store

A feature store sample application built with ScyllaDB
Jupyter Notebook
14
star
37

jepsen

scylladb jepsen clone
Clojure
14
star
38

alternator-load-balancing

Various tricks, scripts, and libraries, for load balancing multiple Alternator nodes
Java
14
star
39

sphinx-scylladb-theme

The documentation toolchain for Scylla projects.
Python
12
star
40

cpp-rust-driver

API-compatible rewrite of https://github.com/scylladb/cpp-driver as a wrapper for Rust driver.
C++
11
star
41

go-sshtools

Go SSH client wrapper and other tools
Go
11
star
42

video-streaming

Video streaming sample app with NextJs + ScyllaDB
TypeScript
10
star
43

terraform-provider-scylladbcloud

Terraform provider for ScyllaDB Cloud.
Go
10
star
44

scylla-rust-udf

Rust
10
star
45

scylla-cqlsh

A fork of the cqlsh code
Python
9
star
46

sstable-tools

Python
7
star
47

scylla-ami

Python
7
star
48

1m-ops-demo

1 million operations/second ScyllaDB demo
HCL
6
star
49

scylla-swagger-ui

Private fork of https://github.com/swagger-api/swagger-ui for scylla
JavaScript
6
star
50

scylla-python3

Python
6
star
51

scylla-stress-orchestrator

Python
5
star
52

cql-stress

Rust
5
star
53

scylla-price-calculator

Compare Scylla Cloud cost to estimations of other DBaaS services cost. Data is served with no guarantees. Please validate with each DBaaS price list per region.
Vue
5
star
54

argus

Svelte
4
star
55

go-log

Wrapper over Uber Zap
Go
4
star
56

scylla-api-client

Scylla CLI Python module and command line utility
Python
4
star
57

jenkins-slave

Jenkins slaves as docker images
4
star
58

k8s-local-volume-provisioner

Go
3
star
59

python-driver-matrix

Groovy
3
star
60

seastar-website

Seastar static website
HTML
3
star
61

scylladb-web-install

Shell
3
star
62

scylla-java-driver-matrix

Groovy
3
star
63

libdeflate

Heavily optimized library for DEFLATE/zlib/gzip compression and decompression
C
2
star
64

scylla-doc-issues

Repository for reporting issues about Scylla documentation (Deprecated)
2
star
65

scylla-artifact-tests

Tests for the scylla distro packages
Python
2
star
66

scylla-cluster-autoscaler

Go
2
star
67

scylla-operator-images

This repository contains the source for shared images used by Scylla Operator projects. We do not provide any support or compatibility guaranties for these images as their sole purpose is only to provide a common base in our projects.
Dockerfile
2
star
68

scylla-cpp-driver-matrix

Groovy
1
star
69

scylla-build-dependencies-docker

Dockerfiles for generating images with scylla build dependencies
1
star
70

scylladb-cloud-doc-issues

A repo for Scylla Cloud docs issues
1
star