• Stars
    star
    562
  • Rank 76,126 (Top 2 %)
  • Language
    Swift
  • License
    Apache License 2.0
  • Created almost 5 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Peer-to-peer cluster implementation for Swift Distributed Actors

Swift Distributed Actors

A peer-to-peer cluster actor system implementation for Swift.

NOTE: This project provides a cluster runtime implementation for the distributed actors language feature.

Introduction

Beta software

Important: This library is currently released as beta preview software. While we anticipate very few changes, please be mindful that until a stable 1.0 version is announced, this library does not guarantee source compatibility.

We anticipate to release a number of 1.0.0-beta-n releases during the beta phase of Swift 5.7, before releasing a source stable 1.0 soon after.

Most APIs and runtime are rather stable, and have proven itself for a long time already. Most of the remaining work is around making sure all APIs work nicely with the latest revision of the distributed actor language feature.

Important: Please ignore and do not use any functionality that is prefixed with an _ (such as types and methods), as they are intended to be removed as we reach the stable 1.0 release.

What are Distributed Actors?

Distributed actors are an extension of the "local only" actor model offered by Swift with its actor keyword.

Distributed actors are declared using the distributed actor keywords (and importing the Distributed module), and enable the declaring of distributed func methods inside such actor. Such methods may then be invoked remotely, from other peers in a distributed actor system.

The distributed actor language feature does not include any specific runtime, and only defines the language and semantic rules surrounding distributed actors. This library provides a feature-rich clustering server-side focused implementation of such runtime (i.e. a DistributedActorSystem implementation) for distributed actors.

To learn more about both the language feature and library, please refer to the reference documentation of this project.

The primary purpose of open sourcing this library early is proving the ability to implement a feature complete, compelling clustering solution using the distributed actor language feature, and co-evolving the two in tandem.

Samples

You can refer to the Samples/ directory to view a number of more realistic sample apps which showcase how distributed actors can be used in a cluster.

The most "classical" example of distributed actors is the SampleDiningPhilosophers.

You can run it all in a single node (run --package-path Samples/ SampleDiningPhilosophers), or in 3 cluster nodes hosted on the same physical machine: run --package-path Samples/ SampleDiningPhilosophers distributed. Notice how one does not need to change implementation of the distributed actors to run them in either "local" or "distributed" mode.

Documentation

Please refer to the rendered docc reference documentation to learn about distributed actors and how to use this library and its various features.

Note: Documentation is still work in progress, please feel free to submit issues or patches about missing or unclear documentation.

Development

This library requires beta releases of Swift (5.7+) and Xcode to function property as the distributed actor feature is part of that Swift release.

When developing on macOS, please also make sure to update to the latest beta of macOS, as some parts of the Swift runtime necessary for distributed actors to work are part of the Swift runtime library which is shipped with the OS.

Distributed actors are not back-deployed and require the latest versions of iOS, macOS, watchOS etc.

When developing on Linux systems, you can download the latest Swift 5.7 toolchains from swift.org/downloads, and use it to try out or run the library like this:

$ export TOOLCHAIN=/path/to/toolchain
$ $TOOLCHAIN/usr/bin/swift test
$ $TOOLCHAIN/usr/bin/swift run --package-path Samples/ SampleDiningPhilosophers dist

IDE Support: Xcode

Latest (beta) Xcode releases include complete support for the distributed language syntax (distributed actor, distributed func), so please use the latest Beta Xcode available to edit the project and any projects using distributed actors.

IDE Support: Other IDEs

It is possible to open and edit this project in other IDEs, however most IDEs have not yet caught up with the latest language syntax (i.e. distributed actor) and therefore may have trouble understanding the new syntax.

VSCode

You can use the Swift Server Work Group maintained VSCode plugin to edit this project from VSCode.

You can install the VSCode extension from here.

The extension uses sourcekit-lsp and thus should be able to highlight and edit distributed actor using sources just fine. If it does not, please report issues!

CLion

The project is possible to open in CLion as a SwiftPM package project, however CLion and AppCode do not yet support the new distributed syntax, so they might have issues formatting the code until this is implemented.

See also the following guides by community members about using CLion for Swift development:

Warnings

The project currently is emitting many warnings about Sendable, this is expected and we are slowly working towards removing them.

Much of the project's internals use advanced synchronization patterns not recognized by sendable checks, so many of the warnings are incorrect but the compiler has no way of knowing this. We will be removing much of these internals as we move them to use the Swift actor runtime instead.

Documentation workflow

Documentation for this project is using the Doc Compiler, via the SwiftPM DocC plugin.

If you are not familiar with DocC syntax and general style, please refer to its documentation: https://developer.apple.com/documentation/docc

The project includes two helper scripts, to build and preview documentation.

To build documentation:

./scripts/docs/generate_docc.sh

And to preview and browse the documentation as a web-page, run:

./scripts/docs/preview_docc.sh

Which will result in an output similar to this:

========================================
Starting Local Preview Server
	          http://localhost:8000/documentation/distributecluster

Integration tests

Integration tests include running actual multiple nodes of a cluster and e.g. killing them off to test the recovery mechanisms of the cluster.

Requirements:

  • macOS: brew install coreutils to install stdbuf

Supported Versions

This project requires Swift 5.7+.

More Repositories

1

swift

The Swift Programming Language
C++
65,899
star
2

ml-stable-diffusion

Stable Diffusion with Core ML on Apple Silicon
Python
16,002
star
3

swift-evolution

This maintains proposals for changes and user-visible enhancements to the Swift Programming Language.
Markdown
15,013
star
4

foundationdb

FoundationDB - the open source, distributed, transactional key-value store
C++
13,947
star
5

turicreate

Turi Create simplifies the development of custom machine learning models.
C++
11,153
star
6

darwin-xnu

The Darwin Kernel (mirror). This repository is a pure mirror and contributions are currently not accepted via pull-requests, please submit your contributions via https://developer.apple.com/bug-reporting/
C
10,558
star
7

swift-package-manager

The Package Manager for the Swift Programming Language
Swift
9,592
star
8

ml-ferret

Python
7,576
star
9

swift-nio

Event-driven network application framework for high performance protocol servers & clients, non-blocking.
Swift
7,274
star
10

swift-algorithms

Commonly used sequence and collection algorithms for Swift
Swift
5,622
star
11

swift-corelibs-foundation

The Foundation Project, providing core utilities, internationalization, and OS independence
Swift
5,189
star
12

swift-protobuf

Plugin and runtime library for using protobuf with Swift
Swift
4,446
star
13

password-manager-resources

A place for creators and users of password managers to collaborate on resources to make password management better.
JavaScript
4,010
star
14

coremltools

Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
Python
3,974
star
15

ml-mgie

Python
3,682
star
16

tensorflow_macos

TensorFlow for macOS 11.0+ accelerated using Apple's ML Compute framework.
Shell
3,643
star
17

swift-collections

Commonly used data structures for Swift
Swift
3,434
star
18

pkl

A configuration as code language with rich validation and tooling.
Java
3,360
star
19

swift-argument-parser

Straightforward, type-safe argument parsing for Swift
Swift
3,163
star
20

sourcekit-lsp

Language Server Protocol implementation for Swift and C-based languages
Swift
3,110
star
21

swift-log

A Logging API for Swift
Swift
2,931
star
22

swift-syntax

A set of Swift libraries for parsing, inspecting, generating, and transforming Swift source code.
Swift
2,887
star
23

swift-async-algorithms

Async Algorithms for Swift
Swift
2,695
star
24

swift-markdown

A Swift package for parsing, building, editing, and analyzing Markdown documents.
Swift
2,586
star
25

HomeKitADK

C
2,456
star
26

ml-ane-transformers

Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)
Python
2,431
star
27

swift-corelibs-libdispatch

The libdispatch Project, (a.k.a. Grand Central Dispatch), for concurrency on multicore hardware
C
2,420
star
28

swift-format

Formatting technology for Swift source code
Swift
2,261
star
29

homebrew-apple

Ruby
2,227
star
30

swift-foundation

The Foundation project
Swift
2,088
star
31

cups

Apple CUPS Sources
C
1,828
star
32

sample-food-truck

SwiftUI sample code from WWDC22
Swift
1,695
star
33

ml-fastvit

This repository contains the official implementation of the research paper, "FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization" ICCV 2023
Python
1,693
star
34

ml-cvnets

CVNets: A library for training computer vision networks
Python
1,664
star
35

swift-book

The Swift Programming Language book
Markdown
1,616
star
36

swift-numerics

Advanced mathematical types and functions for Swift
Swift
1,602
star
37

ml-hypersim

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Python
1,495
star
38

swift-crypto

Open-source implementation of a substantial portion of the API of Apple CryptoKit suitable for use on Linux platforms.
C
1,400
star
39

swift-docker

Docker Official Image packaging for Swift
Dockerfile
1,331
star
40

ml-neuman

Official repository of NeuMan: Neural Human Radiance Field from a Single Video (ECCV 2022)
Python
1,233
star
41

swift-openapi-generator

Generate Swift client and server code from an OpenAPI document.
Swift
1,142
star
42

swift-system

Low-level system calls and types for Swift
Swift
1,137
star
43

swift-corelibs-xctest

The XCTest Project, A Swift core library for providing unit test support
Swift
1,120
star
44

swift-docc

Documentation compiler that produces rich API reference documentation and interactive tutorials for your Swift framework or package.
Swift
1,093
star
45

swift-llbuild

A low-level build system, used by Xcode and the Swift Package Manager
C++
1,067
star
46

swift-atomics

Low-level atomic operations for Swift
Swift
1,004
star
47

swift-testing

Swift
981
star
48

servicetalk

A networking framework that evolves with your application
Java
881
star
49

swift-http-types

Version-independent HTTP currency types for Swift
Swift
815
star
50

swift-llvm

LLVM
815
star
51

swift-driver

Swift compiler driver reimplementation in Swift
Swift
764
star
52

swift-protobuf-plugin

Moved to apple/swift-protobuf
757
star
53

swift-lldb

This is the version of LLDB that supports the Swift programming language & REPL.
C++
673
star
54

swift-clang

C++
673
star
55

unityplugins

C#
645
star
56

ml-mobileone

This repository contains the official implementation of the research paper, "An Improved One millisecond Mobile Backbone".
Swift
641
star
57

ml-gaudi

602
star
58

ml-aim

This repository provides the code and model checkpoints of the research paper: Scalable Pre-training of Large Autoregressive Image Models
Python
602
star
59

swift-metrics

Metrics API for Swift
Swift
602
star
60

axlearn

Python
564
star
61

ARKitScenes

This repo accompanies the research paper, ARKitScenes - A Diverse Real-World Dataset for 3D Indoor Scene Understanding Using Mobile RGB-D Data and contains the data, scripts to visualize and process assets, and training code described in our paper.
Python
552
star
62

sample-backyard-birds

Swift
506
star
63

device-management

Device management schema data for MDM.
506
star
64

ccs-calendarserver

The Calendar and Contacts Server.
Python
470
star
65

ml-facelit

Official repository of FaceLit: Neural 3D Relightable Faces (CVPR 2023)
Python
457
star
66

swift-3-api-guidelines-review

Swift
452
star
67

swift-org-website

Swift.org website
SCSS
438
star
68

GCGC

Jupyter Notebook
436
star
69

swift-nio-http2

HTTP/2 support for SwiftNIO
Swift
405
star
70

swift-tools-support-core

Contains common infrastructural code for both SwiftPM and llbuild.
Swift
390
star
71

swift-nio-ssh

SwiftNIO SSH is a programmatic implementation of SSH using SwiftNIO
Swift
364
star
72

swift-nio-ssl

TLS Support for SwiftNIO, based on BoringSSL.
C
345
star
73

ml-gmpi

Official PyTorch implementation of GMPI (ECCV 2022, Oral Presentation)
Python
329
star
74

example-package-dealer

Example package for use with the Swift Package Manager
Swift
319
star
75

swift-collections-benchmark

A benchmarking tool for Swift Collection algorithms
Swift
316
star
76

example-package-playingcard

Example package for use with the Swift Package Manager
Swift
308
star
77

swift-docc-render

Web renderer for Swift-DocC documentation.
JavaScript
300
star
78

indexstore-db

Index database library for use with sourcekit-lsp
C++
299
star
79

swift-playdate-examples

A technical demonstration of Embedded Swift running on Playdate by Panic
Swift
295
star
80

swift-docc-plugin

Swift Package Manager command plugin for Swift-DocC
Swift
295
star
81

ml-hierarchical-confusion-matrix

Neo: Hierarchical Confusion Matrix Visualization (CHI 2022)
TypeScript
292
star
82

ml-gsn

Python
284
star
83

swift-llbuild2

A fresh take on a low-level build system API.
Swift
280
star
84

swift-source-compat-suite

The infrastructure and project index comprising the Swift source compatibility suite.
Python
278
star
85

sample-cloudkit-sharing

Swift
275
star
86

swift-xcode-playground-support

Logging and communication to allow Swift toolchains to communicate with Xcode.
Swift
270
star
87

swift-experimental-string-processing

An early experimental general-purpose pattern matching engine for Swift.
Swift
263
star
88

ml-sigma-reparam

Python
255
star
89

swift-standard-library-preview

Swift
253
star
90

swift-nio-transport-services

Extensions for SwiftNIO to support Apple platforms as first-class citizens.
Swift
252
star
91

swift-stress-tester

Stress testing utilities for Swift's tooling
Swift
207
star
92

swift-service-discovery

A service discovery API for Swift.
Swift
203
star
93

swift-certificates

An implementation of X.509 for Swift
Swift
195
star
94

swift-nio-examples

examples of how to use swift-nio
Swift
195
star
95

swift-cluster-membership

Distributed Membership Protocol implementations in Swift
Swift
191
star
96

swift-aoc-starter-example

Swift starter project for solving Advent of Code challenges.
Swift
189
star
97

sample-cloudkit-coredatasync

Swift
187
star
98

swift-distributed-tracing

Instrumentation library for Swift server applications
Swift
186
star
99

pfl-research

Simulation framework for accelerating research in Private Federated Learning
Python
186
star
100

swift-internals

HTML
182
star