• Stars
    star
    167
  • Rank 226,635 (Top 5 %)
  • Language
    TeX
  • Created about 6 years ago
  • Updated about 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Messaging for Web3

A Decentralised Privacy-Preserving Communication Protocol

Messaging for Web3.

Motivation

We need communications protocols that protect metadata, partially as an ethical end goal and partially for business reasons, but also as a fundamental building block of decentralised applications.

We cannot control all information asymmetries, especially the metadata observed by large infrastructure providers. Yet, information asymmetry or symmetry are an extremely important assumption in economics models. We thus consider increasing our control over information asymmetry to be an essential building block, which makes privacy a crucial tool, including metadata protection.

We have witnessed adoption of centralised messaging solutions being influenced by users' perceptions of privacy, at least at a personal level. We thus believe providing real metadata protections should help establish a stable user base, which may help solidify or stabilise other services provided by the protocol, including associated financial services.

We also consider privacy to be a fundamental human right. Article 12 of the Universal Declaration of Human Rights names "privacy" and "correspondence" for protection, which logically covers metadata.

No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks.

We also believe that user anonimity and stronger privacy protections aid adoption.

If we lack metadata protections, then infrastructure providers will act as adversaries who collect and abuse user data in a myriad of ways, such as by adjusting prices, employing strategic voting, front running, etc.

Whisper

In the current decentralised application landscape, we are seeing projects struggle to achieve mainstream adoption. We believe that the underlying protocols do not have sufficient capability to enable the necessary level of adoption. Part of the problem is that these applications often require the exchange of transient messages; however, it does not make sense to transmit these messages via a blockchain. Gavin Wood realised this problem in the early days of Ethereum, and suggested that DApps would require a decentralised messaging protocol that would provide this capability. This protocol is called Whisper.

Unfortunately, the evolution and adoption of Whisper has been stunted, which is despite the rapid advancement of applications. The lack of development means that Whisper is not able to provide the scaling that we need, nor is it feasible for projects to create their own bespoke messaging protocol. What we really need is a new protocol to handle transient messaging at scale. We believe this is a necessary cornerstone for enabling the mainstream adoption of DApps.

Project

We would like to gather a number of projects together to align and support this protocol:

  • Researchers
    • Academia
    • Web 3 projects
  • Protocol implementers
    • Core dev teams from Web 3 space
  • Application builders
    • User messaging application
    • State channels
    • Latency-agnostic streaming protocols
    • Other applications requiring transient messaging

The goal is to end up with at least one viable implementation, spec and a theoretical analysis of the protocol properties.

The idea is to start with gathering requirements and aligning on goals, the Web3 Foundation (W3F) has discussed with a number of projects to better understand what the needs are. W3F has also started to do an initial exploration of potential components to be used in the protocol.

Plan (to be evolved)

  1. Refine this document to reflect the motivation, requirements for the project and initial mapping of the space until initial contributors are happy with it.
  2. W3F to organise a workshop with all the relevant parties
  3. Come up with clear work packages to be done by all contributors.
  4. Readjust the plan together with the project contributors.
  5. Achieve the goal.

Project contributors (in alphabetical order)

To contribute your project will need to commit some time to help specifying the motivation, requirements and mapping of the solution space. Following that contributors will be invited to participate in a joint workshop.

Role of the protocol

Layer Purpose Examples
Application Application logic Chat app
Storage / Sync Sync data, make messages persistent
-> Protocol, DHT <- Scalable, decentralised metadata protection
P2P Overlay routing, NAT traversal libp2p, WebRTC
Network Underlay routing TCP / IP

Adversary model

For the adversary model, see a detailed description

Protocol requirements

For a more detailed description, see a detailed description of the below listed requirements.

Metadata protection:

1. Sender Anonymity (who sent a message?)

2. Receiver Anonymity (who read a message?)

3. Sender-Receiver Unlinkability (who is talking to whom?)

Convenience, Usability:

4. Reasonable Latency (<5s, to allow for IM [XXX])

5. Reasonable Bandwidth (not specified, mobile data plan in undeveloped countries)

6. Adaptable Anonymity (adjustable resource consumption)

Decentralization:

7. Scalable (up to, say, ~1M active nodes)

8. No Specialized Services (pure p2p)

Incentives to achieve mass adoption:

9. Incentivisation for relayers (not necessarily economical)

Things that are explicitly out of scope

  • Trust Establishment - provenance of long term keys to some known identity
  • Conversational Security - authentication, confidentiality, integrity, perfect forward secrecy, accountability.

Additionally, see below for other things that may be out of scope at this layer.

Questions

What about Incentives?

The Protocol will be used in conjunction with Ethereum and similar technologies. There's also a strong need for incentive-compatible designs. This means it is useful to consider incentives and payment mechanisms as the protocol layer. However, an ideal protocol suite should be layered and have a clear separation of concern. This means there's a simple design with minimal dependencies. As inspiration, see Bittorrent economics paper (pdf), which is a separate protocol layer that people can choose to use or not to get better quality of service (request and choking).

What about Message Reliability?

The protocol should do Best Effort Delivery. Reliable Delivery can be provided on top, similar to TCP/IP (or BSP/BTP for Briar), to accommodate things like:

  • Guaranteed Message Delivery
  • Message Ordering
  • And possibly: Asynchronous Messaging (some protocols deal with it at this layer, so this may or may not be desirable)

Depending on the specifics of the reliability mechanism, throughput etc, this may have consequences for the above Reasonable Latency requirement. The Reasonable Latency requirement outlined above is for End to End messaging.

How to deal with Network Spam?

  • One approach is to use a Friend-to-Friend (F2F) network. This is what Briar and Secure Scuttlebutt (SSB) does.
  • For open DHT-based, another approach is to rely on proof of work like Whisper. This isn't very practical for mobile / resource restricted devices, and appears to have limited usability.
  • More approaches are likely possible, such as traditional rate limiting, basic peer reputation, payments, etc.
  • Global network attacks more relevant here than a specific node. What does this imply for a DHT?

How to deal with Asynchronous Messaging?

  • One approach is to punt this problem to data sync layer.
  • Another example is Briar requiring two entities to both be online
  • Not clear that it is a necessary component of AC layer.
  • One idea is to use Aggregation Points (Xolotl, lake mixnet) as providers similar to Loopix, but presumably with less HA guarantees.

Initial work

Network

We would like to develop a protocol that leverages the best research and protocols from the past in order to create something for the future of decentralised applications. We want a “roadmap without potholes” for providing stronger privacy assurances than Tor for both senders and receivers, while also scaling well and providing short-term message storage for offline users. In other words, we accept that a project this large requires a piecemeal approach but we shall understand and avoid design dead ends that prevent either scaling or rigorous analysis of anonymity properties.

We think mix networks occupy a sweet spot in which nodes run extremely efficiently but rigorous analysis remains possible, if challenging. There are designs like Tor with more efficient designs, but they cannot provide rigorous anonymity properties. There are also numerous academic schemes designed to support rigorous analysis, but at extreme sacrifices in efficiency. We choose a Sphinx-like packet format because it’s efficient, adaptable enough, and has good security proofs.

We leave actually analyzing our scheme for future work in collaboration with academics. We shall however use Poison mixing and cover traffic strategy similar to Loopix, which appears analyzable although academic work to do so remains ongoing. We concede that mix networks impose latency on users, but any faster design would definitely not admit rigorous analysis of anonymity properties, and could not credibly claim to be stronger than Tor.

We shall tweak Sphinx slightly to accommodate both receiver anonymity and short-term message storage simultaneously, which may require updating its security proofs eventually. We consider short-term message storage essential to user experience and advise against doing it via a second layer protocol.

We foresee the public key infrastructure (PKI) being the ultimate scaling bottleneck for all existing anonymity scheme designs. We could avoid this with gossip protocols but these enable epistemic attacks. We largely leave this to future work, but suggest investigating a verifiable gossip based protocol. We shall therefore use an insecure gossip based protocol initially with the hope that it meshes best with later designs. If we used a more secure design inspired by the Tor consensus, then we might make assumptions elsewhere that limit scalability later.

We know rewards for node operators remains a contentious question in the anonymity community with seemingly unforeseeable consequences. We nevertheless think rewards represents our best hope for a network large enough to challenge today’s centralized providers that operate on surveillance capitalism. We do not imagine rewards obliviate the need to steward relay operator culture, possibly quite the opposite.

Messaging types

We’re focusing on one-to-one messaging for now. We actually do require messaging layer crypto, even after all the mix net layers, so expect an Axolotl-like ratchet for this.

We can adapt our short-term message storage plans for small group messaging, but not with exactly the same privacy assurances. We leave designing this to future work.

We think one-to-mass messaging should be done by using the mix network to send to a broadcast protocol like Whisper v1 or perhaps a blockchain.

Payment

We’re designing an accounting scheme to prevent abuse and reward nodes, without damaging users’ anonymity. We’re currently working on several designs based on fundamentally different methodologies, primarily payment channels, blind signatures, and secret shopper, so as to more fairly evaluate them. We’re keeping these designs as agnostic as possible to questions like if the users actually pay anything ever.

Implementation

In order to leverage existing work done in the space we would like to leverage libp2p for networking and make sure that at least one implementation is fully runnable in the browser leveraging Javascript and Wasm.

Copyright

Copyright and related rights waived via CC0.

More Repositories

1

Grants-Program

Web3 Foundation Grants Program
JavaScript
1,018
star
2

General-Grants-Program

Web3 Foundation General Grants Program
590
star
3

polkadot-wiki

The source of truth for Polkadot.
JavaScript
370
star
4

schnorrkel

Schnorr VRFs and signatures on the Ristretto group
Rust
309
star
5

polkadot-validator-setup

Polkadot Validator Secure Setup
JavaScript
215
star
6

polkadot-spec

The Polkadot Protocol Specification
TeX
178
star
7

PSPs

Polkadot Smart Contract Proposals
152
star
8

unbounded

Open source, freely available and on-chain funded font.
152
star
9

Grant-Milestone-Delivery

Repository to submit finished milestones
104
star
10

polkadot-deployer

Tool for deploying polkadot networks
JavaScript
101
star
11

research

Overview of W3F research initatives
JavaScript
95
star
12

consensus

Consensus for Web3
TeX
88
star
13

staking-rewards-collector

JavaScript
78
star
14

bls

Aggregatable BLS sigantures
Rust
65
star
15

1k-validators-be

Thousand Validators Program backend.
TypeScript
63
star
16

polkadot-wiki-old

The Polkadot wiki.
HTML
55
star
17

apk-proofs

Rust
50
star
18

polkadot-legacy-spec

A more technical description of Polkadot protocol
47
star
19

ring-vrf

TeX
36
star
20

jamtestvectors

The latest test vectors for JAM.
Python
31
star
21

substrate-telemetry-exporter

JavaScript
30
star
22

polkadot

Rust
30
star
23

hd-ed25519

Hierarchical derivations on Ed25519
Rust
25
star
24

polkadot-charts

Helm charts for deploying Polkadot networks.
Smarty
22
star
25

CardsAgainstBlockchain

Cards Against Blockchain
TeX
22
star
26

mooc-exercises

Exercises for Web3 MOOC
Rust
21
star
27

fflonk

Rust
21
star
28

validator-security

A collaborative document for good practice with validator security
20
star
29

1KC

Thousand Contributors Programme
20
star
30

polkadot-payouts

TypeScript
19
star
31

polkadot-watcher-validator

TypeScript
18
star
32

offences-monitor

Monitors slashable offences registered on a Substrate based chain.
JavaScript
18
star
33

w3f-education

Technical Education at Web3 Foundation
JavaScript
17
star
34

polkadot-registrar-challenger

Polkadot Registrar Service (beta)
Rust
15
star
35

ring-proof

ring-vrf ring proof v2.5
Rust
14
star
36

educhain

Parachain developed and maintained by Tech Ed team
Rust
14
star
37

polkadot-registrar-watcher

TypeScript
13
star
38

polkadot-light-paper

Light Polkadot info
12
star
39

ipfs-cluster-chart

Helm Chart for: https://cluster.ipfs.io/documentation/guides/k8s/
Shell
11
star
40

polkadot-tests

Polkadot Protocol Conformance Tests
Rust
11
star
41

polkadot-lab

Testing framework for Polkadot networks
TypeScript
11
star
42

polkadot-watcher-csv-exporter

polkadot-watcher-csv-exporter
TypeScript
11
star
43

chainspec-generator

CLI for generating the Polkadot and Kusama chain specification from Ethereum state.
TypeScript
11
star
44

faucet-bot

A DOTs-giving bot frontend to the faucet.
JavaScript
10
star
45

parachain-implementers-guide

9
star
46

ark-scale

Arkworks serialization wrapped in Parity SCALE codec
Rust
8
star
47

helm-charts

8
star
48

substrate-legacy

Rust
8
star
49

xcmp_prototype_playground

Prototyping several xcmp approaches
Rust
8
star
50

matrixbot

ChatBot for infrastructure interactions
Python
8
star
51

substrate-telemetry-chart

Smarty
7
star
52

polkadot-api-client-ts

TypeScript
7
star
53

polkadot-watcher-transaction

TypeScript
7
star
54

polkadot-dashboard

6
star
55

matrix-server-charts

Shell
6
star
56

injection-tool

Tools, scripts and utilities for making injections.
TypeScript
6
star
57

substrate-benchmarks-role

Ansible role for substrate runtime module benchmarking
Python
6
star
58

algorithmacs

Algorithmic style for Texmacs
TypeScript
6
star
59

ethereum-tracker

JavaScript
6
star
60

polkadot-docs

Polkadot Developer Documentation
6
star
61

validator-selection-tool

TypeScript
6
star
62

polkadot-claims

Claim a DOT allocation to a Polkadot public key.
Solidity
6
star
63

KTFPs

Kusama Treasury Funding Proposals
6
star
64

terraform-digitalocean-polkadot-deployer

Go
5
star
65

polkadot-react-icons

TypeScript
5
star
66

PPPs

Polkadot Protocol Proposals
TypeScript
5
star
67

terraform-ts

TypeScript
4
star
68

test-utils-ts

TypeScript
4
star
69

matrix-recorder-chart

HTML
4
star
70

helm-ts

TypeScript
4
star
71

substrate-alertrules-chart

Shell
4
star
72

polkadot-account-monitoring

Rust
4
star
73

kusama-guide-staging

staging server for kusama guide
HTML
4
star
74

components-ts

TypeScript
4
star
75

kusama-guide-hosting

Repository to deploy the Kusama Guide for hosting on GitHub Pages.
HTML
4
star
76

ethberlin4

Solidity
4
star
77

terraform-google-polkadot-lab

Creates the infrastructure for running polkadot network tests
HCL
3
star
78

polkadot-wiki-staging

polkadot wiki dev server build branch (github pages deployment)
JavaScript
3
star
79

edgeware-deployment

Dockerfile
3
star
80

cloudflare-ts

TypeScript
3
star
81

web3

3
star
82

terratest-polkadot-deployer

Go
3
star
83

polkadot-checker

JavaScript
3
star
84

terraform-google-polkadot-deployer

Go
3
star
85

teleport-role

HTML
3
star
86

terraform-azure-polkadot-deployer

HCL
3
star
87

node-docker

Dockerfile
3
star
88

NPoS-Economics

Jupyter Notebook
3
star
89

node-exporter-dashboard

3
star
90

polkadot-validator-ansible

Python
3
star
91

harvester-chart

Smarty
3
star
92

uptime-probe

Rust
3
star
93

1k-validators-candidate-verification

Rust
3
star
94

ghost-staging

Shell
3
star
95

disc2020-scalability-and-interoperability-workshop

HTML
3
star
96

algebraic-torus

A library to facilitate comptutation with algebraic torus
Sage
3
star
97

hs-p4p

p2p networking library in Haskell
Haskell
3
star
98

terraform-aws-polkadot-deployer

HCL
3
star
99

1k-watcher-claimed-payouts

A tool for generating reports about reward claims of all the 1k Validator Programme candidates.
Rust
3
star
100

crypto-ts

TypeScript
3
star