• Stars
    star
    128
  • Rank 279,377 (Top 6 %)
  • Language
    C++
  • License
    Apache License 2.0
  • Created over 4 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Private Set Intersection Cardinality protocol based on ECDH and Bloom Filters

om-logo

CD License OpenCollective

PSI

Private Set Intersection protocol based on ECDH and Golomb Compressed Sets or Bloom Filters.

Protocol

The Private Set Intersection (PSI) protocol involves two parties, a client and a server, each holding a dataset. The goal of the protocol is for the client to determine the intersection between their dataset and the server's dataset, without revealing any information about their respective datasets to each other.

The protocol proceeds as follows:

  1. Setup (server)

The server encrypts all its elements x under a commutative encryption scheme, computing H(x)^s where s is its secret key. The encrypted elements are then inserted into a container and sent to the client in the form of a serialized protobuf and resembles* the following:

[ H(x_1)^(s), H(x_2)^(s), ... , H(x_n)^(s) ]
  1. Request (client)

The client encrypts all their elements x using the commutative encryption scheme, computing H(x)^c, where c is its secret key. The client sends its encrypted elements to the server along with a boolean flag, reveal_intersection, indicating whether the client wants to learn the elements in the intersection or only its size (cardinality). The payload is sent as a serialized protobuf and resembles* the following:

[ H(x_1)^(c), H(x_2)^(c), ... , H(x_n)^(c) ]
  1. Response (server)

For each encrypted element H(x)^c received from the client, the server encrypts it again under the commutative encryption scheme with its secret key s, computing (H(x)^c)^s = H(x)^(cs). The result is sent back to the client in a serialized protobuf and resembles* the following:

[ H(x_1)^(cs), H(x_2)^(cs), ... , H(x_n)^(cs) ]
  1. Compute intersection (client)

The client decrypts each element received from the server's response using its secret key c, computing (H(x)^(cs))^(1/c) = H(x)^s. It then checks whether each decrypted element is present in the container received from the server, and reports the number of matches as the intersection size.

It's worth noting that the protocol has several variants, some of which introduce a small false-positive rate, while others do not generate false positives. This behavior is selective, and the false-positive rate can be tuned. The selection has implications on communication costs as well.

NOTE resembles*: The protocol has configurable containers. Golomb Compressed Sets (Gcs) is the default container but it can be overridden to be BloomFilter or Raw encrypted strings. Gcs and BloomFilter will have false positives whereas Raw will not. Using Raw increases the communication cost as it is sending raw strings over the wire while the other two options drastically reduce the cost at the price of having false positives.

Security

See SECURITY.md.

Requirements

There are requirements for the entire project which each language shares. There also could be requirements for each target language:

Global Requirements

These are the common requirements across all target languages of this project.

  • A compiler such as clang or gcc
  • Bazel

Installation

The repository uses a folder structure to isolate the supported targets from one another:

private_set_intersection/<target language>/<sources>

C++

See the C++ README.md

JavaScript

See the JavaScript README.md

Go

See the Go README.md

Python

See the Python README.md

Rust

See the Rust README.md

Usage

To use this library in another Bazel project, add the following to your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
   name = "org_openmined_psi",
   remote = "https://github.com/OpenMined/PSI",
   branch = "master",
)

load("@org_openmined_psi//private_set_intersection:preload.bzl", "psi_preload")

psi_preload()

load("@org_openmined_psi//private_set_intersection:deps.bzl", "psi_deps")

psi_deps()

load("@pip_deps//:requirements.bzl", "install_deps")

install_deps()

load("@build_bazel_rules_nodejs//:index.bzl", "node_repositories", "npm_install")

node_repositories()

npm_install(
    name = "npm",
    package_json = "//:package.json",
    package_lock_json = "//:package-lock.json",
)

load("@emsdk//:emscripten_deps.bzl", emsdk_emscripten_deps = "emscripten_deps")

emsdk_emscripten_deps()

A full description of the protocol can be found in the documentation of the PsiClient class. The corresponding server class is PsiServer. An example of how to interleave the different phases of the protocol can be found in psi_server_test.cpp.

Changes

See CHANGES.md.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

Contributors

See CONTRIBUTORS.md.

License

Apache License 2.0

More Repositories

1

PySyft

Perform data science on data that remains in someone else's server
Python
9,418
star
2

TenSEAL

A library for doing homomorphic encryption operations on tensors
C++
790
star
3

PyGrid-deprecated---see-PySyft-

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science
Python
614
star
4

PyDP

The Python Differential Privacy Library. Built on top of: https://github.com/google/differential-privacy
Python
486
star
5

private-ai-resources

SOON TO BE DEPRECATED - Private machine learning progress
469
star
6

PipelineDP

PipelineDP is a Python framework for applying differentially private aggregations to large datasets using batch processing systems such as Apache Spark, Apache Beam, and more.
Python
271
star
7

PyVertical

Privacy Preserving Vertical Federated Learning
Python
211
star
8

SyferText

A privacy preserving NLP framework
Python
196
star
9

courses

A place where our community can discuss OpenMined Courses, including posting questions, sharing feedback, or providing comments for discussion!
168
star
10

syft.js

The official Syft worker for Web and Node, built in Javascript
JavaScript
147
star
11

Roadmap

This repository contains OpenMined's official development and community roadmap.
131
star
12

SyMPC

A SMPC companion library for Syft
Python
96
star
13

KotlinSyft

The official Syft worker for secure on-device machine learning
Kotlin
83
star
14

PyDentity

A repository for leveraging Self-Sovereign Identity in applications
Jupyter Notebook
65
star
15

PySyft-TensorFlow

SOON TO BE DEPRECATED - The TensorFlow bindings for PySyft
Python
57
star
16

Threepio

A multi-language library for translating commands between PyTorch, TensorFlow, and TensorFlow.js
Python
56
star
17

sycret

Function Secret Sharing library for Python and Rust with hardware acceleration
Rust
50
star
18

SwiftSyft

The official Syft worker for iOS, built in Swift
Swift
47
star
19

openmined-website

The OpenMined website...
JavaScript
43
star
20

covid-alert

A privacy-preserving app for comparing last-known locations of coronavirus patients
JavaScript
43
star
21

PyFE

A library for running Functional Encryption on tensors
Python
41
star
22

PIR

Private Information Retrieval protocol
C++
41
star
23

PyZPK

Python wrapper for open source Zero Proof Knowledge Library
C++
27
star
24

openmined

OpenMined courses application
TypeScript
25
star
25

opus

Python
22
star
26

PyAriesFL

Federated Learning on HyperLedger Aries
Python
21
star
27

syft-proto

Defines types for all Serde encoding across languages
JavaScript
20
star
28

datasets

Jupyter Notebook
16
star
29

pygrid-admin

The user interface for PyGrid!
TypeScript
13
star
30

JavaDP

Differential privacy implementation in the Java family of languages (Java, Kotlin, Scala etc...)
11
star
31

aries-did.js

A repo for exploring the use of Hyperledger Aries to facilitate decentralised identity services.
TypeScript
11
star
32

syft_experimental

Deliberate experimental Rust implementation of Syft
Rust
11
star
33

SwiftDP

Swift wrapper for Google's Differential Privacy Project
Objective-C++
11
star
34

writing

11
star
35

sgx-experiments

Trusted execution experiments with Intel SGX
Makefile
11
star
36

omui

The OpenMined UI component system for usage in all our web applications and Framer prototyping
TypeScript
10
star
37

design

This is the main hub for those interested in design in the OpenMined community
Jupyter Notebook
10
star
38

CampX

Tensor Based Environment Framework for Training RL Agents - Pre Alpha
Python
8
star
39

.github

All our community health files
7
star
40

design-assets

All OpenMined design assets
7
star
41

Bootcamps

7
star
42

serverless-website-api

SOON TO BE DEPRECATED - A Github statistics fetcher, running on a cron job, with permanent storage to DynamoDB, for the OpenMined community.
JavaScript
7
star
43

privacy-conference

The website for our 2020 privacy conference
JavaScript
6
star
44

PyDPValidator

Validation assets for core OpenMined libraries
Jupyter Notebook
6
star
45

X-PenTest

Repository for carrying out Pentesting on OM Infrastructure
6
star
46

NetworkRegistry

5
star
47

miner

A collection of web scraping technologies focused around making it easy for users to download their data.
5
star
48

paillier.js

A pure javascript implementation of paillier - runnable in browser, node, or react native
TypeScript
5
star
49

research

5
star
50

Hackathon-DSA

Jupyter Notebook
4
star
51

openmined-ghost-theme

SOON TO BE DEPRECATED - The theme for the OpenMined and Weekly Digs blogs.
SCSS
4
star
52

GridMonitor

SOON TO BE DEPRECATED - A user interface for monitoring a network router for PyGrid Platform
CSS
3
star
53

diffPrivR

R implementation of google's differential privacy library
3
star
54

daa.js

A javascript wrapper around https://github.com/xaptum/ecdaa
3
star
55

syft-enclave

Python
3
star
56

syft.cpp

SOON TO BE DEPRECATED - A library for encrypted, privacy preserving machine learning
C++
3
star
57

OpenGridNodes

1
star
58

KotlinPSI

A Kotlin library for private set intersection
1
star
59

clojure-dp

Clojure
1
star
60

trasterisk

kwarger is a Flake8 plugin which enforces named kwargs or trasterisks in your function arguments
Python
1
star
61

SwiftPSI

A Swift library for private set intersection
1
star