• Stars
    star
    225
  • Rank 177,187 (Top 4 %)
  • Language
  • License
    Other
  • Created almost 8 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Community curated resources for Dat Project [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]

deprecated

More info on active projects and modules at dat-ecosystem.org


Dat awesome Awesome

A curated list of the Dat Project ecosystem.

Please read the contribution guidelines before contributing.

Want to learn more? Check out:

Dat Applications

User facing applications for sharing, downloading, and managing dats.

  • dat - command line interface for managing dat archives
  • Dat Desktop - desktop app for managing dat archives
  • Beaker Browser - a peer-to-peer browser with tools to create and host websites.

Community Applications

Projects built using Dat to share and transfer data. Open a PR to add your project here!

  • sciencefair - The open source p2p desktop science library that puts users in control πŸ”¬ πŸ“– http://sciencefair-app.com
  • hyperirc - bot that mirrors irc channels to a hypercore read-only log
  • soundcloud-archiver - decentralized archives rescue music in case soundcloud shuts down
  • hypervision - watch and broadcast peer-to-peer live video streams
  • hypertweet - stream your twitter feed to a hypercore feed
  • dat-photos-app - decentralized, peer-to-peer photo sharing app for beaker browser
  • Dat Installer - Android mobile app to distribute APK updates
  • hypercore-accounting - a very basic double entry accounting system (proof of concept of a business idea)
  • dcp - remote file copy cli tool with an API reminiscent of scp/rsync
  • dillo-dat - Dillo web browser plugin for Dat protocol

Using Dat

Modules that help you build things on top of Dat:

High-Level APIs

High-level APIs that act as glue for many of the Dat modules:

  • dat-node - Node module for creating Dat applications with distributed file systems.
  • dat-js - A pure JavaScript browser-friendly api for using dat over webrtc
  • pauls-dat-api - Library of functions that make working with dat / hyperdrive easier.
  • node-dat-archive - node api that supports beaker browser DatArchive format, uses pauls-dat-api

Hosting & Dat Management

Tools for hosting Dats, managing sets of Dats, etc.

  • hypercore-archiver - archiver peer that backs up multiple hypercore / hyperdrive feeds to disk
  • hypercloud - p2p + ☁
  • hashbase - hosting for the peer-to-peer web
  • dat-now - publish live syncing and versioned websites, files or whatever to now.sh instantly
  • dat-share-all - quickly share all dats located in a folder from the command-line

Utilities for hypercore-archiver:

Dat project runs a registry at datproject.org. We use these tools to manage our dats:

  • dat-registry-api - account registry api for dat archives with user accounts, uses township
  • dat-registry-client - client for registry api for user registration, login and publishing

Managing & Aggregating Dats

  • multidat - manage dat archives in multiple locations, uses a dat factory, based on multidrive
  • multidrive - manage multiple hyperdrive archives located anywhere on the filesystem
  • dat-pki - A public key infrastructure with many encryption utilities for Dat filesharing
  • webdb - decentralized table/records-based database with sql-like behaviour, uses node-dat-archive
  • assassin - Blazing fast, easy to use NoSQL database designed for web workers and built with hyperdrive.

Http Hosting

  • hyperdrive-http - serve hyperdrive archives over http
  • dathttpd - A Web server for Dat and HTTPS, with zero-config TLS.

Dat Link Utilties

Resolving, parsing, encoding dat links.

  • dat-dns - issue dns lookups for dat archives using https requests to a target host
  • dat-link-resolve - resolve dat url's, links to a dat key using common methods, uses dat-dns
  • parse-dat-url - node's url.parse updated to support versioned dat url's
  • dat-encoding - encoder and decoder that supports the dat url-scheme

Dat Utilities

Utilities to show information about an existing dat archive.

  • dat-log - simple cli that lists the history of a dat archive
  • dat-ls - simple cli that lists all the changes in a dat archive
  • hyperhealth - monitor health of hyperdrive or dat archives, e.g. peer count and peer mirror %
  • hyperdrive-network-speed - track upload and download speeds on hyperdrive archives
  • dat-shell - Open a remote dat and explore with cd, ls, cp, (pseudo)ln, etc.
  • dat-walk - Recursive directory walker for dat archives
  • dat-glob - Glob implementation for dat archives
  • dat-rm - Rimraf equivalent for dat archives

File Imports & Exports

Hypercore Tools

Tools for using hypercore feeds

  • hyperpipe - simple cli to pipe and read files into live hypercore feeds

Dat Core Modules

Things we used to build Dat. Dat tools (CLI, Desktop, dat-node) are opinonated versions of hyperdrive that work well for using facing applications.

  • hyperdrive - secure, decentralized peer-to-peer file system on top of hypercore
  • hypercore - decentralized peer-to-peer append-only logs using hypercore protocol
  • hyperdb - distributed scalable database built on hypercore (hyperdb will be used in hyperdrive for multidrive after release).

CLI Utilities

Utilities used in our command line interface

  • dat-doctor - diagnose networking problems for dat, comes bundled with dat-cli
  • dat-ignore - check files against .datignore before adding to a dat archive
  • dat-json - read and write dat.json files, uses toiletdb

Networking

  • hyperdiscovery - join the p2p swarm for hypercore feeds, uses discovery-swarm
  • discovery-swarm - discover and connect to peers, uses discovery-channel
  • webrtc-swarm - create a swarm of p2p connections using webrtc and a signalhub
  • dat-swarm-defaults - default configuration for dns and dht for use with discovery-swarm

Lower level networking modules

  • discovery-channel - search discovery networks to find answering peers
  • dns-discovery - discover peers using regular- and multicast-dns
  • multicast-dns - low-level multicast-dns implementation in pure javascript
  • bittorrent-dht - complete js implementation of DHT peer discovery protocol
  • utp-native - utp protocol implementation, based on libutp native bindings
  • signalhub - simple signalling server that can be used to coordinate handshaking with webrtc

Storage

Random Access

Dat relies on random access storage. Any of these modules can be used to provide the storage for a dat archive.

The following are specific implementations of abstract-random-access:

Other Related Dat Project Modules

More modules from Dat project that are related to Dat but may not be used currently.

  • peer-network - create internet-accessible servers/clients listening on names, not hostnames
  • hyperdht - DHT that supports peer discovery and distributed hole punching

Dat Project Organization Stuff

  • datproject.org - datproject.org on github
  • discussions - general dat project discussion, ideas, feature requests
  • styleguide - styleguide and visual assets for the dat project
  • dat-elements - reusable ui elements for dat-based apps, such as loader, sprite, icon
  • dat-colors - css color definitions that match dat styleguide
  • dat-icons - svg icon definitions that match dat styleguide
  • dat.json specification - specification of the dat.json metadata format

Outdated

Modules that are currently outdated. We released a major breaking change (hypercore v6, hyperdrive v9) in May 2017. This change included major speed and storage improvements. These modules need to be updated to support the new API.

If you want to update one of these, we are happy to help you choose one depending on your interests and what may still be useful.

  • dat.haus - dat + http + unix, the composable http api to the dat network
  • hyperfeed - publish decentralized rss, atom or rdf feeds, based on hyperdrive and feed
  • normcore - no-config distributed streams using hypercore
  • github-to-hypercore - stream github event feeds into hypercore feeds, uses normcore
  • hyperspark - decentralized data processing platform for dat archives, inspired by spark
  • hypercore-index - linear asynchronous stateful indexing of a hypercore feed
  • hyperdrive-protocol - message encoding used by hyperdrive
  • hyperdrive-http-server - small cli to serve hyperdrive archives over http
  • dat-publish - small cli to publish dat archives to a hyperdrive-http or dat-archiver server
  • dat-push - small cli for pushing files to a dat-archiver or dat-publish server
  • dat-backup - backup a dat archive as a single file to local storage and retain full history
  • archiver-server - serve hypercore-archiver feeds over the dat network and http, uses discovery-swarm
  • archiver-api - simple rest api for accessing hypercore-archiver peers
  • hyperdrive-ln - create symbolic links between hyperdrive archives
  • hyperdrive-multiwriter - present a bundle of hyperdrive archives together as a multi-writer view
  • hyperdrive-named-archives - create hyperdrive archives that store and load link keys from names
  • git-dat - git plugin to use dat archives as remotes for a git repository
  • jawn - distributed version control for tabular data, based on hypercore
  • dat-archiver - archiver peer that backs up dat archives, based on hypercore-archiver
  • hyperdrive-stats - live and persistent statistics tracker for hyperdrive archives
  • hyperdrive-stats-server - server for sending hypercore / hyperdrive stats over server-side events
  • hyperdrive-stats-ui - html-based user interface to hypercore-stats-server
  • zip-to-hyperdrive - import contents of a zip archive into a hyperdrive archive
  • url-dat - import files from http url's into a hyperdrive archive, uses tar-dat
  • tar-dat - stream tar files into a hyperdrive archive
  • hyperdrive-duplicate - check if a file is a duplicate to a hyperdrive archive entry

License

CC0

To the extent possible under law, the Dat team, and contributors have waived all copyright and related or neighboring rights to this work.

More Repositories

1

dat-desktop

Peer to peer data syncronization [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
648
star
2

docs

Documentation resources for dat and the surrounding ecosystem [ DEPRECATED - see https://github.com/hypercore-protocol/new-website/tree/master/guides for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
530
star
3

dat-node

Node module for creating dat compatible tools on file systems [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
505
star
4

discovery-channel

Search for a key across multiple discovery networks and find peers who answer. [ DEPRECATED - see https://github.com/hyperswarm/dht for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
299
star
5

datBase

Open data sharing powered by Dat [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
248
star
6

gasket

Build cross platform data pipelines [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
191
star
7

how-dat-works

Dat documentation [ DEPRECATED - see https://github.com/hypercore-protocol/new-website/tree/master/guides for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
HTML
169
star
8

DEPs

Dat Enhancement Proposals. Contains all specs for the Dat protocol, including drafts. [ DEPRECATED - see https://github.com/hypercore-protocol/hypercore-proposals for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
165
star
9

rabin

node native addon for rabin fingerprinting data streams [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
C++
146
star
10

dat-js

A pure JavaScript browser-friendly api for using dat [ DEPRECATED - see https://github.com/datproject/sdk for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
142
star
11

hyperdiscovery

[ DEPRECATED - see https://github.com/hyperswarm/replicator for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
135
star
12

hypercloud

A hosting server for Dat. [ DEPRECATED - see github.com/beakerbrowser/hashbase for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
97
star
13

hyperdrive-http

HTTP request handler for Hyperdrive and Hypercore [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
68
star
14

book

Documentation on how to implement the Dat Protocol [ DEPRECATED - see https://github.com/hypercore-protocol/new-website/tree/master/guides for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
Shell
68
star
15

datproject-discussions

a repo for discussions and other non-code organizing stuff [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
65
star
16

svalbard

A global metadata vault [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
62
star
17

whitepaper

2017 Dat Whitepaper - Added for Archival Purposes (see DEPs for latest spec) [ DEPRECATED - see https://github.com/dat-ecosystem-archive/DEPs for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
TeX
61
star
18

datasilo

Build a home server that adds high speed storage and upload capacity to a global volunteer science network [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
59
star
19

dat-dns

Issue DNS lookups for Dat archives using HTTPS requests to the target host. [ DEPRECATED - see https://github.com/martinheidegger/hyper-dns for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
57
star
20

node-dat-archive

A nodejs API for Dat which is compatible with Beaker's DatArchive API [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
50
star
21

dat.foundation

[ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
47
star
22

design

Design Resources and Assets [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
SCSS
43
star
23

datscript

A compile-to-JSON data pipeline scripting language [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
43
star
24

dat-storage

Dat specific storage provider for Hyperdrive [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
35
star
25

organization

Dat Project Organization Information (Grants, Accounting, Etc.) [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
28
star
26

dat.json

The WIP specification for the dat.json meta format [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
27
star
27

dat-swarm-defaults

Dat Defaults for Discovery Swarm [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
27
star
28

project-updates

updates from the various projects working on or with dat [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
27
star
29

dat-http

a http transport/storage provider for dats [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
26
star
30

planning

Dat planning documents [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
25
star
31

old-website

Dat Project Website [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
23
star
32

web-specs

Specifications for the Dat protocol in Web browsers [ DEPRECATED - see https://github.com/dat-ecosystem-archive/DEPs for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
22
star
33

multidat

Manage multiple dat instances [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
22
star
34

dat-ls

Small program that lists all the changes in a dat [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
22
star
35

hyperdrive-import-files

Import some files and folders into a hyperdrive [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
21
star
36

dat-secret-storage

hyperdrive storage module for dat secret keys [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
21
star
37

dat-link-resolve

Dat link resolver [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
19
star
38

dat-encoding

Dat's way of encoding and decoding dat links [ DEPRECATED - see https://github.com/mafintosh/abstract-encoding and https://github.com/compact-encoding for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
19
star
39

hyperdrive-network-speed

track upload and download speed on a hyperdrive archive [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
18
star
40

dat.land

[ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
17
star
41

multidrive

Manage multiple hyperdrive instances [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
13
star
42

dat-doctor

πŸ₯ extension for dat [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
13
star
43

projects

Dat Project Projects [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
13
star
44

dat-deep-link

[DEPRECATED] An experiment to define an URI scheme that implements 'hypercore-strong-link'
JavaScript
12
star
45

dat-log

Log History of Dat [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
12
star
46

dat-foundation-website

[ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
Vue
11
star
47

datprotocol.org

Dat Protocol Website [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
HTML
10
star
48

dat-registry-api

Creates an api and database for tracking users and dats. [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
10
star
49

dat-json

manage dat.json file via toiletdb [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
10
star
50

dat-ignore

Check if you should ignore a file before adding to Dat [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
9
star
51

anacapa-container

A containerized way to run the Anacapa eDNA processing toolkit on your own machine or server [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
Shell
9
star
52

getdat

Bulk data and/or metadata downloader for large lists of HTTP and FTP urls [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
9
star
53

dat-shaka-player-plugin

[DEPRECATED] A plugin for 'shaka-player' to load 'dat://' URI
JavaScript
8
star
54

dat-data.com

OLD website for the dat project [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
8
star
55

hyperdrive-staging-area

Staging area for local, uncommited writes that can sync to a hyperdrive [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
8
star
56

dat-registry-client

Node.js Client for datBase [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
6
star
57

dat-elements

Dat UI elements [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
5
star
58

hyperdrive-to-zip-stream

Create a zipfile from a hyperdrive [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
5
star
59

analytics-server

fair analytics server w/ docker deployment [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
4
star
60

dat.land-roadmap

[ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
3
star
61

integration-tests

A mininet test suite for Dat implementations.[ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
Sass
3
star
62

Code-of-Conduct

Code of Conduct Discussions [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
3
star
63

discovery-deployment

WIP Deploy your own peer discovery infrastructure [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
2
star
64

datproject.org

JavaScript
1
star
65

desktop-website

πŸ–₯️ Dat Desktop project site [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
1
star
66

dat-glob

Glob implementation for dat archives [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
1
star
67

dat-walk

Recursive directory walker for dat archives [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
1
star
68

dat-rm

Delete files from dat archives using glob patterns [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]
JavaScript
1
star