thi-ng/umbrella

• About
• Getting Started
  • Project templates
• Examples & Showcase
• Blog posts
• Community, contributing, getting help
• Projects
  • Latest updates
• Building
• Acknowledgements
• License
• Contributors

About

"A collection of functional programming libraries that can be composed together. Unlike a framework, thi.ng is a suite of instruments and you (the user) must be the composer of. Geared towards versatility, not any specific type of music." — @loganpowell via Twitter

Please visit thi.ng for additional information & topic based search of packages relevant to your use cases...

This project is NOT a framework! This mono-repository is home to currently 180 individual TypeScript/ES6 projects (and ~120 example projects), a wide and jointly developed collection of packages, tools and general purpose building blocks for (non-exhaustive list of topics, see package overview below):

• Functional programming (ES6 iterators/generators, composition, memoization, transducers, multi-methods)
• Reactive programming, stream / transducer based dataflow graphs / pipelines / DOM
• Data structures & data transformations for wide range of use cases (maps, sets, heaps, queues, graphs etc.)
• WebAssembly bridge APIs & data structure bindings code generators for multiple target languages
• PEG-style functional parser combinators w/ (optional) custom grammar definition language
• 2D geometry generation, shape primitives, math, manipulation, intersections, conversions & visualizations
• Canvas abstractions, pixel buffers & SVG serialization/conversion
• Vector, matrix (dense & sparse), ECS implementations with optional support for strided memory layouts
• Semi-declarative WebGL 1/2 abstraction layer
• DSL for shader functions defined in TypeScript and cross-compilation to GLSL, JS, VEX etc.
• Value-based equivalence (vs. native object identity)
• DSP building blocks: oscillators, noise generators, filters, 1D FFT/IFFT, muxers, rate converters
• Immutable data handling, state containers, transacted state updates, Undo-Redo history
• Data driven UI component toolkits (DOM-based, canvas-based, immediate-mode, multiple approaches...)
• Multi-format, multi-channel pixel buffers (int & float based), conversions, dithering, Porter-Duff alpha-blending operators
• Color space/format conversions, matrix based color manipulation, gradient generators, color palettes
• Date-time abstraction, relative dates, iterators, formatters, math
• WebWorker workflow abstractions
• Forth-style pointfree DSL for functional composition and DSL development/extension
• S-expression parser & runtime (interpreter) infrastructure for custom DSL creation
• SIMD batch-processing of vector data
• Pen-plotter (AxiDraw) toolchain & geometry conversions
• Various interpolations, math helpers, automatic differentiation (Dual numbers)
• etc.

Once more, this project is NOT a framework, provides no turn-key, one-size-fits-all approach and instead encourages a mix & match philosophy for various key aspects of application design (in & outside the browser). Customization points are provided wherever possible and usually only expect certain interfaces/type signatures rather than concrete implementations.

All packages:

• are versioned independently
• released via thi.ng/monopub
• distributed as ESM modules (ES2020 syntax) with export maps, TypeScript typings & change logs
• highly modular with largely only a single function / class (only closely related functions) per file to help w/ selective imports and tree shaking
• provide re-exports of all their publics for full library imports
• have either none or only @thi.ng internal runtime dependencies (w/ very few exceptions!)
• declare public interfaces, enums & types in an api.ts file
• have auto-generated online documentation at docs.thi.ng
• licensed under Apache Software License 2.0

Getting started

The sheer number and varied nature & purpose of these packages makes it impossible to provide traditional "Getting started" tutorials. To compensate, this repo provides ~120 example projects, detailed readmes (at the very least for all the larger and/or more important packages) and hundreds of small usage examples/snippets in various docstrings.

If unsure about something, please reach out!

Project templates

You might find one of the following template repos an useful starting point:

• tpl-umbrella-basic: Bare-bones template repo for browser-based projects
• tpl-umbrella-fxhash: Project template repo for generative art projects on the fx(hash) platform
• tpl-umbrella-zig: Minimal browser project template for hybrid TypeScript & Zig (WebAssembly) apps

Examples & Showcase

There's a steadily growing number (~120) of standalone examples of different complexities (often combining functionality from several packages) in the examples directory.

Example screenshots	(small selection)

awesome.thi.ng

Due to other priorities still very much in its infancy & planning stage, but please help to document your own usage of these packages by contributing project information to the awesome.thi.ng repo, which will be used to build a showcase site... Thank you!

Generative art projects

Several generative art projects by Karsten Schmidt on fx(hash) have been created exclusively with libraries from this collection.

De/Frag series
Quasiflock	C-SCAPE	ASCII-SCAPE
Bubblemania	Danza (unreleased)	S-TRACE (unreleased)

Blog posts

• "Of umbrellas, transducers, reactive streams & mushrooms" (ongoing series):
  • Part 1 - Project & series overview
  • Part 2 - HOFs, Transducers, Reducers
  • Part 3 - Convolution, 1D/2D Cellular automata
  • Part 4 - Disjoint Sets, Graph analysis, Signed Distance Fields
• How to UI in 2018
• The Jacob's Ladder of coding

Videos

• Building a web editor for creating/testing parse grammars
• Building a shader graph editor (WebGL, shader AST transpiler, UI)
• Crash course: TypeScript mapped types in action
• thi.ng/umbrella livestream #3

Community, contributing, getting help

Join the (still new) discussions here on Github, get in touch via Mastodon or use the issue tracker. If you'd like to contribute in other ways, please first read this document.

In general, we welcome contributions of all kinds (docs, examples, bug fixes, feature requests, financial contributions etc.). You can find a fairly detailed overview for contributors here: CONTRIBUTING.md.

Note: The default branch for this repo is develop and all PRs should be based on this branch. This too means, the README files on this branch might refer to yet-unreleased features or packages. Please use the main branch for viewing the most recently released version(s)!.

Wiki

So far the wiki has only been updated sporadically, but please be sure to check it out for project-wide glossary, information, cookbooks, useful snippets etc.

Projects

Latest updates

As of: 2023-06-29

Status	Package	Version	Changelog
	@thi.ng/leb128		changelog
	@thi.ng/timestep		changelog
	@thi.ng/wasm-api		changelog
	@thi.ng/wasm-api-bindgen		changelog
	@thi.ng/wasm-api-dom		changelog

Fundamentals

Project	Version	Changelog	Description
@thi.ng/args		changelog	Declarative & functional CLI arg parsing & coercions
@thi.ng/api		changelog	Common types, decorators, mixins
@thi.ng/bench		changelog	Basic benchmarking helpers
@thi.ng/checks		changelog	Type & value checks
@thi.ng/compare		changelog	Comparators
@thi.ng/compose		changelog	Functional composition helpers
@thi.ng/date		changelog	Date/time iterators, formatters, rounding
@thi.ng/defmulti		changelog	Dynamic multiple dispatch
@thi.ng/distance		changelog	n-D distance metrics & K-nearest neighborhoods
@thi.ng/equiv		changelog	Deep value equivalence checking
@thi.ng/errors		changelog	Custom error types
@thi.ng/expose		changelog	Conditional global variable exposition
@thi.ng/hex		changelog	Hex value formatters for U4-64 words
@thi.ng/logger		changelog	Basis infrastructure for arbitrary logging
@thi.ng/memoize		changelog	Function memoization w/ customizable caching
@thi.ng/oquery		changelog	Pattern based query engine for JS objects
@thi.ng/parse		changelog	Parser combinators & AST generator/transformer
@thi.ng/paths		changelog	Immutable nested object accessors
@thi.ng/strings		changelog	Higher-order string formatting utils
@thi.ng/system		changelog	Minimal life cycle container for stateful app components
@thi.ng/testament		changelog	Minimal test runner
@thi.ng/transclude		changelog	Template engine for text document generation
@thi.ng/units		changelog	Extensible SI unit conversions

Maths

Project	Version	Changelog	Description
@thi.ng/dual-algebra		changelog	Dual number algebra / automatic differentiation
@thi.ng/dsp		changelog	DSP utils, composable signal gens/processors
@thi.ng/fuzzy		changelog	Fuzzy logic primitives & rule inference engine
@thi.ng/intervals		changelog	Open/closed intervals, queries, set ops
@thi.ng/math		changelog	Assorted common math functions & utilities
@thi.ng/matrices		changelog	Matrix operations
@thi.ng/sparse		changelog	Sparse matrix & vector impls
@thi.ng/timestep		changelog	Fixed timestep simulation updates with state interpolation
@thi.ng/vectors		changelog	Fixed & arbitrary-length vector ops

Randomness

Project	Version	Changelog	Description
@thi.ng/colored-noise		changelog	1D colored noise generators
@thi.ng/ksuid		changelog	K-sortable unique identifiers, binary & base-N encoded
@thi.ng/lowdisc		changelog	n-D Low discrepancy sequence generators
@thi.ng/random		changelog	Seedable PRNG implementations, distributions & utilities
@thi.ng/random-fxhash		changelog	PRNG impl & utilities for fxhash projects

File / file format / hardware support

Project	Version	Changelog	Description
@thi.ng/axidraw		changelog	Minimal, declarative AxiDraw plotter controller
@thi.ng/bencode		changelog	Bencode binary format encoding
@thi.ng/csv		changelog	Customizable CSV parser/object mapper
@thi.ng/dot		changelog	Graphviz DOM & export
@thi.ng/dsp-io-wav		changelog	WAV file format exporter
@thi.ng/file-io		changelog	Assorted file I/O utils for NodeJS
@thi.ng/geom-io-obj		changelog	Wavefront OBJ model parser
@thi.ng/hiccup-css		changelog	CSS from nested JS data structures
@thi.ng/hiccup-html		changelog	Type-checked HTML5 element wrappers for hiccup
@thi.ng/hiccup-markdown		changelog	Hiccup-to-Markdown serialization
@thi.ng/hiccup-svg		changelog	hiccup based SVG vocab
@thi.ng/iges		changelog	IGES format geometry serialization
@thi.ng/markdown-table		changelog	Markdown table generator / formatter
@thi.ng/mime		changelog	File extension to MIME type mappings
@thi.ng/pixel-io-geotiff		changelog	GeoTIFF reader support for thi.ng/pixel
@thi.ng/pixel-io-netpbm		changelog	1/8/16/24bit NetPBM image format reader/writer
@thi.ng/prefixes		changelog	Linked Data, RDF & xmlns prefixes/URLs
@thi.ng/sax		changelog	SAX-like XML parser / transducer
@thi.ng/tangle		changelog	Literate programming utilities

Iterator, stream & sequence processing

Project	Version	Changelog	Description
@thi.ng/csp		changelog	Channel based async ops
@thi.ng/fsm		changelog	FSM / parser primitives
@thi.ng/grid-iterators		changelog	2D grid iterator strategies
@thi.ng/iterators		changelog	ES6 generators / iterators
@thi.ng/seq		changelog	Lisp/Clojure-style sequence abstraction
@thi.ng/transducers		changelog	Composable data transformations
@thi.ng/transducers-binary		changelog	Binary data related transducers
@thi.ng/transducers-fsm		changelog	Finite state transducer
@thi.ng/transducers-hdom		changelog	Transducer based hdom UI updates
@thi.ng/transducers-patch		changelog	Patch-based, array & object editing
@thi.ng/transducers-stats		changelog	Technical / statistical analysis

Reactive programming

Project	Version	Changelog	Description
@thi.ng/rstream		changelog	Push-based, reactive event stream primitves
@thi.ng/rstream-csp		changelog	Adapter bridge CSP -> rstream
@thi.ng/rstream-dot		changelog	Graphviz visualization of rstream topologies
@thi.ng/rstream-gestures		changelog	Mouse & touch event stream abstraction
@thi.ng/rstream-graph		changelog	Declarative dataflow graph construction
@thi.ng/rstream-log		changelog	Hierarchical structured data logging
@thi.ng/rstream-log-file		changelog	Log-file output handler
@thi.ng/rstream-query		changelog	Triple store & query engine

Algorithms & data structures

Project	Version	Changelog	Description
@thi.ng/adjacency		changelog	Adjacency matrices & graph algorithms
@thi.ng/arrays		changelog	Array utilities
@thi.ng/associative		changelog	Alt Set & Map implementations
@thi.ng/atom		changelog	Immutable value wrappers, views, history
@thi.ng/bitfield		changelog	1D/2D bit field implementations
@thi.ng/cache		changelog	In-memory caches / strategies
@thi.ng/cellular		changelog	Highly configurable 1D Cellular automata
@thi.ng/dcons		changelog	Doubly-linked list
@thi.ng/diff		changelog	Array & object diffing
@thi.ng/dgraph		changelog	Dependency graph
@thi.ng/ecs		changelog	Entity-Component System
@thi.ng/egf		changelog	Extensible Graph Format
@thi.ng/gp		changelog	Genetic programming helpers / AST gen
@thi.ng/heaps		changelog	Binary & d-ary heap impls
@thi.ng/idgen		changelog	Versioned ID generation / free-list
@thi.ng/k-means		changelog	K-means clustering of n-D data
@thi.ng/ramp		changelog	Parametric, interpolated lookup tables
@thi.ng/quad-edge		changelog	Quad-edge, dual-graph data structure
@thi.ng/resolve-map		changelog	DAG computations & value resolution
@thi.ng/vclock		changelog	Vector clock functions / comparators
@thi.ng/zipper		changelog	Immutable tree editing / navigation

Frontend / UI

Project	Version	Changelog	Description
@thi.ng/adapt-dpi		changelog	HDPI canvas adaptation / styling util
@thi.ng/dl-asset		changelog	Asset/canvas/file download helpers
@thi.ng/emoji		changelog	Bi-directional emoji lookup tables (names/chars)
@thi.ng/hdiff		changelog	String diffing w/ hiccup output (includes CLI)
@thi.ng/hdom		changelog	Hiccup based VDOM & diffing
@thi.ng/hdom-canvas		changelog	hdom adapter for hiccup-canvas
@thi.ng/hdom-components		changelog	hdom based UI components
@thi.ng/hdom-mock		changelog	hdom mock implementation (testing / prototyping)
@thi.ng/hiccup		changelog	S-expression based HTML/XML serialization
@thi.ng/hiccup-canvas		changelog	hiccup interpreter for canvas api
@thi.ng/hiccup-carbon-icons		changelog	IBM Carbon icons in hiccup format
@thi.ng/imgui		changelog	Immediate mode GUI
@thi.ng/interceptors		changelog	Composable event handlers & processor
@thi.ng/rdom		changelog	Reactive, diff-less, async UI components
@thi.ng/rdom-canvas		changelog	rdom component wrapper for thi.ng/hiccup-canvas
@thi.ng/rdom-components		changelog	Unstyled, customizable component collection
@thi.ng/router		changelog	Customizable browser & non-browser router
@thi.ng/text-canvas		changelog	Text-mode canvas, drawing, tables, charts
@thi.ng/text-format		changelog	Color text formatting w/ ANSI & HTML presets

Geometry, image & visualization

Project	Version	Changelog	Description
@thi.ng/color		changelog	Color conversions, gradients
@thi.ng/color-palettes		changelog	Collection of image-based color palettes
@thi.ng/dgraph-dot		changelog	Dependency graph -> Graphviz
@thi.ng/distance-transform		changelog	Image based distance field generation
@thi.ng/fuzzy-viz		changelog	Visualization, instrumentation for @thi.ng/fuzzy
@thi.ng/geom		changelog	2D only geometry types & ops
@thi.ng/geom-accel		changelog	Spatial indexing data structures
@thi.ng/geom-api		changelog	Shared types & interfaces
@thi.ng/geom-axidraw		changelog	Shape conversions for AxiDraw pen plotter
@thi.ng/geom-arc		changelog	2D elliptic arc utils
@thi.ng/geom-clip-line		changelog	2D line clipping
@thi.ng/geom-clip-poly		changelog	2D convex polygon clipping
@thi.ng/geom-closest-point		changelog	Closest point helpers
@thi.ng/geom-fuzz		changelog	Fuzzy 2D shape drawing / filling
@thi.ng/geom-hull		changelog	2D convex hull (Graham scan)
@thi.ng/geom-isec		changelog	Point & shape intersection tests
@thi.ng/geom-isoline		changelog	2D contour line extraction
@thi.ng/geom-poly-utils		changelog	2D polygon helpers
@thi.ng/geom-resample		changelog	nD polyline / curve resampling
@thi.ng/geom-sdf		changelog	2D SDF creation, conversions, operators, utilities
@thi.ng/geom-splines		changelog	nD cubic / quadratic spline ops
@thi.ng/geom-subdiv-curve		changelog	nD iterative subdivision curves
@thi.ng/geom-tessellate		changelog	nD convex polygon tessellators
@thi.ng/geom-trace-bitmap		changelog	bitmap image to vector conversion
@thi.ng/geom-voronoi		changelog	2D iterative delaunay/voronoi
@thi.ng/lsys		changelog	Extensible L-System architecture
@thi.ng/pixel		changelog	Multi-format pixel buffers
@thi.ng/pixel-dither		changelog	Image dithering w/ various algorithm presets
@thi.ng/poisson		changelog	nD Poisson disk sampling
@thi.ng/porter-duff		changelog	Alpha blending / compositing ops
@thi.ng/rasterize		changelog	Shape drawing, filling & rasterization
@thi.ng/scenegraph		changelog	Extensible 2D/3D scenegraph
@thi.ng/simd		changelog	WebAssembly SIMD vector batch processing
@thi.ng/viz		changelog	Declarative & functional data visualization toolkit

WebGL / GPGPU

Project	Version	Changelog	Description
@thi.ng/shader-ast		changelog	AST DSL for x-platform shader code
@thi.ng/shader-ast-glsl		changelog	GLSL code generator
@thi.ng/shader-ast-js		changelog	JS code generator
@thi.ng/shader-ast-optimize		changelog	AST code optimization strategies
@thi.ng/shader-ast-stdlib		changelog	100+ useful AST shader functions
@thi.ng/webgl		changelog	WebGL 1/2 / GPGPU facilities
@thi.ng/webgl-msdf		changelog	MSDF font rendering
@thi.ng/webgl-shadertoy		changelog	Shadertoy-like WebGL setup

Low-level, binary, memory management, interop

Project	Version	Changelog	Description
@thi.ng/base-n		changelog	Arbitrary base-n encoding/decoding with presets
@thi.ng/binary		changelog	Assorted binary / bitwise ops, utilities
@thi.ng/bitstream		changelog	Bitwise input / output streams
@thi.ng/dlogic		changelog	Digital logic ops / constructs
@thi.ng/leb128		changelog	WASM based LEB128 varint encoder / decoder
@thi.ng/malloc		changelog	Raw & typed array memory pool & allocator
@thi.ng/morton		changelog	Z-order-curve / Morton coding
@thi.ng/range-coder		changelog	Binary data Range encoder / decoder
@thi.ng/rle-pack		changelog	Run-length encoding data compression
@thi.ng/soa		changelog	Memory mapped data structures & serialization
@thi.ng/unionstruct		changelog	Wrapper for C-like structs / unions
@thi.ng/vector-pools		changelog	Data structures for memory mapped vectors
@thi.ng/wasm-api		changelog	Modular, extensible JS/WebAssembly bridge API
@thi.ng/wasm-api-bindgen		changelog	Polyglot bindings code generator for C/Zig/TS
@thi.ng/wasm-api-canvas		changelog	WASM-side Canvas2D API bindings
@thi.ng/wasm-api-dom		changelog	WASM-side DOM manipulation
@thi.ng/wasm-api-schedule		changelog	WASM-side delayed code execution/scheduling

DSLs

Project	Version	Changelog	Description
@thi.ng/pointfree		changelog	Stack-based DSL & functional composition
@thi.ng/pointfree-lang		changelog	Forth-like syntax layer for @thi.ng/pointfree
@thi.ng/sexpr		changelog	S-Expression parser & runtime infrastructure

Building

git clone https://github.com/thi-ng/umbrella.git
cd umbrella

yarn install
yarn build

Once the entire mono-repo has been fully built at least once before, individual packages can then be (re)built like so:

yarn workspace @thi.ng/transducers run build

# or

(cd packages/transducers && yarn build)

Building example projects

Please see the example build instructions in the wiki for further details.

Testing

(Most, but not all packages have tests)

Due to various build/config issues/complexities, we're now using our own minimal test runner @thi.ng/testament

yarn test

# or individually
yarn workspace @thi.ng/rstream run test

Documentation

Autogenerated documentation (using TypeDoc) will be saved under /packages/*/doc/ and is also available at docs.thi.ng.

yarn doc

Furthermore, an experimental tsdoc-based documentation repo provides an alternative markdown output including cross-references between all packages, but currently isn't frequently updated (only every few months):

https://github.com/thi-ng/umbrella-docs-temp

Acknowledgements

Maintaining a large monorepo like this requires a lot of infrastructure and I'm grateful for the tooling provided by the following projects to simplify those tasks:

• esbuild
• html-minifier-terser
• rimraf
• ts-node
• typedoc
• TypeScript
• Vite
• @thi.ng/monopub
• @ygoe/msgpack

License

© 2015 - 2023 Karsten Schmidt // Apache Software License 2.0

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Karsten Schmidt 💻 📖 🚧 💵	Alberto 💻 💡 🐛 🤔 💵	Arthur Carabott 💻 🤔 💡 📝 💵	André Wachter 💻 🤔 🐛	Gavin Cannizzaro 💻 🐛 🤔	Logan Powell 📖 🐛 🤔 💵	Marcin Ignac 🐛
arcticnoah 💻	allforabit 🐛 💻 🤔 💵	Yifan Wu 🐛 📖	stwind 💻 🐛	evilive 💻	Bnaya Peretz 💻 🐛 🤔	oljeger 🐛
Nik Shevchenko 🐛 💻	Matei Adriel 💻 🐛 🤔	Pierre Grimaud 📖	Matt Huebert 💵	Raphael Saunier 💵	Eric Ihli 💵	David Pham 💵
TBD 🐛 🤔	Pedro Henriques dos Santos Teixeira 💵	Jamie Owen 💻 🐛	Robert Kesteson 🐛 💻	Chancy Kennedy 💵	Jarred Sumner 🐛	Jamie Slome 🐛 🛡️
d3v53c 🐛 🛡️	Jannis Pohlmann 🐛	Shakthi Prasad G S 🐛 💻	Robin Gower 🐛	Michael Latzoni 🐛	Z Yin 🐛	Damien Seguin 🐛 💻
Rui Gil 🐛	Ja|nz 💻 🚇 🚧	Tyler Freeman 🐛 💻	blackhuman 🐛	David Negstad 💻	Muhammad Ridho 🐛 💻	MarcusWagberg 💻
Masashi Yoshikawa 🐛	Guido Schmidt 🐛 💵	tsukamotohideki 💵	Dave Meehan 💻 🤔	Aurélien Bottazini 🐛	Alex 🐛 💵	Ross Cairns 💵
Ruslan Prakapchuk 💵	Jarrod Davis 💵

This project follows the all-contributors specification. Contributions of any kind welcome!