• Stars
    star
    107
  • Rank 313,457 (Top 7 %)
  • Language
    Julia
  • License
    MIT License
  • Created about 4 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Parameterise all the things

Functors.jl

Functors.jl provides tools to express a powerful design pattern for dealing with large/ nested structures, as in machine learning and optimisation. For large machine learning models it can be cumbersome or inefficient to work with parameters as one big, flat vector, and structs help manage complexity; but it is also desirable to easily operate over all parameters at once, e.g. for changing precision or applying an optimiser update step.

Functors.jl provides fmap to make those things easy, acting as a 'map over parameters':

julia> using Functors

julia> struct Foo
         x
         y
       end

julia> @functor Foo

julia> model = Foo(1, [1, 2, 3])
Foo(1, [1, 2, 3])

julia> fmap(float, model)
Foo(1.0, [1.0, 2.0, 3.0])

It works also with deeply-nested models:

julia> struct Bar
         x
       end

julia> @functor Bar

julia> model = Bar(Foo(1, [1, 2, 3]))
Bar(Foo(1, [1, 2, 3]))

julia> fmap(float, model)
Bar(Foo(1.0, [1.0, 2.0, 3.0]))

The workhorse of fmap is actually a lower level function, functor:

julia> xs, re = functor(Foo(1, [1, 2, 3]))
((x = 1, y = [1, 2, 3]), var"#21#22"())

julia> re(map(float, xs))
Foo(1.0, [1.0, 2.0, 3.0])

functor returns the parts of the object that can be inspected, as well as a re function that takes those values and restructures them back into an object of the original type.

To include only certain fields, pass a tuple of field names to @functor:

julia> struct Baz
         x
         y
       end

julia> @functor Baz (x,)

julia> model = Baz(1, 2)
Baz(1, 2)

julia> fmap(float, model)
Baz(1.0, 2)

Any field not in the list will not be returned by functor and passed through as-is during reconstruction. This is done by invoking the default constructor, so structs that define custom inner constructors are expected to provide one that acts like the default.

It is also possible to implement functor by hand when greater flexibility is required. See here for an example.

For a discussion regarding the need for a cache in the implementation of fmap, see here.

Use exclude for more fine-grained control over whether fmap descends into a particular value (the default is exclude = Functors.isleaf):

julia> using CUDA

julia> x = ['a', 'b', 'c'];

julia> fmap(cu, x)
3-element Array{Char,1}:
 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)
 'b': ASCII/Unicode U+0062 (category Ll: Letter, lowercase)
 'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)

julia> fmap(cu, x; exclude = x -> CUDA.isbitstype(eltype(x)))
3-element CuArray{Char,1}:
 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)
 'b': ASCII/Unicode U+0062 (category Ll: Letter, lowercase)
 'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)

Related Packages

More Repositories

1

Flux.jl

Relax! Flux is the ML library that doesn't make you tensor
Julia
4,359
star
2

Zygote.jl

21st century AD
Julia
1,431
star
3

model-zoo

Please do not feed the models
Julia
878
star
4

FastAI.jl

Repository of best practices for deep learning in Julia, inspired by fastai
Julia
578
star
5

GeometricFlux.jl

Geometric Deep Learning for Flux
Julia
348
star
6

Metalhead.jl

Computer vision models for Flux
Julia
314
star
7

MacroTools.jl

MacroTools provides a library of tools for working with Julia code and expressions.
Julia
301
star
8

Torch.jl

Sensible extensions for exposing torch in Julia.
Julia
200
star
9

NNlib.jl

Neural Network primitives with multiple backends
Julia
188
star
10

MLJFlux.jl

Wrapping deep learning models from the package Flux.jl for use in the MLJ.jl toolbox
Julia
137
star
11

ONNX.jl

Read ONNX graphs in Julia
Julia
137
star
12

FluxTraining.jl

A flexible neural net training library inspired by fast.ai
Julia
114
star
13

IRTools.jl

Mike's Little Intermediate Representation
Julia
107
star
14

Flux3D.jl

3D computer vision library in Julia
Julia
100
star
15

Mjolnir.jl

A little less conversation, a little more abstraction
Julia
87
star
16

Optimisers.jl

Optimisers.jl defines many standard optimisers and utilities for learning loops.
Julia
68
star
17

DaggerFlux.jl

Distributed computation of differentiation pipelines to use multiple workers, devices, GPU, etc. since Julia wasn't fast enough already
Julia
65
star
18

Gym.jl

Gym environments in Julia
Julia
54
star
19

FluxML-Community-Call-Minutes

The FluxML Community Team repo
51
star
20

Tracker.jl

Flux's ex AD
Julia
50
star
21

XLA.jl

"Maybe we have our own magic."
Julia
47
star
22

FluxJS.jl

I heard you like compile times
Julia
42
star
23

DataAugmentation.jl

Flexible data augmentation library for machine and deep learning
Julia
41
star
24

HuggingFaceApi.jl

Julia
33
star
25

Hydra.jl

SPMD + Neural Nets
Julia
31
star
26

ParameterSchedulers.jl

Common hyperparameter scheduling for ML
Julia
28
star
27

Alloc.jl

Julia
26
star
28

Trebuchet.jl

throw stuff
Julia
21
star
29

fluxml.github.io

Flux Website
HTML
20
star
30

YaoFlux.jl

Differentiable programming on quantum circuits with Flux
Julia
19
star
31

OneHotArrays.jl

Memory efficient one-hot array encodings
Julia
17
star
32

ZygoteRules.jl

Julia
15
star
33

FluxBench.jl

Benchmarks for the FluxML ecosystem for deep learning, scientific machine learning, differentiable programming etc including AD and CUDA accelerated workloads
Julia
14
star
34

NNlibCUDA.jl

CUDA integration for the NNlib API
Julia
14
star
35

DiffImages.jl

Differentiable Computer Vision using pure Julia
Julia
14
star
36

Fluxperimental.jl

Experimental features for Flux.jl
Julia
13
star
37

SafeTensors.jl

Julia
9
star
38

MetalheadWeights

Pre-trained model weight artifacts for Metalhead.jl
Julia
7
star
39

FluxMLBenchmarks.jl

A benchmarking suite for the FluxML org
Julia
6
star
40

IArrays.jl

Julia
5
star
41

differentiable.dev

HTML
3
star
42

FluxCUDA.jl

Julia
3
star
43

FluxMLDocs

Unified documentation across the FluxML ecosystem
Julia
1
star
44

.github

Repository for default community health files
1
star