• Stars
    star
    239
  • Rank 168,763 (Top 4 %)
  • Language
    Go
  • License
    MIT License
  • Created almost 7 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Golang framework to build an AI that can understand and speak back to you, and everything else you want

GoReportCard GoDoc

Golang framework to build an AI that can understand and speak back to you, and everything else you want.

WARNING: the code below doesn't handle errors for readability purposes, however you SHOULD!

Demos

Here's a list of AIs built with astibob (if you're using astibob and want your project to be listed here, please submit a PR):

How it works

Overview

Overview

  • humans operate the AI through the Web UI
  • the Web UI interacts with the AI through the Index
  • the Index keeps an updated list of all Workers and forwards Web UI messages to Workers and vice versa
  • Workers have one or more Abilities and are usually located on different machines
  • Abilities run simple tasks such as reading an audio input (e.g. a microphone), executing speech-to-text analyses or doing speech-synthesis
  • Abilities can communicate directly between each other even if on different Workers
  • all communication is done via JSON messages exchanged through HTTP or Websocket

FAQ

  • Why split abilities between several workers?

    Because abilities may need to run on different machines located in different part of the world. The simplest example is wanting to read microphones inputs located in several rooms of your house. Each microphone is an ability whereas each room of your house is a worker.

Install the project

Run the following command:

$ go get -u github.com/asticode/go-astibob/...

I want to see some code

Index

// Create index
i, _ := index.New(index.Options{
    Server: astibob.ServerOptions{
        Addr:     "127.0.0.1:4000",
        Password: "admin",
        Username: "admin",
    },
})

// Make sure to properly close the index
defer i.Close()

// Handle signals
i.HandleSignals()

// Serve
i.Serve()

// Blocking pattern
i.Wait()

Worker

// Create worker
w := worker.New("Worker #1", worker.Options{
    Index: astibob.ServerOptions{
        Addr:     "127.0.0.1:4000",
        Password: "admin",
        Username: "admin",
    },
    Server: astibob.ServerOptions{Addr: "127.0.0.1:4001"},
})

// Make sure to properly close the worker
defer w.Close()

// Create runnables
r1 := pkg1.NewRunnable("Runnable #1")
r2 := pkg2.NewRunnable("Runnable #2")

// Register runnables
w.RegisterRunnables(
	worker.Runnable{
        AutoStart: true,
        Runnable:  r1,
    },
	worker.Runnable{
        Runnable:  r2,
    },
)

// Create listenables
l1 := pkg3.NewListenable(pkg3.ListenableOptions{
	OnEvent1: func(arg1 string) { log.Println(arg1) },
})
l2 := pkg4.NewListenable(pkg4.ListenableOptions{
	OnEvent2: func(arg2 string) { log.Println(arg2) },
})

// Register listenables
w.RegisterListenables(
	worker.Listenable{
        Listenable: l1,
        Runnable:   "Runnable #1",
        Worker:     "Worker #1",
    },
	worker.Listenable{
        Listenable: l2,
        Runnable:   "Runnable #3",
        Worker:     "Worker #2",
    },
)

// Handle an event and send a message to one of the runnables
w.On(astibob.DispatchConditions{
    From: astibob.NewRunnableIdentifier("Runnable #1", "Worker #1"),
    Name: astikit.StrPtr("Event #1"),
}, func(m *astibob.Message) (err error) {
    // Send message
    if err = w.SendMessages("Worker #1", "Runnable #1", pkg2.NewMessage1("Hello world")); err != nil {
        err = errors.Wrap(err, "main: sending message failed")
        return
    }
    return
})

// Handle signals
w.HandleSignals()

// Serve
w.Serve()

// Register to index
w.RegisterToIndex()

// Blocking pattern
w.Wait()

Abilities

The framework comes with a few abilities located in the abilities folder:

Audio input

This ability allows you reading from an audio stream e.g. a microphone.

Dependencies

It's strongly recommended to use PortAudio and its astibob wrapper.

To know which devices are available on the machine run:

$ go run abilities/audio_input/portaudio/cmd/main.go

Runnable and operatable

// Create portaudio
p := portaudio.New()

// Initialize portaudio
p.Initialize()

// Make sure to close portaudio
defer p.Close()

// Create default stream
s, _ := p.NewDefaultStream(portaudio.StreamOptions{
    BitDepth:             32,
    BufferLength:         5000,
    MaxSilenceLevel:      5 * 1e6,
    NumInputChannels:     2,
    SampleRate:           44100,
})

// Create runnable
r := audio_input.NewRunnable("Audio input", s)

// Register runnables
w.RegisterRunnables(worker.Runnable{
    AutoStart: true,
    Runnable:  r,
})

// Register listenables
// This is mandatory for the Web UI to work properly
w.RegisterListenables(worker.Listenable{
    Listenable: r,
    Runnable:   "Audio input",
    Worker:     "Worker #1",
})

Listenable

// Register listenables
w.RegisterListenables(
    worker.Listenable{
        Listenable: audio_input.NewListenable(audio_input.ListenableOptions{
            OnSamples: func(from astibob.Identifier, samples []int, bitDepth, numChannels, sampleRate int, maxSilenceLevel float64) (err error) {
                // TODO Do something with the samples
                return
            },
        }),
        Runnable: "Audio input",
        Worker:   "Worker #1",
    },
)

Speech to Text

This ability allows you to execute speech-to-text analyses.

Dependencies

It's strongly recommended to install DeepSpeech and its astibob wrapper.

I don't want to train a new model

  • create a working directory (for simplicity purposes, we'll assume its absolute path is /path/to/deepspeech)

  • download a client native_client.<your system>.tar.xz" matching your system at the bottom of this page

  • create the /path/to/deepspeech/lib directory and extract the client content inside it

  • create the /path/to/deepspeech/include directory and download deepspeech.h inside it

  • create the /path/to/deepspeech/model/en directory, and download and extract the english model inside it

  • whenever you run a worker that needs deepspeech, make sure to have the following environment variables:

      CGO_CXXFLAGS="-I/path/to/deepspeech/include"
      LIBRARY_PATH=/path/to/deepspeech/lib:$LIBRARY_PATH
      LD_LIBRARY_PATH=/path/to/deepspeech/lib:$LD_LIBRARY_PATH
    

I want to train a new model

In addition to the steps above:

  • create the /path/to/deepspeech/model/custom directory
  • run git clone https://github.com/mozilla/DeepSpeech inside /path/to/deepspeech
  • install the dependencies

Runnable and Operatable

// Create deepspeech
mp := "/path/to/deepspeech/model/en"
d := deepspeech.New(deepspeech.Options{
    AlphabetPath:   mp + "/alphabet.txt",
    BeamWidth:      1024,
    ClientPath:     "/path/to/deepspeech/DeepSpeech/DeepSpeech.py",
    LMPath:         mp + "/lm.binary",
    LMWeight:       0.75,
    ModelPath:      mp + "/output_graph.pb",
    PrepareDirPath: "/path/to/deepspeech/prepare",
    TrainingArgs: map[string]string{
        "checkpoint_dir":   "/path/to/deepspeech/model/custom/checkpoints",
        "dev_batch_size":   "4",
        "export_dir":       "/path/to/deepspeech/model/custom",
        "noearly_stop":     "",
        "test_batch_size":  "4",
        "train_batch_size": "20",

        // Mozilla values
        "learning_rate": "0.0001",
        "dropout_rate":  "0.15",
        "lm_alpha":      "0.75",
        "lm_beta":       "1.85",
    },
    TriePath:             mp + "/trie",
    ValidWordCountWeight: 1.85,
})

// Make sure to close deepspeech
defer d.Close()

// Initialize deepspeech
d.Init()

// Create runnable
r := speech_to_text.NewRunnable("Speech to Text", d, speech_to_text.RunnableOptions{
    SpeechesDirPath: "/path/to/speech_to_text/speeches",
})

// Initialize runnable
r.Init()

// Make sure to close the runnable
defer r.Close()

// Register runnables
w.RegisterRunnables(worker.Runnable{
    AutoStart: true,
    Runnable:  r,
})

// Send samples
w.SendMessage(worker.MessageOptions{
    Message:  speech_to_text.NewSamplesMessage(
        from,
        samples,
        bitDepth,
        numChannels,
        sampleRate,
        maxSilenceLevel,
    ),
    Runnable: "Speech to Text",
    Worker:   "Worker #3",
})

Listenable

// Register listenables
w.RegisterListenables(
    worker.Listenable{
        Listenable: speech_to_text.NewListenable(speech_to_text.ListenableOptions{
            OnText: func(from astibob.Identifier, text string) (err error) {
                // TODO Do something with the text
                return
            },
        }),
        Runnable: "Speech to Text",
        Worker:   "Worker #3",
    },
)

Text to Speech

This ability allows you to run speech synthesis.

Dependencies

It's strongly recommended to use astibob wrapper.

If you're using Linux it's strongly recommended to use ESpeak.

Runnable

// Create speaker
s := speak.New(speak.Options{})

// Initialize speaker
s.Initialize()

// Make sure to close speaker
defer s.Close()

// Register runnables
w.RegisterRunnables(worker.Runnable{
    AutoStart: true,
    Runnable:  text_to_speech.NewRunnable("Text to Speech", s),
})

// Say something
w.SendMessage(worker.MessageOptions{
    Message:  text_to_speech.NewSayMessage("Hello world"),
    Runnable: "Text to Speech",
    Worker:   "Worker #1",
})

Create your own ability

Creating your own ability is pretty straight-forward: you need to create an object that implements the astibob.Runnable interface. Optionally it can implement the astibob.Operatable interface as well.

If you want other abilities to be able to interact with it you'll need to create another object that implements the astibob.Listenable interface.

I strongly recommend checking out how provided abilities are built and trying to copy them first.

Runnable

The quickest way to implement the astibob.Runnable interface is to add an embedded astibob.BaseRunnable attribute to your object.

You can then use astibob.NewBaseRunnable to initialize it which allows you providing the proper options.

Operatable

The quickest way to implement the astibob.Operatable interface is to add an embedded astibob.BaseOperatable attribute to your object.

You can then use the cmd/operatable command to generate an operatable.go file binding your resources folder containing your static and template files. You can finally add custom routes manually to the astibob.BaseOperatable using the AddRoute method.

Listenable

No shortcut here, you need to create an object that implements the astibob.Listenable interface yourself.

Contribute

If you've created an awesome Ability and you feel it could be of interest to the community, create a PR here.

More Repositories

1

go-astilectron

Build cross platform GUI apps with GO and HTML/JS/CSS (powered by Electron)
Go
4,745
star
2

go-astits

Demux and mux MPEG Transport Streams (.ts) natively in GO
Go
479
star
3

go-astisub

Manipulate subtitles in GO (.srt, .ssa/.ass, .stl, .ttml, .vtt (webvtt), teletext, etc.)
Go
463
star
4

go-astilectron-demo

Discover the power of Astilectron through a demo app
Go
397
star
5

go-astiencoder

Go
308
star
6

astilectron

Electron app that provides an API over a TCP socket that allows executing Electron's method as well as capturing Electron's events
JavaScript
274
star
7

go-astideepspeech

Golang bindings for Mozilla's DeepSpeech speech-to-text library
Go
170
star
8

go-astilectron-bundler

Bundle your Astilectron app with ease
Go
125
star
9

go-astiav

Better C bindings for ffmpeg in GO
Go
86
star
10

go-astilectron-bootstrap

Create a one-window application using Astilectron
Go
60
star
11

go-astitodo

Parse TODOs in your GO code
Go
60
star
12

go-asticoqui

Golang bindings for Coqui's speech-to-text library
Go
29
star
13

go-astikit

Set of golang helpers that don't require any external dependencies
Go
29
star
14

go-astivid

Set of video tools available through a nice UI
Go
27
star
15

go-astisrt

SRT server, client and socket in GO
Go
26
star
16

go-astitools

Set of augmented functions for the GO programming language (http://golang.org)
Go
17
star
17

go-astitello

Golang implementation of DJI Tello SDK
Go
10
star
18

go-texttospeech

Text to speech manager relying on the OS speech recognition software
Go
10
star
19

go-astiffprobe

Use your FFProbe binary to gather quality information about your video files
Go
10
star
20

go-astisplash

Cross platform splash screen
Go
9
star
21

go-astichat

A lightweight encrypted chat written in GO
Go
8
star
22

go-astilog

Golang logger
Go
8
star
23

js-toolbox

Set of components and methods to ease HTML/CSS/JS developments
JavaScript
6
star
24

go-astiws

Wrapper on top of websockets
Go
5
star
25

go-bindata

Hard fork from https://github.com/jteeuwen/go-bindata after it has disappeared
Go
5
star
26

go-stopwatch

Deprecated
Go
5
star
27

go-astitesseract

Wrapper for the Tesseract OCR project
Go
5
star
28

go-asticrypt

Send encrypted messages through your favourite apps
Go
4
star
29

go-astiffmpeg

Use your FFMpeg binary to manipulate your video files
Go
4
star
30

go-astiocr

Go
4
star
31

go-keyboardemulator

Cross-OS keyboard emulator that can take control of your keyboard through code
Go
4
star
32

go-slack

TODO: clone and rename
Go
2
star
33

go-astimysql

Wrapper on top of mysql to provide proper configuration
Go
2
star
34

go-astiratp

Clients for the RATP APIs
Go
2
star
35

go-astiamqp

Wrapper on top of amqp to provide proper configuration and error handling
Go
2
star
36

go-bob

Bob is an AI capable of taking over your keyboard based on voice commands
Go
2
star
37

go-astiproxy

Wrapper on top of http and url to provide proper proxy configuration
Go
2
star
38

go-astichartjs

Go
2
star
39

asticrypt

Encrypt your conversations and stop being tracked when using your favorite apps
JavaScript
2
star
40

php-deployment-manager

Deployment manager to enable automatic deployment of PHP or GO projects on your server after a GIT push
PHP
2
star
41

go-astislack

Clear your Slack history easily
Go
1
star
42

go-astimgo

Wrapper on top of mgo to provide proper configuration
Go
1
star
43

go-ftp

TODO: clone and rename
Go
1
star
44

go-speechtotext

Speech to text manager relying on the OS speech recognition software
1
star
45

go-astiredis

Wrapper on top of redis to provide proper configuration
Go
1
star
46

go-test

Sandbox for GO
Go
1
star
47

go-astiudp

Go
1
star
48

go-astilectron-deployer

Go
1
star
49

go-astitwitter

Wrapper on top of Twitter API
Go
1
star
50

go-astibank

Simple tool to monitor your bank accounts
Go
1
star
51

test

1
star
52

php-cache-manager

Cache manager for PHP
PHP
1
star
53

go-gozzle

Deprecated
Go
1
star
54

go-astibike

Should I travel by bike this week?
Go
1
star
55

go-pprof

TODO: merge stopwatch + add ticker
Go
1
star
56

go-astipatch

Patch manager written in GO
Go
1
star
57

go-astibob-demos

Official astibob demos
Go
1
star
58

php-data-mapper

Mapper and Repository factories that implements the Data Mapper structure
PHP
1
star
59

python-asticredits

1
star
60

js-astiyoga

JavaScript
1
star
61

php-file-manager

File manager to handle cross-datasources copy as well as simple file actions on the most common datasources
PHP
1
star