• Stars
    star
    237
  • Rank 169,885 (Top 4 %)
  • Language
  • Created over 6 years ago
  • Updated about 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Documentation and Wiki for SEPIA. Please post your questions and bug-reports here in the issues section! Thank you :-)

Welcome to the documentation page for SEPIA

Here you will (hopefully) find everything you need to know to get started with SEPIA.

S.E.P.I.A. Logo

Overview of SEPIA ecosystem (note: some parts are still in the dev branches).
For image icon attributions please check the homepage

Downloads

Wiki, Blog & News

Checkout the Wiki for detailed descriptions: S.E.P.I.A. Framework Wiki
Visit Twitter or Mastodon for the latest news: S.E.P.I.A. Twitter Feed | S.E.P.I.A. Mastodon Feed
Visit the blog for summaries and guides: S.E.P.I.A. Blog

Intro

S.E.P.I.A. is an acronym for: self-hosted, extendable, personal, intelligent assistant. It is a modular, open-source framework equipped with all the required tools to build your own, full-fledged digital voice-assistant, including speech recognition (STT), wake-word detection, text-to-speech (TTS), natural-language-understanding, dialog-management, SDK(s), a cross-platform client app and much more.
The framework consists of several, highly customizable micro-services that work together seamlessly to form the SEPIA Open Assistant. It follows the client-server principle using a lightweight Java server and Elasticsearch DB as "brain" and a Javascript based client that works as smart-speaker, smart-display, mobile assistant app or whatever smart-device you come up with 😃. All components work on Linux, Windows and Mac and have been optimized to even run smoothly on a Raspberry Pi 😌 🤖.
Out-of-the-box SEPIA currently has smart-services for: news, music (radio), timers, alarms, reminders, to-do and shopping lists, smart home (e.g. using open-source tools like openHAB), navigation, places, weather, Wikipedia, web-search, soccer-results (Bundesliga), a bit of small-talk and more. To realize your own ideas you can use tools like the SEPIA SDK and the code editor integrated into the SEPIA Control HUB to build services or write custom HTML widgets 👨‍🔧👩‍🔬!

Architecture

The SEPIA Framework consists of 2 core parts: The SEPIA Client and the Assist-Server.

SEPIA Client: The user interface that handles voice, text or touch interactions and manages the "dialog" with the SEPIA server. Server responses can be presented as text (chat), graphical elements (cards, buttons) and/or sound including speech synthesis (text-to-speech) and music (media-player). The client usually takes care of the speech-recognition (on-device or via SEPIA STT server) to transform voice into text and can even listen to wake-words like Hey SEPIA (thanks to Porcupine by Picovoice). There are clients for the browser, Android, iOS and a DIY version that even works "headless" for example on a Raspberry Pi.

Assist-Server: The "brain" of SEPIA that receives requests from the client via the HTTP REST API and takes care of the natural-language-understanding (intent and NER), conversation flow, smart-service integration (like a to-do list or news service), user-accounts, Text-to-Speech (TTS) and more. The Assist-Server can run on it's own hardware for example on SBCs like a Raspberry Pi 3 or parallel to the client on more powerful systems (RPi4, desktop PC ect.).

Because speech-recognition is a very delicate topic for multiple reasons (privacy, accuracy, performance, control etc.) the SEPIA Framework includes another major component: The Speech-To-Text (STT) server.

SEPIA STT Server: An open-source server for real-time speech-recognition that runs on most systems (x86, ARM), including Raspberry Pi and supports custom, dynamic ASR models (thanks to great tools like Kaldi, Vosk or Zamia speech).

Other notable components of the SEPIA Framework are the Control HUB to manage server, "headless" clients, Smart Home and more, the WebSocket server for multi-channel chats and duplex data transfer, the Teach-Server to store custom commands and a Java SDK to create powerful custom services.

Languages

Currently SEPIA works in German and English with basic support to create custom commands in other common languages. Some services like news and soccer-results are optimized for German meaning you will get an answer in English but might still see a mix of English and German news outlets or soccer results for the Bundesliga. The smart-services are constantly improving though and you can easily edit the list of outlets yourself.

Quick-start (for users)

To use S.E.P.I.A. your personal, digital, open-source voice assistant you need 2 things:

  1. Access to a S.E.P.I.A. server. This can be your own one, running e.g. on a Raspberry Pi or your Windows/Mac PC (see below) or you can find an open one hosted by a friend or a company (Note: we are currently not hosting any public servers).
  2. One of S.E.P.I.A.'s client apps, e.g. the web version: https://sepia-framework.github.io/app/ (hosted on your server as well) or the official Android app

To connect to a custom server simply open the app, change the "hostname" in the log-in screen and restart the app. A typical hostname could be the IP of the server, "raspberrypi.local", "my-server.example.org/sepia" or simply keep "localhost" (for test-servers on the same machine).

Quick-start (for makers)

Basic steps to install the server:

  • Make sure you have Java JDK 8 or 11 installed
  • Download the latest SEPIA-Home bundle from here
  • Extract the zip and run "setup" (.bat for Windows, .sh for Linux/Mac)
  • Optional (advanced users): If you need a reverse-proxy install Nginx and use the included SEPIA configuration
  • Start the server with one of the "run-sepia"-scripts and check out the links mentioned at the end of the process
  • Use any web browser to open the SEPIA Control-HUB (the 'tools' link) and create a new user
  • Continue with "Quick-start (for users)" above :-)

Instructions and an (almost) automatic installation script for Raspberry Pi can be found -HERE-

Instructions for the installation of the S.E.P.I.A. server stack on Linux, Windows or Mac can be found -HERE-

Questions and bug-reports

If you have any questions, need help or want to report a bug please go here or start a discussion here.

API keys for services

Some services integrated in SEPIA require an API key to run properly (e.g. navigation/reverse geo-coding). Find out how to get them (for free) here.

Final notes

If you run your own server and decide to open it to the public or to your friends please make sure it is properly secured and inform the users about your data privacy policy since you are operating a database with potentially sensitive, personal information.

More Repositories

1

sepia-installation-and-setup

Instructions, scripts, tools and files to install, setup and run the SEPIA-Framework on Raspberry Pi, Windows, Mac and other Linux systems.
Shell
122
star
2

sepia-stt-server

SEPIA server to support open-source speech recognition via WebSocket connection.
Python
120
star
3

sepia-assist-server

Core server of the SEPIA Framework responsible for NLU, conversation, smart-service integration, user-accounts and more.
Java
93
star
4

sepia-html-client-app

Application to communicate with SEPIA via browser, iOS and Android. Works as chat messenger with personal-assistant, ASR and TTS integration.
JavaScript
62
star
5

sepia-web-audio

Create modular, cross-browser, web audio pipelines to record and process audio in background threads. Comes with modules for VAD, ASR, resampling and much more...
JavaScript
44
star
6

sepia-sdk-java

Java SDK for SEPIA services
Java
11
star
7

sepia-mesh-nodes

SEPIA Mesh Nodes are small, lightweight servers that can be distributed in your network to run tasks securely.
Java
9
star
8

sepia-admin-tools

Tools to connect to your SEPIA servers, change settings, organize users, test the APIs and more.
JavaScript
8
star
9

sepia-extensions

Extensions (the "E" in SEPIA) are plugins and smart services that enhance SEPIA's capabilities.
Java
7
star
10

SEPIA-Framework.github.io

SEPIA framework website: https://sepia-framework.github.io/
JavaScript
6
star
11

sepia-core-tools-java

The Core-Tools repository contains classes and methods shared by SEPIA servers and e.g. the SDK. It also has useful tools to simplify HTTP calls, file and database access, JSON stuff and more.
Java
4
star
12

sepia-wakeword-tools

Wake-word tools and implementations for S.E.P.I.A.
Python
4
star
13

sepia-teach-server

The teach-API is responsible for adding new stuff to the assistant like answers and commands and it collects feedback.
Java
3
star
14

sepia-open-hardware

A collection of open hardware projects or tutorials useful for voice assistants etc.
3
star
15

sepia-node-js-client

Basic Node.js client for SEPIA with Node-RED support
3
star
16

sepia-reverse-proxy

Tiny reverse proxy for SEPIA-Framework
Java
3
star
17

sepia-websocket-server-java

Java server for SEPIA WebSocket communication interface.
Java
2
star
18

sepia-browser-extensions

Browser extensions that add additional features when using SEPIA in Chromium kiosk-mode etc.
JavaScript
2
star
19

sepia-python-bridge

The Python-bridge-server connects other SEPIA components to the Python runtime.
Python
2
star