• Stars
    star
    566
  • Rank 78,749 (Top 2 %)
  • Language
    JavaScript
  • Created over 11 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Converts an audio stream to speech events in the browser

Hark

Hark is a tiny browser/commonJS module that listens to an audio stream, and emits events indicating whether the user is speaking or not.

With browserify:

npm install hark

Without browserify download and use:

hark.bundle.js

Example:

npm install hark

If you aren't using browserify, you'll want hark.bundle.js.

  var hark = require('../hark.js')

  var getUserMedia = require('getusermedia')

  getUserMedia(function(err, stream) {
    if (err) throw err

    var options = {};
    var speechEvents = hark(stream, options);

    speechEvents.on('speaking', function() {
      console.log('speaking');
    });

    speechEvents.on('stopped_speaking', function() {
      console.log('stopped_speaking');
    });
  });

How does hark work?

Hark uses the webaudio API to FFT (get the power of) the audio in the audio stream. If the power is above a threshold, it's determined to be speech.

Usage

var speech = hark(stream, options);
speech.on('speaking', function() {
  console.log('Speaking!');
});
  • Pass hark either a webrtc stream which has audio enabled, or an audio element, and an optional options hash (see below for options).
  • hark returns an event emitter with the following events:
    • speaking emitted when the stream appears to be speaking
    • stopped_speaking emitted when the audio doesn't seem to be speaking
    • volume_change emitted on every poll event by the event emitter with the current volume (in decibels) and the current threshold for speech
  • The hark object also has the following methods to update the config of hark. Both of these options can be passed in on instantiation, but you may wish to alter them either for debug or fine tuning as your app runs.
    • setInterval(interval_in_ms) change
    • setThreshold(threshold_in_db) change the minimum volume at which the audio will emit a speaking event
  • hark can be stopped by calling this method
    • stop() will stop the polling and events will not be emitted.

Options

  • interval (optional, default 100ms) how frequently the analyser polls the audio stream to check if speaking has started or stopped. This will also be the frequency of the volume_change events.
  • threshold (optional, default -50db) the volume at which speaking/stopped\_speaking events will be fired
  • play (optional, default true for audio tags, false for webrtc streams) whether the audio stream should also be piped to the speakers, or just swallowed by the analyser. Typically for audio tags you would want to hear them, but for microphone based webrtc streams you may not to avoid feedback.
  • audioContext (optional, default is to create a single context) If you have already created an AudioContext, you can pass it to hark to use it instead of an internally generated one.

Understanding dB/volume threshold

Fine tuning the volume threshold is the main configuration setting for how this module will behave. The level of -50db have been chosen based on some basic experimentation on mysetup, but you may wish to change them (and should if it improves your app).

What is dB? Decibels are how sound is measured. The loudest sounds on your system will be at 0dB, and silence in webaudio is -100dB. Speech seems to be above -50dB depending on the volume and type of source. If speaking events are being fired too frequently, you would make this number higher (i.e. towards 0). If they are not firing frequently enough (you are speaking loudly but no events are firing), make the number closer to -100dB).

Demo:

Clone and open example/index.html or view it online

Requirements:

  • Chrome 27+, remote streams require Chrome 49+
  • Firefox
  • Microsoft Edge, support for remote streams is under consideration

License

MIT

More Repositories

1

otalk-im-client

Stanza.io webrtc capable xmpp client <3
JavaScript
306
star
2

getUserMedia

Cross-browser getUserMedia shim with a node.js style error-first API
JavaScript
233
star
3

RTCPeerConnection

A tiny browser module that gives normalizes and simplifies the API for WebRTC peer connections.
JavaScript
200
star
4

getScreenMedia

A browser module for attempting to get access to a MediaStream of a user's screen. With a nice node-like API.
JavaScript
154
star
5

restund

Restund. Forked from http://www.creytiv.com/
C
127
star
6

jingle.js

A generic Jingle session manager implementation, suitable for integration by other XMPP libraries.
JavaScript
89
star
7

attachMediaStream

Simple abstraction for handling differences between browsers for attaching a media stream to a video element
JavaScript
48
star
8

iOS-demo

Objective-C
46
star
9

sdp

SDP parsing and serialization utlities
JavaScript
46
star
10

mediastream-gain

A tiny browser module for creating a gain/volume controller for the audio channels in a MediaStream.
JavaScript
44
star
11

webrtc-ios

Objective-C
44
star
12

sdp-jingle-json

A parser/serializer for SDP to JSON. Useful for converting SDP to other formats like Jingle for WebRTC signalling
JavaScript
41
star
13

webrtc-tester

WebRTC Deployment Testing Toolkit
Shell
34
star
14

TLKSimpleWebRTC

Objective-C
28
star
15

jxt

JSON/XML Translation for the Browser
JavaScript
26
star
16

jxt-xmpp

JXT definitions for XMPP
JavaScript
23
star
17

stunturncheck

Check WebRTC STUN and TURN connectivity
JavaScript
17
star
18

localmedia

WebRTC abstraction for creating and managing local media streams.
JavaScript
16
star
19

rtcpeerconnection-shim

Implementation of the RTCPeerConnection API ontop of ORTC
JavaScript
15
star
20

TLKWebRTC

Objective-C
13
star
21

filetransfer

filetransfer via a webrtc datachannel
JavaScript
13
star
22

mod_muc_focus

Prosody + Jingle/COLIBRI = multimedia MUC!
Lua
10
star
23

traceablepeerconnection

Lowlevel RTCPeerConnection wrapper that traces API calls
JavaScript
10
star
24

rtt-buffer

Realtime Text Buffer
JavaScript
9
star
25

jingle-media-session

Jingle Media Session
JavaScript
8
star
26

xmpp-jid

Parse & manage XMPP addresses
JavaScript
7
star
27

otalk-media-controller

Track local & remote media streams
JavaScript
6
star
28

xmpp-uri

Parse XMPP URIs
JavaScript
5
star
29

jxt-xmpp-types

JXT types for working with XMPP data
JavaScript
5
star
30

otalk-media-stream-view

Video/audio box, with controls
JavaScript
5
star
31

sdp-jingle-table

A parser/serializer for SDP to Lua tables. Useful for converting SDP to other formats like Jingle for WebRTC signaling.
Lua
4
star
32

xmpp-constants

Protocol constants for XMPP
JavaScript
4
star
33

rtcpeerconnection-jingle

rtcpeerconnection, extended with a JSON format. And easy to map to Jingle
JavaScript
3
star
34

hostmeta.js

Easily retrieve and parse a host's /.well-known/host-meta[.json] file.
JavaScript
3
star
35

mod_influxdb

Prosody Metrics to Influxdb
Lua
3
star
36

jingle-session

Generic base Jingle session
JavaScript
3
star
37

simplewebrtc.com

This site is DEPRECATED. simplewebrtc.com is a project of &yet and has been moved to andyet/simplewebrtc.com
CSS
3
star
38

jingle-filetransfer-session

Jingle FileTransfer Session
JavaScript
3
star
39

otalk-model-media

A model for WebRTC media streams
JavaScript
2
star
40

otalk-model-message

Ampersand model for XMPP messages
JavaScript
2
star
41

otalk-specs

XMPP extensions for Otalk functionality
XSLT
2
star
42

otalk-model-peer

A base model for XMPP peer entities
JavaScript
1
star
43

mod_muc_allowners

Lua
1
star
44

otalk-media-devices

Track user media devices
JavaScript
1
star
45

mod_turncredentials

XEP-0215 implementation for the prosody xmpp server
Lua
1
star
46

otalk-model-disco

Model for XMPP service discovery information
JavaScript
1
star
47

mod_muc_allhidden

Make all MUC rooms hidden by default
Lua
1
star
48

otalk.org

otalk.org website
CSS
1
star