SmartMirror

It's a Magic Mirror, you all know what it is. This is a complete rewrite in React and Node.js of my original Magic Mirror written in Java. Everything is improved and it has lots of new functions.

Both voice recognition and the UI is in Swedish, however English is also supported by setting language to 'en-US' in the config. For porting to other languages take a look in the locales folder, should be pretty straight forward

Images

My Home automation and monitoring app which can control the mirror supports iOS, however you must pay to be a developer to run it on an actual device, so I made a simple web app to control the mirror from.

Features

Voice Recognation using Google Cloud Speech

Since Google Cloud Speech costs, I use Snowboy to listen for my hotword "Spegel" or "Mirror" on english. When it's detected the Mirror records untill end of speech and sends it to Google Cloud Speech.

• Create reminders on my Home automation and monitoring backend.
• Create tasks in a Wunderlist list.
• Control Philips Hue Lights. Turn on and off individual lamps or all, for example "Turn the lights off in the bedroom" or similar.
• Hide/show individual elements on the Mirror
• Turning off the Mirror
• When reading articles/blogs you can skip to next article

All voice parsing is done very simple, however I tried to make all commands as dynamic as possible. My implementation is nothing fancy, but it works very well with the features I have right now. It looks for keywords in the spoken senetence and tries to extract it's meaning. Extending it with a lot of speech commands will most likely result in chaos.

Reminders can be created in various ways (In Swedish, but translated here to english):

• Remind [me] in 20 minutes/hours/days to take a walk.
• Remind [me] to walk the dog in 2 minutes/hours/days.
• Remind [me] tomorrow at 12:35 to do the dishes.
• Remind [me] on weekday to wash the car.
• Remind [me] to buy/shop [more/extra] milk. // Adds it to a Wunderlist list (Shopping list in my case)

When a reminder is successfully created, the Mirror will speak something like "Will remind you to do X".

Led strip

Behind the Mirror I have a RGB led strip with individually controllable leds. The leds can be controlled from my Home automation and monitoring app.

Components

All components can be individually shown/hidden by the Web App/API endpoint. The default behaviour is configured in the config.

Shopping list

It displays the items on a Wunderlist list (in my case our shopping list).

Weather

Displays current weather including temperature outside, feels like temperature wind speed and direction. Also sunrise/sunset and 5 day forecast.

Inside temperature

Which is retreived from a DS18B20 temperature sensor connected to the Raspberry PI running the Mirror.

Motion detection

Motion detection is retreived from a connected PIR motion detector connected to the Raspberry PI. When motion is detected, a message will appear on the Mirror. It changes depending on the time of the day. Also it notifies my Home automation and monitoring backend that motion was detected, so my Home automation and monitoring app will notify me if I'm not at home and someone is in my apartment.

Swedish news from SVT

Headlines and short descriptions are changing in the bottom of the Mirror every 20 seconds or so.

Spotify currently playing

Shows your currently playing track, because why not.

Moisture level of a plant

Shows if your plant needs watering, data is sent from a battery powered esp8266 with a moisture sensor, placed in one of our plants.

OctoPi

Shows some information of your current 3D print, such as the filename, bed/nozzle temperatures and the progress.

Running locally

Set up config file

1. Copy and rename config.js.default to config.js
2. Edit config.js and select which modules/functions you want to enable

export const config = {
  YAHOO_WOEID: '897819',
  SMHI_COORD: {
    longitude: '13',
    latitude: '55.6'
  },
  svtNewsUrl: 'https://api.rss2json.com/v1/api.json?rss_url=http%3A%2F%2Fwww.svt.se%2Fnyheter%2Frss.xml',

  serverBaseURL: process.env.NODE_ENV === 'production' ? 'http://localhost:3001' : 'http://localhost:3000',
  wsServerBaseURL: process.env.NODE_ENV === 'production' ? 'localhost:3001/' : 'localhost:3001/',

  modules: {
    dateTime: true,
    wunderlistTasks: false,
    transfer: false,
    weather: true,
    forecast: true,
    news: true,
    tempPirSensor: false,
    googleCloudSpeech: false,
    philipsHue: false,
    temperatureGraph: false,
    articles: false,
  }

};

Add .env file to add custom config parameters (optional)

RuleThemAllBackendAccessToken=''
wunderlistAccessToken=''
wunderlistClientID=''
wunderlistListID='' // The ID of the list to fetch/create tasks in.

GOOGLE_APPLICATION_CREDENTIALS = '/home/user/../credentials.json'
GCLOUD_PROJECT = 'project_ID'

HUE_HOSTNAME='192.168.X.X' // Your bridge IP
HUE_USERNAME='username' // See https://github.com/peter-murray/node-hue-api

target='Krantz-Ubuntu' // Set to 'PI' on your Raspberry Pi. Avoids errors initializing gpio when not on Pi.

Run it

NOTE: This project is only tested on Node v7.6.0. New versions of Node may not work. If you get wierd errors, downgrade to Node v7.6.0.

git clone https://github.com/jakkra/SmartMirror.git
cd SmartMirror
npm i

cd client
npm i

cd ..
npm start

Note: The current package.json are compatible with node 7.6.0 If you get errors building with later version of node, use below to downgrade:

sudo npm cache clean -f
sudo npm install -g n
sudo n 7.6.0

Code style

prettier --print-width 120 --jsx-bracket-same-line --single-quote --trailing-comma es5 --write ./**/*.js

Building

Running npm run build creates the static bundle.

cd client/
npm run build

Move client/webApp-move2build to client/build and rename it to whatever. It's accessed on / afterwards.

Solutions

Webpack doesn't reload when saving:

$echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

Auto open chrome in SSH on RaspberryPi:

export DISPLAY=:0.0
npm start

Useful commands

$ sudo apt-get install unclutter
$ unclutter -display :0.0 -idle 5 # Hides cursor after 5 seconds of inactivity

$ export DISPLAY=:0.0 # When starting chromium over SSH this must be done
$ chromium-browser --kiosk --incognito http://localhost:3001 # Launch Chromium in kiosk mode

$ sudo apt-get install -y fonts-tlwg-sawasdee # Installs the font I use.

$ sudo iptables -t nat -I PREROUTING --source 0/0 --destination 0/0 -p tcp --dport 80 -j REDIRECT --to-ports 3001 # Redirect traffic on port 80 to our server at 3001. Allows access on local network to web app on url <hostname>/app.
$ sudo apt-get install iptables-persistent # Keep iptables after reboots
# Flashing new software to the arduino from the terminal
# Install libraries
$ arduino --install-library "Adafruit NeoPixel"
$ arduino --install-library "MsTimer2"

$ arduino --board arduino:avr:uno --port /dev/ttyACM0 --save-prefs # store config
$ arduino --upload $(pwd)/ledstrip.ino # Flashes the new software

#Find port of CC2531 USB stick
$ls -l /dev/serial/by-id 

Starting and updating the mirror on boot

Save this file somewhere:

#!/bin/sh

cd /home/pi/Documents/SmartMirror # Edit to path of project

git fetch

if [ $(git rev-parse HEAD) != $(git rev-parse @{u}) ]; then
  echo "Updating mirror"
  git stash # Just incase we wanted to save some change
  git reset --hard origin/master
  cd client
  npm run build
  cd ..
  cp -r client/webApp-move2build client/build/app
fi

NODE_ENV=production /usr/bin/npm run server > mirrorLog.txt  &
export DISPLAY=:0.0
/bin/sleep 30 # Let the server startup before trying to access the website, otherwise we get page not found.
sudo -u pi chromium-browser --kiosk --incognito http://localhost:3001 # Chromium must not be ran as root.

Add @/home/pi/start.sh to ~/.config/lxsession/LXDE-pi/autostart