Deep-Spying

Spying using Smartwatch and Deep Learning

This repository contains the code implemented for my Master's thesis project submitted in fulfillment of the requirements for the degree of Master of Science at the IT University of Copenhagen supervised by Professor Sebastian Risi.

• A video demo of the system can be seen here
• The paper is available at http://arxiv.org/abs/1512.05616

The following software is shared for educational purpose only. The author of the code and its affiliated institution are not responsible in any manner whatsoever for any damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of the use or inability to use this software. Neither the names of the author or the name of its affiliated institution may be used to endorse or promote products derived from this software. Please find more details in the provided Licence file.

Abstract

Wearable technologies are today on the rise, becoming more common and broadly available to mainstream users. In fact, wristband and armband devices such as smartwatches and fitness trackers already took an important place in the consumer electronics market and are becoming ubiquitous. By their very nature of being wearable, these devices, however, provide a new pervasive attack surface threatening users privacy, among others.

In the meantime, advances in machine learning are providing unprecedented possibilities to process complex data efficiently. Allowing patterns to emerge from high dimensional unavoidably noisy data.

The goal of this work is to raise awareness about the potential risks related to motion sensors built-in wearable devices and to demonstrate abuse opportunities leveraged by advanced neural network architectures.

The LSTM-based implementation presented in this research can perform touchlogging and keylogging on 12-keys keypads with above-average accuracy even when confronted with raw unprocessed data. Thus demonstrating that deep neural networks are capable of making keystroke inference attacks based on motion sensors easier to achieve by removing the need for non-trivial pre-processing pipelines and carefully engineered feature extraction strategies. Our results suggest that the complete technological ecosystem of a user can be compromised when a wearable wristband device is worn.

Keywords

Security, Side-Channel Attack, Keystroke Inference, Motion Sensors, Deep Learning, Recurrent Neural Network, Wearable Computing

International media coverage

• Wired UK
• GEEK
• El Pais
• Vice
• Gizmodo
• Lifehacker Australia
• Huffington Post
• IFL Science
• XDA Developers
• Naked Security
• Softpedia
• Version2
• Forbes
• phys.org

On Danish national TV channel TV2 NewScience
University blog post here
Read comments on Hacker News

Citation

@article{beltramelli2015deep,
  title={Deep-Spying: Spying using Smartwatch and Deep Learning},
  author={Beltramelli, Tony and Risi, Sebastian},
  journal={arXiv preprint arXiv:1512.05616},
  year={2015}
}

Fun fact

The original project name was "SWAT: Spying using Wearable Wristband/Armband Technology", which explains why some packages still reflect this old name.