• Stars
    star
    941
  • Rank 47,585 (Top 1.0 %)
  • Language
    Python
  • Created about 10 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Automatically collect and visualize usage statistics in Ubuntu/OSX environments.

ulogme

How productive were you today? How much code have you written? Where did your time go?

Keep track of your computer activity throughout the day: visualize your active window titles and the number of keystrokes in beautiful HTML timelines. Current features:

  • Records your active window title throughout the day
  • Records the frequency of key presses throughout the day
  • Record custom note annotations for particular times of day, or for day in general
  • Everything runs completely locally: none of your data is uploaded anywhere
  • Beautiful, customizable UI in HTML/CSS/JS (d3js).

The project currently only works on Ubuntu and OSX, and uses new fancy Promises feature of ECMAScript 6. This might not be implemented in all browsers. My Chrome has it, but for example my Firefox doesn't.

Demo

See a blog post (along with multiple screenshots) describing the project here.

Getting Started

To start recording

  1. Clone the repository to some folder: $ git clone https://github.com/karpathy/ulogme.git
  2. If you're on Ubuntu, make sure you have the dependencies: $ sudo apt-get install xdotool wmctrl. On Fedora, you may also need sudo yum install gnome-screensaver.
  3. cd inside and run $ ./ulogme.sh (note: this will ask you for sudo authentication which is required for showkey command). This will launch two scripts. One records the frequency of keystrokes and the other records active window titles. Both write their logs into log files in the logs/ directory. Every log file is very simply just the unix time stamp followed by data, one per line.
  4. For OSX only: there might be an additional step where you have to go to System Preferences > Security & Privacy > Accessibility, and make sure that Terminal (or iTerm2, or whatever you use to launch ulogme) is checked. If it wasn't checked previously and you just checked it, you may need to restart ulogme. If you don't do this step, you might find that window logging works but keypress logging doesn't.

The user interface

  1. Important. As a one-time setup, copy over the example settings file to your own copy: $ cp render/render_settings_example.js render/render_settings.js to create your own render_settings.js settings file. In this file modify everything to your own preferences. Follow the provided example to specify title mappings: A raw window title comes in, and we match it against regular expressions to determine what type of activity it is. For example, the code would convert "Google Chrome - some cool website" into just "Google Chrome". Follow the provided example and read the comments for all settings in the file.
  2. Once that's set up, start the web server viewer: $ python ulogme_serve.py, and go to to the provided address) for example http://localhost:8123) in your browser. Hit the refresh button on top right every time you'd like to refresh the results based on most recently recorded activity
  3. If your data isn't loading, try to explicitly run python export_events.py and then hit refresh. This should only be an issue the very first time you run ulogme.

User Interface

The user interface can switch between a single day view and an overview view by link on top. You have to hit the refresh button every time you'd like to pull in new data.

Single day page

  • You can enter a reminder "blog" on top if you'd like to summarize the day for yourself or enter other memos.
  • Click on any bar in the barcode view to enter a custom (short) note snippet for the time when the selected activity began. I use this to mark meetings, track my coffee/food intake, sleep time, or my total time spent running/swimming/gym or to leave notes for certain patterns of activity, etc. These could all later be correlated with various measures of productivity, in future.

Overview page

  • You can click the window titles to toggle them on and off from the visualization
  • Clicking on the vertical bars takes you to the full statistics for that day.

Known issues

  • One Ubuntu user reported broken view with no data. On further inspection we found that the logs were corrupt. One of the lines in a file in /logs was, instead of looking as {timestamp} {data} looked as @@@@@@@{timestamp} {data}, in other words an odd character was appended to the timestamp somehow. We manually erased these characters from the log file to fix the issue.
  • Legacy code note: if you used ulogme from before 28 July, you will have to run $ python legacy_split_events.py to convert your events files, once.
  • You may see "address already in use" if you try to run python ulogme_serve.py. Sometimes the system can get confused and takes a while to update what ports are being used. Use the optional argument to specify a different port, for example python ulogme_serve.py 8124 and then go to http://localhost:8124 instead, for example.
  • Overview page is blank. Are you sure your browser supports ECMAScript 6? Chrome should be fine, Firefox might not be, yet.

Contributing

The Ubuntu and OSX code base are a little separate on the data collection side. However, they each just record very simple log files in /logs. Once the log files are written, export_events.py takes the log files, does some simple processing and writes the results into .json files in /render. The Javascript/HTML/CSS UI codebase is all common and all lives in /render.

Ubuntu

ulogme has three main parts:

  1. Recording scripts keyfreq.sh and logactivewin.sh. You probably won't touch these.
  2. Webserver: ulogme_serve.py which wraps Python's SimpleHTTPServer and does some basic communication with the UI. For example, the UI can ask the server to write a note to a log file, or for a refresh.
  3. The UI. Majority of the codebase is here, reading the .json files in /render and creating the visualizations. There are several common .js files, and crucially the index.html and overview.html files. I expect that most people might be able to contribute here to add features/cleanup/bugfix.

OSX code

Things get a little ugly in OSX if you want to change anything with recording the log files because you have to recompile these portions any time you make any changes. It's ugly and it had to be done. However, if you're only interested in hacking with the UI, just change Javascript in render and no recompile is necessary, naturally.

Related projects

  • selfspy, Log everything you do on the computer, for statistics, future reference and all-around fun.
  • arbtt, automatic rule-based time tracker.

For more projects, this question on Personal Productivity Stack Exchange might be also worth a look.

License

MIT

More Repositories

1

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
Python
22,607
star
2

minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
Python
15,735
star
3

char-rnn

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
Lua
11,228
star
4

convnetjs

Deep Learning in Javascript. Train Convolutional Neural Networks (or ordinary ones) in your browser.
JavaScript
10,642
star
5

nn-zero-to-hero

Neural Networks: Zero to Hero
Jupyter Notebook
8,476
star
6

micrograd

A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API
Jupyter Notebook
5,613
star
7

neuraltalk2

Efficient Image Captioning code in Torch, runs on GPU
Jupyter Notebook
5,426
star
8

neuraltalk

NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.
Python
5,352
star
9

arxiv-sanity-preserver

Web interface for browsing, search and filtering recent arxiv submissions
Python
4,943
star
10

ng-video-lecture

Python
2,074
star
11

reinforcejs

Reinforcement Learning Agents in Javascript (Dynamic Programming, Temporal Difference, Deep Q-Learning, Stochastic/Deterministic Policy Gradients)
HTML
1,273
star
12

makemore

An autoregressive character-level language model for making more things
Python
1,217
star
13

cryptos

Pure Python from-scratch zero-dependency implementation of Bitcoin for educational purposes
Jupyter Notebook
1,142
star
14

randomfun

Notebooks and various random fun
Jupyter Notebook
996
star
15

recurrentjs

Deep Recurrent Neural Networks and LSTMs in Javascript. More generally also arbitrary expression graphs with automatic differentiation.
HTML
918
star
16

arxiv-sanity-lite

arxiv-sanity lite: tag arxiv papers of interest get recommendations of similar papers in a nice UI using SVMs over tfidf feature vectors based on paper abstracts.
Python
864
star
17

tsnejs

Implementation of t-SNE visualization algorithm in Javascript.
JavaScript
815
star
18

pytorch-normalizing-flows

Normalizing flows in PyTorch. Current intended use is education not production.
Jupyter Notebook
790
star
19

paper-notes

Random notes on papers, likely a short-term repo.
660
star
20

svmjs

Support Vector Machine in Javascript (SMO algorithm, supports arbitrary kernels) + GUI demo
JavaScript
636
star
21

pytorch-made

MADE (Masked Autoencoder Density Estimation) implementation in PyTorch
Python
510
star
22

karpathy.github.io

my blog
CSS
472
star
23

lecun1989-repro

Reproducing Yann LeCun 1989 paper "Backpropagation Applied to Handwritten Zip Code Recognition", to my knowledge the earliest real-world application of a neural net trained with backpropagation.
Jupyter Notebook
425
star
24

deep-vector-quantization

VQVAEs, GumbelSoftmaxes and friends
Jupyter Notebook
422
star
25

covid-sanity

Aspires to help the influx of bioRxiv / medRxiv papers on COVID-19
Python
351
star
26

find-birds

Find people you should follow on Twitter based on who the people you follow follow
Python
305
star
27

forestjs

Random Forest implementation for JavaScript. Supports arbitrary weak learners. Includes interactive demo.
JavaScript
284
star
28

researchlei

An Academic Papers Management and Discovery System
Python
194
star
29

Random-Forest-Matlab

A Random Forest implementation for MATLAB. Supports arbitrary weak learners that you can define.
MATLAB
172
star
30

researchpooler

Automating research publications discovery and analysis. For example, ever wish your computer could automatically open papers that are most similar to a paper at an arbitrary url? How about finding all papers that report results on some dataset? Let's re-imagine literature review.
Python
167
star
31

nipspreview

Scripts that generate .html to more easily see NIPS papers
Python
147
star
32

ttmik

Talk to me in Korean Anki cards and related scripts
Python
103
star
33

tf-agent

tensorflow reinforcement learning agents for OpenAI gym environments
Python
99
star
34

gitstats

A lightweight/pretty visualizer for recent work on a git code base in multiple branches. Helps stay up to date with teams working on one git repo in many branches.
HTML
85
star
35

EigenLibSVM

A wrapper for LibSVM that lets you train SVM's directly on Eigen library matrices in C++
C++
74
star
36

MatlabWrapper

C++ convenience class to communicate with a Matlab instance. Send matrices back and forth, execute arbitrary Matlab commands, or drop into interactive Matlab session right in the middle of your C++ code.
C++
52
star
37

twoolpy

useful scripts to work with Twitter + Python. Requires the tweepy library.
Python
50
star
38

notpygamejs

Game making library for using Canvas element
JavaScript
41
star
39

scholaroctopus

A set of tools/pages that help explore academic literature
33
star
40

karpathy

root repo
19
star