• Stars
    star
    941
  • Rank 47,269 (Top 1.0 %)
  • Language
    Python
  • License
    MIT License
  • Created about 6 years ago
  • Updated about 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A toy implementation of monocular SLAM written while livestreaming

twitchslam

A toy implementation of monocular SLAM written while livestreaming.

Stream

https://www.twitch.tv/tomcr00s3

Will be streaming again tomorrow. May 28, 2018

By the end of the day, twitchslam will be decent.

Usage

export REVERSE=1   # Hack for reverse video
export F=500       # Focal length (in px)

./slam.py <video.mp4>

# good example
F=525 ./slam.py videos/test_freiburgxyz525.mp4

# ground truth
F=525 ./slam.py videos/test_freiburgrpy525.mp4 videos/test_freiburgrpy525.npz

# kitti example
REVERSE=1 F=984 ./slam.py videos/test_kitti984_reverse.mp4

# extract ground truth
tools/parse_ground_truth.py videos/groundtruth/freiburgrpy.txt videos/test_freiburgrpy525.npz 

Classes

  • Frame -- An image with extracted features
  • Point -- A 3-D point in the Map and it's 2-D Frame correspondences
  • Map -- A collection of points and frames
  • Display2D -- SDL2 display of the current image
  • Display3D -- Pangolin display of the current map

Libraries Used

  • SDL2 for 2-D display
  • cv2 for feature extraction
  • pangolin for 3-D display
  • g2opy for optimization (soon!)

Rendered Scene Test

./fakeslam.py

NOTE: The test currently doesn't work reliably. It seems adding a small amount of Gaussian noise to the point positions can cause the optimizer to fall into really bad local minima. This may just be caused by poor initialization, as I'm not sure how stable Essential matricies are.

TODO: Investigate the stability of Essential matrix recovery.

DONE

  • BUGFIX: Why is window initting small? (fixed? ish, zoom is broken sometimes)
  • BUGFIX: Turning doesn't work well (fixed?)
  • Stop using essential matrix for pose estimation once you have a track (done)
  • Add kinematic model (done)
  • Run g2o to only optimize the latest pose (done)
  • Add search by projection to refind old map points (done)
  • Check if points are in the field of view of the camera (done)
  • BUGFIX: Fix moving up! (y axis flipped? nah, it's okay)
  • Add loading and saving of map support

TODO

  • Investigate if we need KeyFrames!
  • BUGFIX: Improve lockups to happen less
  • Improve init to not need REVERSE environment variable
  • Add optimizer for F
  • Add multiscale feature extractor
  • Add Linux libraries and OS check
  • Profile and speed up more (tomorrow 6/1/18)
  • Profile with flame!
  • Search by projection less stupidly
  • Don't add all points to the optimization graph for pose at least
  • Add automated test for freiburg running on commit
  • Check accuracy with ground truth

LICENSE

All my code is MIT licensed. Videos and libraries follow their respective licenses.

More Repositories

1

qira

QEMU Interactive Runtime Analyser
C
3,806
star
2

fromthetransistor

From the Transistor to the Web Browser, a rough outline for a 12 week course
3,512
star
3

minikeyvalue

A distributed key value store in under 1000 lines. Used in production at comma.ai
Go
2,791
star
4

corona

Reverse engineering SARS-CoV-2
Python
2,450
star
5

ai-notebooks

Some ipython notebooks implementing AI algorithms
Jupyter Notebook
959
star
6

configuration

Like some files bro
Haskell
379
star
7

tinyvoice

Letting computers listen to you and really care
Jupyter Notebook
361
star
8

twitchchess

like twitchslam, for chess
Python
349
star
9

lolrecaptcha

We try to break the recaptcha for the Merry Christmas for all!
Go
292
star
10

mergesorts

mergesort in many languages
Shell
254
star
11

twitchcore

It's a core. Made on Twitch.
Verilog
229
star
12

cuda_ioctl_sniffer

Sniff CUDA ioctls
C
147
star
13

eda-reversing

The Embedded Disassembler
C++
110
star
14

kvm-kext

An implementation of /dev/kvm for Mac OS X
C
108
star
15

twitchcoq

It's a poorly named metamath verifier
Prolog
104
star
16

twitchtactoe

Tic Tac Toe in React because it is Simple Skills Sunday
JavaScript
102
star
17

battlechess

A distributed decentralized chess tournament
Python
99
star
18

tinyxxx

tiny corporation website
HTML
96
star
19

hammer-website

HTML
71
star
20

edgetpuxray

Enabling tinygrad compatibility with the Google Edge TPU
C++
68
star
21

pie

Computing digits of pi for the people
JavaScript
68
star
22

eda-2

Even better than eda-reversing...I hope
C++
61
star
23

haskell-scheme

Writing Scheme in Haskell
Haskell
58
star
24

twitchctw

compression = AI
Python
53
star
25

coq-hardy

Formalizing the Theorems from Hardy's "An Introduction to the Theory of Numbers" in coq
Coq
52
star
26

freethedsp

For winners only. Are you a winner?
C
40
star
27

twitchcoins

Python
36
star
28

openhexagon

An attempt at an open source toolchain for the Hexagon DSP
Shell
35
star
29

crappycase

So many shitty coders: Adobe, Blizzard, Valve. This is a case insensitivity emulator.
C
29
star
30

body_loop

comma body does a loop around the office
Python
28
star
31

amdgpu-dkms

Unpacking AMD's dkms packages
C
25
star
32

jenkyiphonetools

iPhone Tools of the lowest quality
Python
25
star
33

lowqualityraytracer

ever wonder how to raytrace? me too. i love america
Python
25
star
34

commaled

comma.ai LED controller cause the car needs some lights bro. SWAG!
Assembly
25
star
35

trinity-osxnew

C
22
star
36

aes_serial

There is so much swag in the world, just some of it is hidden -- Gandalf
C
17
star
37

eda-3

eda-3 from many years ago
JavaScript
13
star
38

collfun

It's Christmas time, you know what it is
Python
11
star
39

nnweights

6
star
40

7900xtx

5
star
41

gpysieve

ghetto sieves in python that don't work
Python
4
star
42

angr-travis

Run travis-ci testing on release version of angr
Shell
4
star
43

tt06-fp4-mac

FP4 MAC Array
Tcl
3
star
44

tt-twitch

tenstorrent kernel from twitch
C++
2
star