• Stars
    star
    204
  • Rank 187,557 (Top 4 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 4 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

👁 Sightseer: TensorFlow library for state-of-the-art Computer Vision and Object Detection models



PyPI PyPI - License

State-of-the-art Computer Vision and Object Detection for TensorFlow.

Made by Rishabh Anand • https://rish-16.github.io

sightseer provides state-of-the-art general-purpose architectures (YOLOv3, MaskRCNN, Fast/Faster RCNN, SSD...) for Computer Vision and Object Detection tasks with 30+ pretrained models written in TensorFlow 1.15.

I'd like to fully credit Huynh Ngoc Anh for their YOLOv3 model architecture code. I've repackaged that chunk as a callable python API wrapper under the model zoo. This project would not be possible without their contribution.

Installation

sightseer is written in Python 3.5+ and TensorFlow 1.15.

Ideally, sightseer should be installed in a virtual environments. If you're unfamiliar with Python virtual environments, check out this tutorial on getting started.

Via PyPi

To use sightseer, you must first have TensorFlow installed. To do so, follow the instructions on the TensorFlow installation page.

When your virtual environment is set up with TensorFlow, you can install sightseer using pip:

pip install sightseer

Model Clients (as of now)

  1. YOLOv3Client (Darknet by Joseph Redmon)

By popular demand, Tiny YOLO will be out in the v1.2.0 release. For more information on model release, check out the Roadmap.

Components of sightseer

The package comes with 4 major components that help with different parts of the object detection process all the way from preparing your raw data to getting predictions and displaying them.

Component Description
Sightseer Obtains image data or video footage
Proc Provides image/frame-wise annotation and inter-format conversion tools
Zoo Stores the wrappers over all state-of-the-art models and configs
Serve Provides deployment and model serving protocols and services

If not using custom datasets, Sightseer and Zoo are the submodules majorly used for generic predictions from pre-trained models. When there is custom data involved, you can use Proc to annotate your datasets and even convert them between XML/JSON/CSV/TFRecord formats.

Serve is an experimental productionising submodule that helps deploy your models on cloud services like AWS and GCP. For more details on future tools and services, check out the Roadmap.

Features

Footage or raw images can be rendered using Sightseer before being ingested into models or further preprocessed.

1a. Loading images

from sightseer import Sightseer

ss = Sightseer()
image = ss.load_image("path/to/image") # return numpy array representation of image

1b. Loading videos

from sightseer import Sightseer

ss = Sightseer()
frames = ss.load_vidsource("path/to/video") # returns nested array of frames

Support for video, webcam footage, and screen recording will be out in the coming v1.2.0 release.

2. Using models from sightseer.zoo

Once installed, any model offered by sightseer can be accessed in less than 10 lines of code. For instance, the code to use the YOLOv3 (Darknet) model is as follows:

from sightseer import Sightseer
from sightseer.zoo import YOLOv3Client

yolo = YOLOv3Client()
yolo.load_model() # downloads weights

# loading image from local system
ss = Sightseer()
image = ss.load_image("./assets/road.jpg")

# getting labels, confidence scores, and bounding box data
preds, pred_img = yolo.predict(image, return_img=True)
ss.render_image(pred_img)

To run the model on frames from a video, you can use the framewise_predict method:

from sightseer import Sightseer
from sightseer.zoo import YOLOv3Client

yolo = YOLOv3Client()
yolo.load_model() # downloads weights

# loading video from local system
ss = Sightseer()
frames = ss.load_vidsource("./assets/video.mp4")

"""
For best results, run on a GPU
"""
# getting labels, confidence scores, and bounding box data
preds, pred_frames = yolo.framewise_predict(frames)
ss.render_footage(pred_frames) # plays the video and saves the footage

The module can even be repurposed into a Command-line Interface (CLI) app using the argparse library.

Contributing

Suggestions, improvements, and enhancements are always welcome! If you have any issues, please do raise one in the Issues section. If you have an improvement, do file an issue to discuss the suggestion before creating a PR.

All ideas – no matter how outrageous – welcome!

Before committing, please check the Roadmap to see if proposed features are already in-development or not.

Note: Please commit all changes to the development experimentation branch instead of master.

Licence

Apache Licencse 2.0

More Repositories

1

gpt2client

✍🏻 gpt2-client: Easy-to-use TensorFlow Wrapper for GPT-2 117M, 345M, 774M, and 1.5B Transformer Models 🤖 📝
Python
367
star
2

aft-pytorch

Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.
Python
226
star
3

grafog

Graph Data Augmentation Library for PyTorch Geometric
Python
127
star
4

Angelfire

Add custom context menus to any DOM element 👼🏻 🔥 (NUSHackers OpenHack 2020 Winner)
JavaScript
102
star
5

tokenlearner-pytorch

Unofficial PyTorch implementation of TokenLearner by Google AI
Python
60
star
6

CycleGANsformer

Unpaired Image-to-Image Translation with Transformer-based GANs in PyTorch [WIP]
Python
55
star
7

involution_pytorch

Unofficial PyTorch implementation of the Involution layer from CVPR 2021
Python
45
star
8

Network-Optimisation

Neural Network Hyper-parameter Tuning with Genetic Algorithms 🤖 🧬
Python
34
star
9

mlp-mixer-tf

Unofficial Implementation of MLP-Mixer in TensorFlow
Python
26
star
10

CS3244-Tutorial-Material

All supplementary material used by me while TA-ing CS3244: Machine Learning
Python
21
star
11

Metroplex

Using Simulated Annealing, Hill Climbing, and Genetic Algorithms to recreate images using shapes 🤖 🧬
Python
16
star
12

TransGAN-PyTorch

[WIP] PyTorch implementation of the TransGAN paper
Python
12
star
13

nam-pytorch

Unofficial PyTorch implementation of Neural Additive Models (NAM) by Agarwal, et al.
Python
12
star
14

gpt3-pytorch

Unofficial PyTorch Implementation of OpenAI's GPT-3
Python
11
star
15

dalle2-pytorch

Unofficial PyTorch implementation of DALL-E 2 by OpenAI
Python
10
star
16

CatGAN

Generative Adversarial Network (GAN) that generates cat faces using Google's Quick, Draw! dataset 🤖 😺
Python
10
star
17

Audily

📑🤖 Convert PDF scans to Audiobooks with a click!
Python
8
star
18

Regolith

🌗 An elegant Pseudocode-like programming language built in Python
Python
7
star
19

vit-tensorflow

[WIP] TensorFlow wrapper of Vision Transformer for SOTA image classification
Python
7
star
20

Hringhorni

A Python library for Cellular Automaton simulations
Python
6
star
21

gym-maze

gym environment for randomly-generated mazes
Jupyter Notebook
5
star
22

Neural-Machine-Translation

Neural Machine Translation model to translate English to German and vice versa.
Jupyter Notebook
5
star
23

Strelitzia

Summaries of Hacker News shared links right in your inbox
Python
4
star
24

audio-tf-pytorch

[WIP] Unofficial PyTorch implementation of the "Audio Transformer" by Verma & Berger (2021)
Python
4
star
25

FGSM-Attacks

TensorFlow implementation of Fast Gradient Sign Method Attack, a White Box Adversarial Attack
Python
4
star
26

Reverse-Image-Search

Small-scale reverse image search
Python
4
star
27

cs4248-project

Code for CS4248 Group Project (AY22/23 Sem 2)
Jupyter Notebook
4
star
28

Sheet-to-Bot

Quickly build chatbots using only Google Sheets
Python
3
star
29

open-vector

A computation and visualisation library for vector operations
Python
3
star
30

NachoDB

🧀 A simple sqlite-like database written in C
C
3
star
31

Golang

Learning experiments with Golang
Go
3
star
32

CommandF

Darkflow integration to CommandF
Python
3
star
33

model-boosting

Pure NumPy implementations of popular Boosting techniques
Python
3
star
34

andromeda

Chrome extension that renders LaTeX in Substack articles
JavaScript
3
star
35

HackNRoll-2021

TypeScript
3
star
36

Hybrid-Recommendation-System

A collaborative and content-based movie recommendation system created using Python.
Python
3
star
37

Quinque

Fun image manipulation experiments with Python
Python
3
star
38

Stock-Prediction

Python code to predict Google (Alphabet) stock using QUANDL datasets. Accuracy may vary between 90% to 100%.
Python
3
star
39

cs4243-project

Fast Image Vector Search Tool built in PyTorch
Jupyter Notebook
3
star
40

ChatWithPerson

Generated personas you can chat with 👾👋🏻
Python
3
star
41

mlblocks-beta

Landing page for the MLBlocks beta platform
CSS
2
star
42

Network-Estimation

Mapping the internal representation of a neural network using a k-nearest neighbours model
Python
2
star
43

Back-to-C

Picking up the C syntax after 8 years
C
2
star
44

Cinimax

Minimax Implementation in C
C
2
star
45

Chirp.me

Convert Twitter Threads to Blog Posts
JavaScript
2
star
46

Keras

Keras experiments playground
Python
2
star
47

shrubtest

🌳🧪In-editor API testing and experimentation extension for VS Code
TypeScript
2
star
48

GraphConv-Lightning

⚡️ Graph Convolutional Network built entirely using PyTorch Lightning
Python
2
star
49

gin-attn-conv-pytorch

Unofficial implementation of Graph Isomorphism Network in Python
Python
2
star
50

gist2blog

Converting a collection of GitHub Gists into a blog
JavaScript
2
star
51

Actor-Critic-Model

TensorFlow implementation of the Actor-Critic model
Python
2
star
52

Mario-Source

Implementation of Mario using runes from Source v1
JavaScript
2
star
53

3D-Environment-Navigation

Python
2
star
54

locoprop-pytorch

Unofficial PyTorch implementation of LocoProp, an enhanced BackProp alternative
Python
2
star
55

BBCS-ML-Winpetition

Datasets and resources for the BuildingBloCS 2019 Conference
2
star
56

Leetcode-Experiments

A repo where I store some basic Leetcode experiments
Python
2
star
57

Vocq-Neural-Summariser

GitHub issues summariser based on Neural Network applications
Python
2
star
58

Long-Short-Term-Memories

A podcast hosted by 2 DialoGPT Conversational Agents
Python
2
star
59

Team-Unicorn-CXA-2019

A platform for SMEs to connect with local producers and sellers in the area and keep track of their inventories and sales in a seamless manner
JavaScript
2
star
60

COVID-Viz

JavaScript
2
star
61

CS2109S-Teaching-Material

Resources used when teaching CS2109S in AY21/22 Sem 2 at NUS Computing
2
star
62

gaia-vscode

The dark Earth VS Code theme to cybernetically enhance your programming
2
star
63

Cheery

Cheery website frontend
CSS
2
star
64

Neural-Additive-Models

TensorFlow 2.0 implementation of Neural Additive Models
Python
2
star
65

Proximal-Policy-Optimization

PPO algorithm on David Ha's Slime Volley environment
2
star
66

PCADA_Demo_1

Source code for SMU Academy PCADA Module 3: Web Scraping and Natural Language Processing
HTML
2
star
67

KL-Divergence

A few experiments with KL Divergence for different probability distributions
Python
2
star
68

SimCLR-pytorch

Unofficial PyTorch implementation of SimCLR by Google Brain
Python
2
star
69

Simulated-Annealing-Attack

Attacking Neural Networks using Stochastic Optimisation
Python
2
star
70

convnext_pytorch

Unofficial PyTorch implementation of ConvNeXt by Facebook AI Research (FAIR)
2
star
71

Lightning-Transformer

Transformer and Attention Mechanism written in PyTorch Lightning ⚡️
Python
2
star
72

Minimax-Heuristic-Search-T3

Using Minimax to generate Tic Tac Toe rollouts
Python
2
star
73

Sobel-Filters

A few experiments with the Solber-Feldman Operator for traditional Edge Detection
Python
2
star
74

Singer2Singer

[WIP] Inter-singer audio/song conversion using Deep Learning
Python
2
star
75

NodeJS-MongoDB-Blog

A complete API for blog posts using a MongoDB AWS backend 📑
HTML
2
star
76

Self-driving-car

A lane detection, vehicle detection and path finding algorithm
Python
1
star
77

Coolor

Syntax highlighting Chrome Extension for code blocks in Medium articles
JavaScript
1
star
78

Stylus.js

CSS substitute fully written in JavaScript to supercharge the process of styling elements.
JavaScript
1
star
79

MLBlocks-Phase-2

Frontend for MLBlocks Phase 2
JavaScript
1
star
80

Natural-Language-Processing

NLP experiments playground
Python
1
star
81

hello-github-actions

1
star
82

Colour-Pop

A web-app to perform the "Colour Pop" effect on images
JavaScript
1
star
83

turismo

Lightweight PyTorch Geometric library to build graph transformers quickly [WIP]
1
star
84

OpenAI-Problems

OpenAI Reinforcement Learning and Algorithms Playground
Jupyter Notebook
1
star
85

pytorch-perceiver

PyTorch implementation of the Perceiver model by DeepMind
Python
1
star
86

Lingua

An open-source embeddable component for code snippets with multi-lingual support
CSS
1
star
87

heimdall

Heimdall: access the power of GPT-3 straight from Siri on any apple product :D
Python
1
star
88

pytorch-graphdl

PyTorch Implementation of popular Graph Neural Networks
Python
1
star
89

gym-navmaze

Custom gym maze environment
Python
1
star
90

H2-Computing-Practicals

Python
1
star
91

PyTorch

Jupyter Notebook
1
star
92

gnn-training-tricks

Analysing the impact of training tricks of GNN optimisation
Python
1
star
93

Opt-In-Opt-Out

What happens when you use an optimiser to optimise an optimiser?
Python
1
star
94

linearly

Discount NumPu implementation using barebones Linear Algebra concepts
Python
1
star
95

Python-Decision-Trees

Pure Python implementation of Decision Trees trained on the Iris Dataset
1
star
96

cs3244-project

Source code for CS3244 AY22/23 Sem 1 Group Project
1
star
97

HackerRank

HackerRank algorithms playground
Python
1
star
98

Artificial-Neural-Networks

A small-scale ANN to predict XOR values of binary digits (Binary Classification)
Python
1
star
99

ANN-with-swapped-training-sets

Implementation of a Deep Neural Network with swapped training and tests sets. Instead of training on pixel values to predict class labels, this implementation trains on the class labels to predict pixel values for each class.
Python
1
star
100

Tumour-Analysis

A python script to predict likelihood of Malignant or Benign Tumours in a patient using 30 different features.
Python
1
star