• Stars
    star
    106
  • Rank 325,871 (Top 7 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 7 years ago
  • Updated about 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Multiframe Image Deconvolution (ICCV17)

Learning Blind Motion Deblurring

TensorFlow implementation of multi-frame blind deconvolution:

Learning Blind Motion Deblurring
Patrick Wieschollek, Michael Hirsch, Bernhard Schölkopf, Hendrik P.A. Lensch
ICCV 2017

Download results from the paper. We propose to use the saccade-viewer to compare images qualitatively.

results

Prerequisites

1. Get YouTube videos

The first step is to gather videos from some arbitrary sources. We use YouTube to get some videos with diverse content and recording equipment. To download these videos, we use the python-tool youtube-dl.

pip install youtube-dl --user

Some examples are given in download_videos.sh. Note, you can use whatever mp4 video you want to use for this task. In fact, for this re-implementation we use some other videos, which also work well.

2. Generate Synthetic Motion Blur

Now, we use optical flow to synthetically add motion blur. We used the most simple OpticalFlow method, wich provides reasonable results (we average frames anyway):

cd synthblur
mkdir build  && cd build
cmake ..
make all

To convert a video input.mp4 into a blurry version, run

./synthblur/build/convert "input.mp4"

This gives you multiple outputs:

  • 'input.mp4_blurry.mp4'
  • 'input.mp4_sharp.mp4'
  • 'input.mp4_flow.mp4'

Adding blur from synthetic camera shake is done on-the-fly (see psf.py).

3. Building a Database

For performance reasons we randomly sample frames from all videos beforehand and store 5+5 consecutive frames (sharp+blurry) into an LMDB file (for training/validation/testing).

I use

#!/bin/bash
for i in `seq 1 30`; do
    python data_sampler.py --pattern '/graphics/scratch/wieschol/YouTubeDataset/train/*_blurry.mp4' --lmdb /graphics/scratch/wieschol/YouTubeDataset/train$i.lmdb --num 5000
done

for i in `seq 1 10`; do
    python data_sampler.py --pattern '/graphics/scratch/wieschol/YouTubeDataset/val/*_blurry.mp4' --lmdb /graphics/scratch/wieschol/YouTubeDataset/val$i.lmdb --num 5000
done

To visualize the training examples just run

python data_provider.py --lmdb /graphics/scratch/wieschol/YouTubeDataset/train1.lmdb --show --num 5000

Training

This re-implementation uses TensorPack instead of the used custom library for the paper. Starting training is done by

python learning_blind_motion_deblurring.py --gpu 0,1 --data path/to/lmdb-files/

Results

See the release section for full-resolution images produced by our approach.

Further experiments

We further tried a convLSTM/convGRU and a multi-scale approach (instead of the simple test from the paper). These script are available in additional_scripts.

Notes

I re-trained a slightly larger model in TensorPack just for testing the TensorPack library some months ago. It seems to have similar performance (although it is not compatible with this GitHub project). Find the inference code/weights here.

Please note, TensorFlow introduce some changes over time. This setup is tested under

  • Python 2.7
  • Tensorflow-gpu v1.9.0
  • Cuda 9.0
  • Tensorpack 0.1.6 (from 17 Feb 2017)

More Repositories

1

NeRD-Neural-Reflectance-Decomposition

NeRD: Neural Reflectance Decomposition from Image Collections - ICCV 2021
Python
246
star
2

ggnn

GGNN: State of the Art Graph-based GPU Nearest Neighbor Search
Cuda
141
star
3

Flex-Convolution

Source code for: Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds), accepted at ACCV 2018
C++
115
star
4

SIGNeRF

SIGNeRF: Scene Integrated Generation for Neural Radiance Fields
Python
112
star
5

Neural-PIL

Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition - NeurIPS2021
Python
99
star
6

Product-Quantization-Tree

GPU-based large scale Approx. Nearest Neighbor Search, accepted at CVPR 2016
Cuda
93
star
7

pointcloud-viewer

Efficient Large-Scale Point-Cloud Viewer based on OpenGL
C++
79
star
8

will-people-like-your-image

Image Aesthetics Estimation (WACV18)
Python
61
star
9

SpatialDETR

Official implementation of SpatialDETR. The paper will be presented at ECCV 2022
Python
55
star
10

tf_custom_op

Boilerplate template for adding custom operations to TensorFlow
C++
14
star
11

emca

EMCA: Explorer of Monte Carlo based Algorithms
Python
12
star
12

low-poly-painter

Python
11
star
13

LearningToSynchronizeVideos

Implementation of our Video Synchronization paper in TensorFlow (ICMLA 2017)
C
7
star
14

infomark-ui

Front-end for the Infomark online platform
Elm
6
star
15

MedicalAnnotationFramework

Python
4
star
16

shinobi

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild
TypeScript
3
star
17

DualQueryMIL

Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification
Python
3
star
18

hyperrealistic_indoor_streetview

🗺️🏠 Software Project SS24 | Hyperrealistic Indoor Street-View
TypeScript
3
star
19

infomark-backend

scalable, modern and open-source online course management system with auto testing of programming assignments (RESTful API Server)
Go
2
star
20

praktikum-ws16-funfair

praktikum-ws16-funfair created by GitHub Classroom
JavaScript
2
star
21

InfoMark-deprecated

A online platform for distributing exercise sheets and testing exercise submission within Docker
Ruby
2
star
22

mitsuba-emca

Integration of EMCA https://github.com/cgtuebingen/emca into Mitsuba
C++
2
star
23

jannik-hofmann-master-thesis

DNN Visualization in the Unreal Engine for Interactive Fly-through Exploration, an extensible framework for visualizing feed-forward TensorFlow/Keras DNNs. It calculates a force-based layout and displays kernels, kernel activations, saliency maps, and integrated gradients. Published under GNU GPLv3.
C
2
star
24

fastsync

C++
1
star
25

hybrid_zoom

This is the simulation of the hybrid zoom based on the SLM-camera.
Python
1
star
26

snapshot_multispectral_imaging

MATLAB
1
star
27

spatial_xr

🥽🌐 Software Project SS24 | SpatialXR
C#
1
star