• Stars
    star
    123
  • Rank 290,145 (Top 6 %)
  • Language
    Python
  • Created over 6 years ago
  • Updated almost 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Tensorflow implementation of Justin Johnson's paper, perceptual losses for real-time style transfer and super-resolution.

Real-Time Style Transfer in TensorFlow

This repository is Tensorflow implementation of Johnson's Perceptual Losses for Real-Time Style Transfer and Super-Resolution.

It takes 385 ms on a GTX1080Ti to style the MIT Stata Center (1024x680).

Requirements

  • tensorflow 1.8.0
  • python 3.5.3
  • numpy 1.14.2
  • scipy 0.19.0
  • moviepy 0.2.3.2

Video Stylization

Here we transformed every frame in a video using various stylle images, then combined the results. Click to go to the full demo on YouTube!

Image Stylization

A photo of Chicago was applied for various style paintings. Click on the ./examples/style folder to see full applied style images. For more results you can find here.

Implementation Details

Implementation uses TensorFlow to train a real-time style transfer network. Same transformation network is used as described in Johnson, except that batch normalization is replaced with Ulyanov's instance normalization, zero padding is replaced by reflected padding to reduce boundary artifacts, and the scaling/offset of the output tanh layer is slightly different.

We follow Logan Engstrom to use a loss function close to the one described in Gatys, using VGG19 instead of VGG16 and typically using "shallower" layers than in Johson's implementation (e.g. relu1_1 is used rather than relu1_2).

Documentation

Training Style Transfer Networks

Use main.py to train a new style transform network. Training takes 6-8 hours on a GTX 1080Ti. Before you run this, you should run setup.sh. Example usage:

python main.py --style_img path/to/style/img.jpg \
  --train_path path/to/trainng/data/fold \
  --test_path path/to/test/data/fold \
  --vgg_path path/to/vgg19/imagenet-vgg-verydeep-19.mat
  • --gpu_index: gpu index, default: 0
  • --checkpoint_dir: dir to save checkpoint in, default: ./checkpoints
  • --style_img: style image path, default: ./examples/style/la_muse.jpg
  • --train_path: path to trainng images folder, default: ../Data/coco/img/train2014
  • --test_path: test image path, default: ./examples/content
  • --test_dir: test oa,ge save dor. default: ./examples/temp
  • --epochs: number of epochs for training data, default: 2
  • --batch_size: batch size for single feed forward, default: 4
  • --vgg_path: path to VGG19 network, default: ../Models_zoo/imagenet-vgg-verydeep-19.mat
  • --content_weight: content weight, default: 7.5
  • --style_weight: style weight, default: 100.
  • --tv_weight: total variation regularization weight, default: 200.
  • --print_freq: print loss frequency, default: 100
  • --sample_freq: sample frequency, default: 2000

Evaluating Style Transfer Networks

Use evaluate.py to evaluate a style transfer network. Evaluation takes 300 ms per frame on a GTX 1080Ti. Takes several seconds per frame on a CPU. Models for evaluation are located here. Example usage:

python evaluate.py --checkpoint_dir path/to/checkpoint /
  --in_path path/to/test/image/folder
  • --gpu_index: gpu index, default: 0
  • --checkpoint_dir: dir to read checkpoint in, default: ./checkpoints/la_muse
  • --in_path: test image path, default: ./examples/test
  • --out_path: destination dir of transformed files, default: ./examples/results

Stylizing Video

Use transform_video.py to transfer style into a video. Requires moviepy. Example usage:

python transform_video.py --checkpoint_dir path/to/checkpoint /
  --in_path path/to/input/video.mp4 /
  --out_path path/to/write/predicted_video.mp4
  • --gpu_index: gpu index, default: 0
  • --checkpoint_dir: dir to read checkpoint in, default: ./checkpoints/la_muse
  • --in_path: input video path, default: None
  • --out_path: path to save processed video to, default: None

Citation

  @misc{chengbinjin2018realtimestyletransfer,
    author = {Cheng-Bin Jin},
    title = {Real-Time Style Transfer},
    year = {2018},
    howpublished = {\url{https://github.com/ChengBinJin/Real-time-style-transfer/}},
    note = {commit xxxxxxx}
  }

Attributions/Thanks

License

Copyright (c) 2018 Cheng-Bin Jin. Contact me for commercial use (or rather any use that is not academic research) (email: [email protected]). Free for research use, as long as proper attribution is given and this copyright notice is retained.

More Repositories

1

V-GAN-tensorflow

A tensorflow implementation of "Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks"
Python
232
star
2

MRI-to-CT-DCNN-TensorFlow

This repository is the implementations of the paper "MR-based Synthetic CT Generation using Deep Convolutional Neural Network Method," Medical Physics 2017.
Python
101
star
3

semantic-image-inpainting

Tensorflow implementation of Semantic Image Inpainting with Deep Generative Models
Python
78
star
4

WGAN-GP-tensorflow

WGAN-GP tensorflow implementation
Python
72
star
5

WGAN-TensorFlow

Wasserstein GAN TensorFlow Implementation
Python
44
star
6

SpineC2M

This repository includes Spine data based CT to MR image synthesis
Python
40
star
7

Fast-Tracking-by-Detection

This repository includes fast tracking by detection method based on SSD detector and multiple Kalman tracker.
Python
32
star
8

MRGAN-TensorFlow

This repository is the implementation of the CT to MR synthesis using TensorFlow
Python
29
star
9

ActionViewer

ICVL Action Dataset Viewer
Python
26
star
10

VAE-Tensorflow

Variational Autoencoder Tensorflow Implementation
Python
24
star
11

License_plate_recognition

This repository supplies some functions for License Plate Recognition (LPR) project.
Python
15
star
12

VanillaGAN-TensorFlow

Vanilla GAN TensorFlow Implementation
Python
14
star
13

pix2pix-tensorflow

pix2pix TensorFlow Implementation
Python
13
star
14

U-Net-TensorFlow

TensorFlow implementation of the U-Net.
Python
12
star
15

PytorchZeroToAll

It's the pytorch practice following Sung Kim's Tutorial
Python
11
star
16

DCGAN-TensorFlow

DCGAN TensorFlow Implementation
Python
11
star
17

GANFromAtoZ

This repository reimplements all kinds of published GAN
Python
11
star
18

LaTex-beginner-lecture

This repository follows [권현우](https://www.youtube.com/watch?v=V1Q6vEuoAQ0&list=PLSS68lwkeqyOH6KEHpCAmCWVSSKbciz3A) LaTex Tutorial.
TeX
11
star
19

Latex_Michelle

LaTeX tutorial using Texmaker. This repository follows [Michelle Krummel's Tutorial](https://www.youtube.com/watch?v=SoDv0qhyysQ&list=PL1D4EAB31D3EBC449&index=1).
TeX
10
star
20

LatexBible

Latex Bible Book Exercise
TeX
10
star
21

Tactile-Sensor

Tactile Sensor Project
Python
10
star
22

deep-neural-network-for-visual-recognition

This repository includes lecture materials of the class, Deep Neural Network for Visual Recognition.
10
star
23

Latex-Resume-Tutorial

This repository follows Luke Smith's Latex Resume Tutorial.
TeX
9
star
24

Python-Exercise

Python library exercise
Jupyter Notebook
9
star
25

DiscoGAN-TensorFlow

DiscoGAN TensorFlow Implementation
Jupyter Notebook
9
star
26

pixelrnn-tensorflow

PixelRNN Tensorflow Implementation
Python
9
star
27

Papers-Category

It's the repository for collecting papers that I read and category them according to the different objectives.
9
star
28

IrisGAN-TensorFlow

TensorFlow Implementation of Iris Synthesis using GAN
Python
8
star
29

Tensorflow-SungKim

This repository is tensorflow tutorial from Sung Kim's 모두를 위한 딥러닝.
Python
8
star
30

TensorFlow-Exercise

This repository includes some test codes for tf_record, tf.train, tf.data and so on
Python
8
star
31

GitExample

Git 연습용 레포지토리
Python
8
star
32

Face-Editing-Collection

This repository is a collection of the face editing papers and projects.
7
star
33

Pydicom-exercise

This repository include exercise codes using pydicom
Python
7
star
34

d-Fingerprint-Generation

Deterministic fingerprint generation
Python
7
star
35

Semantic-Eye-Segmentation

TensorFlow Implementation of the Synthetic Eye Generation
Python
7
star
36

Hvass-Laboratory-Tensorflow

This repository follows Hvass Laboratory's Tensorflow Tutorial.
Python
7
star
37

Adam-Analysis-TensorFlow

This repository analyzes the performance of Adam optimizer while comparing with others.
Python
6
star
38

Semi-supervised-semantic-segmentation

Semi-supervised Semantic Eye Segmentation in TensorFlow
Python
6
star
39

TensorFlow2.0-Tutorial

This repository includes the example codes using TensorFlow2.0
Python
6
star
40

Python-Libraries

collected various API functions
Python
5
star
41

image-animation

Implementations of MonkeyNet, FOMM and MRAA
Python
5
star
42

Synthetic-Eye-Generation

TensorFlow Implementation for the Synthetic-Eye-Generation
Python
5
star
43

TensorFlow-Tutorials-Weihai

TensorFlow Tutorials for Shandon University (Weihai)
Python
5
star
44

Support-Vector-Machine-Python

This repository is a study to deeply understand the theory of SVM.
Jupyter Notebook
4
star
45

leetcode

Solutions for Leetcode
Jupyter Notebook
4
star
46

BrainC2MusingDC2Anet

Brain CT to MR Synthesis using DC2Anet
Python
4
star
47

Lua-Basic-Tutorial

This repository for the Lua basic tutorial following John Sonmez
Lua
3
star
48

ChengBinJin

2
star
49

DirectX-Tutorial

The repository includes C++ DirectX Tutorials
C++
2
star
50

StyleGANv2-Re-Implementation

This project is a re-implementation of the StyleGANv2 to study the coding style.
Python
2
star
51

AttGAN-TensorFlow

This repository is for tensorflow implementation of the AttGAN
Python
2
star
52

Python-Toolbox

Python toolboxes that inludes basic python operations, opencv, tensorflow and pytorch and so on
Python
2
star