irfanICMLL/Auto_painter

Stars
132
Rank 274,205 (Top 6 %)
Language
Python
Created about 7 years ago
Updated about 4 years ago

irfanICMLL/Auto_painter

irfanICMLL

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Recently, realistic image generation using deep neural networks has become a hot topic in machine learning and computer vision. Such an image can be generated at pixel level by learning from a large collection of images. Learning to generate colorful cartoon images from black-and-white sketches is not only an interesting research problem, but also a useful application in digital entertainment. In this paper, we investigate the sketch-to-image synthesis problem by using conditional generative adversarial networks (cGAN). We propose a model called auto-painter which can automatically generate compatible colors given a sketch. Wasserstein distance is used in training cGAN to overcome model collapse and enable the model converged much better. The new model is not only capable of painting hand-draw sketch with compatible colors, but also allowing users to indicate preferred colors. Experimental results on different sketch datasets show that the auto-painter performs better than other existing image-to-image methods.

Auto_painter

News

We have released our dataset for public use. The dataset can be downloaded through following links:

Sketch-image pairs: https://cloudstor.aarnet.edu.au/plus/s/rMSBYCjEZJ70ab2

Sketch with control color blocks: https://cloudstor.aarnet.edu.au/plus/s/ixj8XS0rMmUqq0Z

Orginal README

It is the original implementation of the journal article: Auto-painter: Cartoon image generation from sketch by using conditional Wasserstein generative adversarial networks https://www.sciencedirect.com/science/article/pii/S0925231218306209?via%3Dihub

This project mean to make an end-to-end network for the sketch of cartoon to have color automatically.

Try our demo here: http://103.202.133.77:10086/

Since the lab's server has temporarily expired, the demo is now unavailable. You can see the demo video and train your own model. Or you can build your demo page based on our provided models following this project: https://github.com/irfanICMLL/Auto_painter_demo

New model has been updated!~ The performance is much better than in the orginal paper! See the demo video:https://youtu.be/g9rf-YFGgbg

Have a try~

The pre-trained model can be downloaded from the following link: https://cloudstor.aarnet.edu.au/plus/s/LvyREKsiaH47Aa6

My homepage: https://irfanicmll.github.io/

Welcome to contact me~

Dependencies

python3.5

tensorflow1.4

Vgg model from:https://github.com/machrisaa/tensorflow-vgg(optional, if you use the loss_f)

Data

Color images: Collected on the Internet

Sketch: Generated from the preprocessing/gen_sketch/sketch.py

Quick start

Put you orginal data in the folder preprocessing/gen_sketch/pic_org

Run the sketch.py and you will get the training set in the preprocessing/gen_sketch/pic_sketch folder

Download the pre-train weight of Vgg16, and put the model and the pretrian weight uder the folder of training&test/my_vgg

Run the training command as:

python auto-painter.py --mode train --input_dir $TRAINING_SET --output_dir $OUTPUT --checkpoint None

Run the testing command as:

python auto-painter.py --mode test --input_dir $TESTING_SET --output_dir $OUTPUT_TEST --checkpoint $OUTPUT

structure_knowledge_distillation

The official code for the paper 'Structured Knowledge Distillation for Semantic Segmentation'. (CVPR 2019 ORAL) and extension to other tasks.

CoupleGenerator

Generate your lover with your photo

ETC-Real-time-Per-frame-Semantic-video-segmentation

Enforcing temporal consistency in real-time per-frame semantic video segmentation

TorchDistiller

EMM-for-stock-prediction

We propose a model to analyze sentiment of online stock forum and use the information to predict stock volatility in the Chinese market. By generating a sentimental dictionary, we analyze the sentimental tendencies of each post as sentiment indicators. Such sentimental information will be fused with market data for prediction based on Recurrent Neural Networks (RNNs). We manually labeled the sentiment of forum post and make the data public available for research. Empirical evidence shows that 8 of the 10 stocks perform better with sentimental indicators.

Auto_painter_demo

The code of building a web demo for Auto_painter

SSIW

The code of 'The devil is in the labels: Semantic segmentation from sentences'.

inceptionV2_finetune

Fine-tuning of inceptionV2 on CUB-200 Birds dataset in Tensorflow

stock_predict

This project predicts stock trends on the basis of online user comments and LSTM

colorization

horseSeg

dvn

dvn for semantic segmentation