clovaai/dmfont

Few-shot Compositional Font Generation with Dual Memory (ECCV'20)

NOTICE: We release the unified few-shot font generation repository (clovaai/fewshot-font-generation). If you are interested in using our implementation, please visit the unified repository.

Official PyTorch implementation of DM-Font | Paper | Video on ECCV 2020

Junbum Cha, Sanghyuk Chun, Gayoung Lee, Bado Lee, Seonghyeon Kim, Hwalsuk Lee.
Clova AI Research, NAVER Corp.
In ECCV 2020.

Abstract

Generating a new font library is a very labor-intensive and time-consuming job for glyph-rich scripts. Despite the remarkable success of existing font generation methods, they have significant drawbacks; they require a large number of reference images to generate a new font set, or they fail to capture detailed styles with a few samples. In this paper, we focus on compositional scripts, a widely used letter system in the world, where each glyph can be decomposed by several components. By utilizing the compositionality of compositional scripts, we propose a novel font generation framework, named Dual Memory-augmented Font Generation Network (DM-Font), which enables us to generate a high-quality font library with only a few samples. We employ memory components and global-context awareness in the generator to take advantage of the compositionality. In the experiments on Korean-handwriting fonts and Thai-printing fonts, we observe that our method generates a significantly better quality of samples with faithful stylization compared to the state-of-the-art generation methods in quantitatively and qualitatively.

Other related repositories

You can find more related projects on the few-shot font generation at the following links:

• clovaai/dmfont (ECCV'20) | paper
• clovaai/lffont (AAAI'21) | paper
• clovaai/mxfont (ICCV'21) | paper
• clovaai/fewshot-font-generation The unified few-shot font generation repository

Installation

Install dependencies:

pip install -r requirements.txt

Note that using different version of required packages can effects the results, especially PyTorch. The implementations are tested on Python 3.6.

Dataset preparation

Data sources

Korean-handwriting and Thai-printing datasets were built from UhBee fonts and Thai font collection, respectively. To ensure the style diversity of the dataset, one font was selected for each font family in our experiments.

Dumping dataset

The scripts/prepare_dataset.py script renders glyphs from the collected ttf fonts and dumps them into hdf5 files. For the Thai-printing dataset, we rectify the rendering errors using raqm. It should be installed before making dataset.

• Korean-handwriting

python -m scripts.prepare_dataset kor $FONTSDIR meta/kor_split.json $DUMPDIR

• Thai-printing

python -m scripts.prepare_dataset thai $FONTSDIR meta/thai_split.json $DUMPDIR

How to run

Pretrained models

For convenience, the minimal size checkpoints are provided by excluding training variables, e.g., momentums in optimizer, discriminator, and non-EMA generator.

• Checkpoint trained on the Korean-handwriting dataset
• Checkpoint trained on the Thai-printing dataset

Training

• Korean-handwriting

python train.py $NAME cfgs/kor.yaml

• Thai-printing

python train.py $NAME cfgs/kor.yaml cfgs/thai.yaml

Generation & Pixel-level evaluation

• Korean-handwriting

python evaluator.py $NAME $CHECKPOINT_PATH $OUT_DIR cfgs/kor.yaml --mode cv-save

• Thai-printing

python evaluator.py $NAME $CHECKPOINT_PATH $OUT_DIR cfgs/kor.yaml cfgs/thai.yaml --mode cv-save

• Korean-unrefined

python evaluator.py $NAME $CHECKPOINT_PATH $OUT_DIR cfgs/kor.yaml --mode user-study-save

Citation

@inproceedings{cha2020dmfont,
    title={Few-shot Compositional Font Generation with Dual Memory},
    author={Cha, Junbum and Chun, Sanghyuk and Lee, Gayoung and Lee, Bado and Kim, Seonghyeon and Lee, Hwalsuk},
    year={2020},
    booktitle={European Conference on Computer Vision (ECCV)},
}

License

This project is distributed under MIT license, except modules.py which is adopted from https://github.com/NVlabs/FUNIT.

Copyright (c) 2020-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.