akanimax/natural-language-summary-generation-from-structured-data akanimax/natural-language-summary-generation-from-structured-data

  • Stars
    star
    183
  • Rank 208,826 (Top 5 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 7 years ago
  • Updated over 5 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
akanimax/natural-language-summary-generation-from-structured-data
github

akanimax

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Implementation of the paper -> https://arxiv.org/abs/1709.00155. For converting information present in the form of structured data into natural language text

Natural-Language-Summary-Generation-From-Structured-Data

Implementation (Personal) of the paper titled "Order-Planning Neural Text Generation From Structured Data". The dataset for this project can be found at -> WikiBio

Requirements for training:

  • python 3+
  • tensorflow-gpu (preferable; CPU will take forever)
  • Host Memory 12GB+ (this will be addressed soon)

Architecture


Running the Code

Process of using this code is slightly involved presently. This will be addressed in further development (perhaps with collaboration).

1. Preprocessing:

Please refer to the /TensorFlow_implementation/Data_Preprocessor.ipynb for info about what steps are performed in preprocessing the data. Using the notebook on the full data for preprocessing will be very slow, so please use the following procedure for it.

Step 1: 
(your_venv)$ python fast_data_preprocessor_part1.py

Note that all the tweakable parameters are declared at the beginning of the script (Change them as per your requirement). This will generate a temp.pickle file in the same directory. Do not delete it even after full preprocessing. This is like a backup of the preprocessing pipeline; i.e. if you decide to change something later, you would'nt have to run the entire preprocessing again.

Step 2:
(your_venv)$ python fast_data_preprocessor_part12.py

This will create the following file: /Data/plug_and_play.pickle. Again, tweakable parameters are at the beginning of the script. Please Note that this process requires RAM 12GB+. If you have < 12GB Host memory, please use a subset of the dataset instead of the entire dataset (change data_limit in the script).

2. Training:

Once preprocessing is done, simply run one of the two training Scripts.

(your_venv)$ python trainer_with_copy_net.py
OR
(your_venv)$ python trainer_without_copy_net.py

Again all the hyperparameters are present at the beginning of the script. Example trainer_without_copy_net.py:

''' Name of the model:  '''
# This can be changed to create new models in the directory
model_name = "Model_1(without_copy_net)"

'''
    ========================================================
    || All Tweakable hyper-parameters
    ========================================================
'''
# constants for this script
no_of_epochs = 500
train_percentage = 100
batch_size = 8
checkpoint_factor = 100
learning_rate = 3e-4 # for learning rate 
# but I have noticed that this learning rate works quite well.
momentum = 0.9

# Memory usage fraction:
gpu_memory_usage_fraction = 1

# Embeddings size:
field_embedding_size = 100
content_label_embedding_size = 400 # This is a much bigger 
# vocabulary compared to the field_name's vocabulary

# LSTM hidden state sizes
lstm_cell_state_size = hidden_state_size = 500 # they are 
# same (for now)
'''
    ========================================================
'''

Test Runs:

Once training is started, log-dirs are created for Tensorboard. Start your tensorboard server pointing to the log-dir.

Loss monitor:


Embedding projector:


  • Trained models coming soon ...

Thanks

Please feel free to open PRs (contribute)/ issues / comments (feedback) here.

Best regards,
@akanimax :)

More Repositories

1

BMSG-GAN

[MSG-GAN] Any body can GAN! Highly stable and robust architecture. Requires little to no hyperparameter tuning. Pytorch Implementation
Python
630
star
2

T2F

T2F: text to face generation using Deep Learning
Python
546
star
3

pro_gan_pytorch

Unofficial PyTorch implementation of the paper titled "Progressive growing of GANs for improved Quality, Stability, and Variation"
Python
536
star
4

msg-stylegan-tf

MSG StyleGAN in tensorflow
Python
264
star
5

Variational_Discriminator_Bottleneck

Implementation (with some experimentation) of the paper titled "VARIATIONAL DISCRIMINATOR BOTTLENECK: IMPROVING IMITATION LEARNING, INVERSE RL, AND GANS BY CONSTRAINING INFORMATION FLOW" (arxiv -> https://arxiv.org/pdf/1810.00821.pdf)
Python
152
star
6

msg-gan-v1

MSG-GAN: Multi-Scale Gradients GAN (Architecture inspired from ProGAN but doesn't use layer-wise growing)
Python
151
star
7

fagan

A variant of the Self Attention GAN named: FAGAN (Full Attention GAN)
Python
112
star
8

thr3ed_atom

ReLU Fields The Little Non-linearity That Could
Python
111
star
9

big-discriminator-batch-spoofing-gan

BMSG-GAN with more features
Python
45
star
10

pro_gan_pytorch-examples

Examples trained using the python pytorch package pro-gan-pth
Python
38
star
11

attn_gan_pytorch

python package for self-attention gan implemented as extension of PyTorch nn.Module. paper -> https://arxiv.org/abs/1805.08318
Python
19
star
12

my-deity

I worship the one true neural network architecture that can autonomously learn everything.
Jupyter Notebook
12
star
13

NLP2SQL

A research and review of techniques to provide a natural language interface to RDMS.
Jupyter Notebook
11
star
14

open-styleganv2-pytorch

Open source + Free for Commercial Use implementation of StyleGANv2 in pytorch
Python
7
star
15

capsule-network-TensorFlow

The impending concept of capsule networks has finally arrived at arXiv. link to the publication -> https://arxiv.org/abs/1710.09829 . In this repository, I'll create an implementation using TensorFlow from scratch as an exercise.
Jupyter Notebook
6
star
16

3inGAN

Python
5
star
17

GAN-understanding

Implements gans on toy datasets and preliminary ML datasets for showing certain aspects of convergence and stability. Tries to cover various loss functions defined over the years.
Jupyter Notebook
5
star
18

autoencoder-cifar-10

Implementing an auto-encoder for the cifar10 dataset
Jupyter Notebook
4
star
19

Homecoming

repository for mini-projects
Python
3
star
20

python_ai_project_template

A lightweight template for building AI-based prototype/research POCs in Python. My poison (DL framework :laugh: ) of choice is PyTorch!
Python
3
star
21

some-randon-gan-1

MSG-GAN with self attention. For MSG-GAN head to -> https://github.com/akanimax/MSG-GAN
Python
2
star
22

AI-Literature

A repository to store key research works from the past. It is also an attempt to structure and organize these research papers.
2
star
23

indian-celeb-gans

Various GANs trained on a dataset containing images of Indian Celebrities (procured by me).
Python
2
star
24

some-random-gan-2

More experimentation with the base MSG-GAN architecture. This includes the coord-conv layers in the architecture. For more info about MSG-GAN, head to -> https://github.com/akanimax/msg-stylegan-tf
Python
2
star
25

multithreaded-histogram-equalization-cpp

Explanatory Code for performing Histogram Equalization on Images for contrast improvement. The code uses OpenCV in C++ for image read/write and uses pthread for multithreading
C++
2
star
26

deep-reinforcement-learning

Project for studying and implementing the traditional RL algorithms and also the DL variants of the same.
Jupyter Notebook
1
star
27

dcgan_pytorch

GAN example created using the attn_gan_pytorch package -> https://github.com/akanimax/attn_gan_pytorch
Python
1
star
28

CL-3_lab_2017

repository for assignments of Computer Laboratory 3 - 2016
TeX
1
star
29

SVC2004-deep-learning

A deep learning based solution for the SVC2004 problem.
Jupyter Notebook
1
star
30

toxic-comment-identification-tensorflow

Data -> https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data
Python
1
star
31

REST_MOVIE_TICKET_SYSTEM

A restful system implementing token based authentication for allowing users to book movie tickets online. Use of Play framework for scala
Scala
1
star
32

algorithms

A repository for collecting the coding implementations of some of the most famous algorithms
Python
1
star
33

energy-preserving-neural-network

When a data signal propagates through the Neural Network, it is not mandatory that the energy of the signal will be preserved throughout the neural computations. This research attempts at collecting (perhaps creating) techniques for preserving the Energy throughout the network.
Python
1
star