Deep Learning Study

Study of HeXA at Ulsan National Institute of Science and Technology.

Implementations

• DCGAN-tensorflow : Deep Convolutional GAN implementation in Tensorflow. [code] [demo]
• DQN-tensorflow :: Human-Level Control through Deep Reinforcement Learning implementation in Tensorflow. [code]
• MemN2N-tensorflow : End-To-End Memory Network implementation in Tensorflow. [code]
• NTM-tensorflow : Neural Turing Machine implementation in Tensorflow. [code]
• lstm-char-cnn-tensorflow : Character-Aware Neural Language Models implementation in Tensorflow. [code]
• visual-analogy-tensorflow : Deep Visual Analogy-Making implementation in Tensorflow. [code]
• variational-text-tensorflow : Neural Variational Inference for Text Processing in Tensorflow. [code]
• text-based-game-rl-tensorflow : Language Understanding for Text-based Games using Deep Reinforcement Learning implementation in Tensorflow. [code]
• neural-summary-tensorflow : Attention-Based Summarization implementation in TensorFlow. [code] (in progress)
• attentive-reader-tensorflow : Teaching Machines to Read and Comprehend implementation in TensorFlow. [code] (in progress)

Reasoning

Deep Reasoning presentation (3/17)

• [E2E MN] End-To-End Memory Networks [paper]
  • Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus
• [E2E MN+] The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations [paper]
  • Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston
• [DMN] Ask Me Anything: Dynamic Memory Networks for Natural Language Processing [paper]
  • Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, Richard Socher
• [ReasoningNet] Towards Neural Network-based Reasoning [paper]
  • Baolin Peng, Zhengdong Lu, Hang Li, Kam-Fai Wong
• [Impatient] Teaching Machines to Read and Comprehend [paper]
  • Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, Phil Blunsom
• [Variational] Neural Variational Inference for Text Processing [paper]
  • Yishu Miao, Lei Yu, Phil Blunsom
• [Attentive Pooling] Attentive Pooling Networks [paper]
  • Cicero dos Santos, Ming Tan, Bing Xiang, Bowen Zhou
• [Attention Sum] Text Understanding with the Attention Sum Reader Network [paper]
  • Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, Jan Kleindienst
• [ABCNN] ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs [paper]
  • Wenpeng Yin, Hinrich Schütze, Bing Xiang, Bowen Zhou
• [NTM] Empirical Study on Deep Learning Models for Question Answering [paper]
  • Yang Yu, Wei Zhang, Chung-Wei Hang, Bing Xiang, Bowen Zhou
• [Dynamic] Learning to Compose Neural Networks for Question Answering [paper]
  • Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein

Variational

• Auto-Encoding Variational Bayes [paper]
  • Diederik P Kingma, Max Welling
• Generating Sentences from a Continuous Space [paper]
  • Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, Samy Bengio
• Neural Variational Inference for Text Processing [paper]
  • Yishu Miao, Lei Yu, Phil Blunsom
• Learning Structured Output Representation using Deep Conditional Generative Models [paper]
  • Kihyuk Sohn, Honglak Lee, Xinchen Yan

Learning Algorithm

• Neural GPUs Learn Algorithms [paper]
  • Łukasz Kaiser, Ilya Sutskever
• Learning Efficient Algorithms with Hierarchical Attentive Memory [paper]
  • Marcin Andrychowicz, Karol Kurach
• Neural Random-Access Machines [paper]
  • Karol Kurach, Marcin Andrychowicz, Ilya Sutskever
• Neural Programmer: Inducing Latent Programs with Gradient Descent [paper]
  • Arvind Neelakantan, Quoc V. Le, Ilya Sutskever
• Neural Programmer-Interpreters [paper]
  • Scott Reed, Nando de Freitas
• Reinforcement Learning Neural Turing Machines [paper]
  • Wojciech Zaremba, Ilya Sutskever
• Learning Simple Algorithms from Examples [paper]
  • Wojciech Zaremba, Tomas Mikolov, Armand Joulin, Rob Fergus

Generative

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [paper]
  • Alec Radford, Luke Metz, Soumith Chintala
• Deep Visual Analogy-Making [paper]
  • Scott Reed, Yi Zhang, Yuting Zhang, Honglak Lee
• How to Train Deep Variational Autoencoders and Probabilistic Ladder Networks [paper]
  • Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther

Natural Language Processing

• Exploring the Limits of Language Modeling [paper]
  • Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, Yonghui Wu
• Swivel: Improving Embeddings by Noticing What's Missing [paper]
  • Noam Shazeer, Ryan Doherty, Colin Evans, Chris Waterson

Reinforcement Learning

• Lie Access Neural Turing Machine [paper]
  • Greg Yang
• Asynchronous Methods for Deep Reinforcement Learning [paper]
  • Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu

ETC

• Learning Physical Intuition of Block Towers by Example [paper]
  • Adam Lerer, Sam Gross, Rob Fergus

Links

• papernote