LRNavin/big_data LRNavin/big_data

  • Stars
    star
    1
  • Language
    Scala
  • Created about 5 years ago
  • Updated almost 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
LRNavin/big_data
github

LRNavin

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

In this lab we will put the concepts that are central to Supercomputing with Big Data in some practical context. We will analyze a large open data set and identify a way of processing it efficiently using Apache Spark and the Ama- zon Web Services (AWS). The data set in question is the GDELT 2.0 Global Knowledge Graph (GKG), which indexes persons, organizations, companies, locations, themes, and even emotions from live news reports in print, broad- cast and internet sources all over the world. We will use this data to construct a histogram of the topics that are most popular on a given day, hopefully giving us some interesting insights into the most important themes in recent history.

More Repositories

1

AutoComments

Description: We want to create a deep Neural Network that can automatically generate comments for code snippets passed to it. The motivation behind this is that in software development and maintenance, developers spend around 59% of their time on program comprehension activities. Having comments that are generated automatically will hopefully cut this time down. In order to do this we will combine the recent paper Code2Vec: Learning Distributed Representations of Code by Alon et al. with the paper Deep Code Comment Generation in order to make a better performing model using the newer Code2Vec encoding that was not used in the Deep Code Comment Generation paper. Dataset: The dataset that we will use is the same dataset used by the Deep Code Comment Generation paper, this is a dataset of more than 500,000 code snippets including comments. This also gives us a baseline against which to compare. Papers: Deep Code: https://xin-xia.github.io/publication/icpc182.pdf Code2Vec: https://arxiv.org/abs/1803.09473
Python
43
star
2

conversation_quality

Jupyter Notebook
2
star
3

deeplearning_project

Quarter-3, Deep Learning Project
Jupyter Notebook
2
star
4

cyberdata_analytics

Jupyter Notebook
2
star
5

painters_by_numbers

In this project we have evaluated approaches to the challenge proposed on Kaggle named {Painter by Numbers}. The objective of this challenge is to distinguish whether two paintings were created by the same artist in the process of pairwise comparison. In a broad sense this could improve the identification of forgeries based on learned {artist style}. The dataset for the challenge is a collection of paintings from WikiArt.org {http://wikiart.org}. After analyzing approaches taken by other competitors, we have identified a gap that could be explored: creating a network that not only learns the style from artists included in the provided dataset, but also is able to give a correct verdict for an artist that are not included in the training data. This creates an additional layer of complexity as the network has to facilitate ability to {generalize to unseen artists}. To tackle the problem at hand, a neural network architecture called the siamese network was used.
Jupyter Notebook
2
star