Discover furqanshahid85-python/Corona-Dataset-Analysis-Using-PySpark

Stars
1
Language
Python
Created over 4 years ago
Updated over 4 years ago

furqanshahid85-python

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

This module performs statistical analysis on the noval corona virus dataset. The dataset being used was last updated on May 02, 2020. The Module performs the following Functions: * Displays the statistics of input dataset * Reads data from csv files and stores the aggregated output in parquet format * Counts the Number of records for each country/region and provice/state * Lists max Cases for each country/region and provice/state * Lists max Deaths for each country/region and provice/state * List max Recoveries for each country/region and provice/state *

AWS-Glue-Pyspark-ETL-Job

Python

Python-FTP-File-Ingestion

This module provides the functionality of uploading files to s3 from a FTP server. An SFTP connection is created with the FTP server and all the files present in the specified directory are uploaded to the specified s3 bucket. Following are the key features of this module: Creates a secure ssh connection with FTP server. Handles multipart upload to s3 automatically, if file size is greater than 100MB (can be configured). Automatically handles retires in case of failed uploads during multipart upload. Partitions the data in s3 based on current year,month,day,hour. Ensures which file has been processed or needs to be processed.

Python

Python-Multi_Threading-Vs-Multi_Processing

Python

aws_lambda_img_thumbail_processing

Aws lambda function that creates an img thumbnail of an image uploaded to S3.

Python

python-monerepo

A python project repo in a monorepo codebase architecture