ELOUTMADI/Spark-Project-on-Cloudera-Hadoop-CDH-and-GCP

Stars
1
Language
Scala
Created about 2 years ago
Updated almost 2 years ago

ELOUTMADI/Spark-Project-on-Cloudera-Hadoop-CDH-and-GCP

ELOUTMADI

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Check out the pipeline architecture for better understanding

Dicom-Images-preprocessing

Ultrasound imaging has become one ol the most widely used medical imaging modalities in clinical practice, it is characterized by low image quality with great variability compared to other modalities. Researchers have thus applied computer vision and deep learning techniques to ultrasound DICOM images for classification, detection and segmentation. These techniques, despite offering Interesting performances are very hard to implement at a large scale because of the complexity of the data coming from different sources and with different distributions. Fortunately, the cloud is changing the way applications are designed, including how data is processed and stored before training and deploying deep learning models. In this project, we aim to design and build a DICOM data pipeline in Microsoft Azure that describes how data flows through a solution, wherê it is processed, where it is stored, and how il is consumed by the next component in the pipeline including an annotation platform, a training cluster or a deployment endpoint.

Jupyter Notebook

E2E-ML-Pipeline-With-Docker-Airflow

We will implement this pipeline using Apache Airflow, a popular open-source orchestrator that allows for programmatically building, scheduling, and monitoring workflows. We installed it via #dockercompose and configured it to run in our local environment.

chatbot-using-NLTK-Keras

We will create a retrieval based chatbot using NLTK, Keras, Python, etc. A retrieval-based chatbot uses predefined input patterns and responses. It then uses some type of heuristic approach to select the appropriate response. It is widely used in the industry to make goal-oriented chatbots where we can customize the tone and flow of the chatbot to drive our customers with the best experience.

Detection-of-Fake-News

This high level python task of identifying fakenews manages phony and genuine news. Utilizing sklearn, we fabricate a TfidfVectorizer on our dataset. Then, at that point, we introduce a PassiveAggressive Classifier and fit the model. Eventually, the accuracy score and the confusion matrix disclose to us how well our model is.

Jupyter Notebook

Kafka-Twitter-Producer-Elastic-Search-Consumer

Building a Kafka Twitter Producer where the data can be consumed by inserting it to ElasticSearch

Data-Modeling-With-PostgresSQL

ETL pipeline for populating the Sparkify database, which is designed to enable Sparkify to analyze data on its users, the songs they listen to, and the artists of those songs.

Sentiment_Analysis_Using_Twitter_API

This projet includes a pdf document that explains each taken step during the implementation phase

drowsiness-detection-of-the-driver

The majority of accidents happen because of the drowsiness of the driver. So, to forestall these accidents we will build a system using Python, OpenCV, and Keras which will caution the driver when he feels languid.

Image-Segmentation-

lets implement already well known architecture, UNet

Real-Time-Message-Processing-Application

Check out the pipeline architecture for better understanding

Integrate-mlflow-with-kubeflow

You've been provided with the helm file that would help you to configure MLflow with Kubeflow , this file can be adjusted according to your needs

NLP-Spam-Detection

A simple Flask API to detect spam or ham using Python and sklearn

Analysis-and-Visualization-of-Covid-19-Data

Analysis-and-Visualization-of-Covid-19-Data

Mlops-End-To-End-Machine-Learning-Pipeline-CI-CD

The main objective of this project is to automate the whole machine learning app deployment process. To implement this project we will be using TensorFlow and basic knowledge in dockers and Kubernetes , cloudbuild (GCP)

Jupyter Notebook

Psychological-Platform

The project consists of providing answers to the various subscribers of the platform. We have three actors, the user, the HR and the psychologist

Number-Plate-Recognition

Automatic license plate recognition systems play a major role in a number of safety-related applications. They are used to identify given vehicles quickly and reliably, without disrupting the trac. Applications are far from being limited to fight against crime: license plate recognition systems find their place wherever access is restricted to a type or category of vehicle

Jupyter Notebook

attrition-of-bank-customers

Identify and visualize which factors contribute to the attrition of clients. Classify if a customer is going to quit or not , ideally and in view of model execution, pick a model that will connect a likelihood to the stir to make it simpler for client support to target low draping organic products in their endeavors to forestall quitting

Jupyter Notebook

Scrapping-avito-website

We did the scrapping using the beautiful soup library on the avito.com website. We got a lot of information about its uses.

classification-of-traffic-sign

we will build a neural network model that can classify traffic signs of different images. Thanks to this model, we will be able to understand traffic signs which are a important task for autonomous cars.