behavior-analytics

This project builds a data pipeline to populate the user_behavior_metric table. The user_behavior_metric table is an OLAP table, meant to be used by analysts, dashboarding.

spark-project

data-pipeline-with-gcp

This project implements a data ingestion and processing pipeline to collect, store and process time-series data. The pipeline consists of a publisher, a message queue (Pub/Sub), a consumer, a data warehouse (BigQuery) and a data extractor. The pipeline is designed to be scalable, efficient and easy to maintain.