avensolutions/cdc-at-scale-using-spark

Stars
18
Rank 1,208,065 (Top 24 %)
Language
Python
Created over 5 years ago
Updated over 5 years ago

avensolutions/cdc-at-scale-using-spark

avensolutions

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Scalable CDC Pattern Implemented using PySpark

spark-sql-etl-framework

Multi-stage, config driven, SQL based ETL framework using PySpark

cdc-in-aws-glue

Source Change Detection and Capture using PySpark and AWS Glue

synthetic-cdc-data-generator

Application that generates change sets which can be used to develop and test CDC patterns

cdc-in-pig-and-spark

Source Change Detection Pattern Generation for Hadoop Implemented in Spark (PySpark) or Apache Pig (MR or Tez)

spark-on-gcp

Tutorial on Deploying Apache Spark on GCP

hcat-R

HCatalog Functions for R

ansible-ec2-hadoop-cluster

Deploy HDP Cluster in AWS EC2 Using Ansible

datadog_yarn_metrics

Collects application metrics from YARN and publishes these to DataDog

yarn-stats-collection

Collect MapReduce Job Statistics from a Hadoop Cluster

baconbits

Collection of Apache Pig UDFs

cloud-sql-postgres-availability-tutorial

Tutorial on Using Read Replicas in Cloud SQL

automated-gcs-object-scanning-using-dlp-with-notifications-using-slack

Automate Detection and Notification of Sensitive Data Objects Uploaded to Google Cloud Storage

plantuml-cloud-image-library

Cloud and SaaS resource images available for use in PlantUML diagrams

example-bigquery-dbt-project

Example BigQuery DBT project

terraform-google-app-engine-wordpress

simple-notifications-with-lambda-and-ses

S3 Object Notifications using Lambda and SES

gcs-object-notifications-using-slack

Use Slack for Notification of Newly Created Objects in GCS

hcat-py

Python module to return metadata for objects in HCatalog

simplewebpy

Python Web Programming Made Simple

google-data-workshop

Google data workshop covering Terraform, DBT and Composer

fuzzymatching-pig-udf

Python UDF for Apache Pig and Spark to return n-grams and q-grams used for approximate string matching

json-wrangling-with-golang

Tutorial on JSON Handling with Golang

simple-lambda-ec2-scheduler

Schedule Operations on EC2 Instances Using Lambda and CloudWatch