avensolutions/cdc-at-scale-using-spark avensolutions/cdc-at-scale-using-spark

  • Stars
    star
    18
  • Rank 1,208,065 (Top 24 %)
  • Language
    Python
  • Created over 5 years ago
  • Updated over 5 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
avensolutions/cdc-at-scale-using-spark
github

avensolutions

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Scalable CDC Pattern Implemented using PySpark

More Repositories

1

spark-sql-etl-framework

Multi-stage, config driven, SQL based ETL framework using PySpark
Python
25
star
2

cdc-in-aws-glue

Source Change Detection and Capture using PySpark and AWS Glue
Python
9
star
3

synthetic-cdc-data-generator

Application that generates change sets which can be used to develop and test CDC patterns
Python
5
star
4

cdc-in-pig-and-spark

Source Change Detection Pattern Generation for Hadoop Implemented in Spark (PySpark) or Apache Pig (MR or Tez)
Python
4
star
5

spark-on-gcp

Tutorial on Deploying Apache Spark on GCP
HCL
2
star
6

hcat-R

HCatalog Functions for R
R
2
star
7

ansible-ec2-hadoop-cluster

Deploy HDP Cluster in AWS EC2 Using Ansible
Shell
2
star
8

datadog_yarn_metrics

Collects application metrics from YARN and publishes these to DataDog
Python
2
star
9

yarn-stats-collection

Collect MapReduce Job Statistics from a Hadoop Cluster
Python
2
star
10

baconbits

Collection of Apache Pig UDFs
Java
2
star
11

cloud-sql-postgres-availability-tutorial

Tutorial on Using Read Replicas in Cloud SQL
HCL
2
star
12

automated-gcs-object-scanning-using-dlp-with-notifications-using-slack

Automate Detection and Notification of Sensitive Data Objects Uploaded to Google Cloud Storage
HCL
2
star
13

plantuml-cloud-image-library

Cloud and SaaS resource images available for use in PlantUML diagrams
2
star
14

example-bigquery-dbt-project

Example BigQuery DBT project
1
star
15

terraform-google-app-engine-wordpress

HCL
1
star
16

simple-notifications-with-lambda-and-ses

S3 Object Notifications using Lambda and SES
HCL
1
star
17

gcs-object-notifications-using-slack

Use Slack for Notification of Newly Created Objects in GCS
HCL
1
star
18

hcat-py

Python module to return metadata for objects in HCatalog
Python
1
star
19

simplewebpy

Python Web Programming Made Simple
HTML
1
star
20

google-data-workshop

Google data workshop covering Terraform, DBT and Composer
Python
1
star
21

fuzzymatching-pig-udf

Python UDF for Apache Pig and Spark to return n-grams and q-grams used for approximate string matching
Python
1
star
22

json-wrangling-with-golang

Tutorial on JSON Handling with Golang
Go
1
star
23

simple-lambda-ec2-scheduler

Schedule Operations on EC2 Instances Using Lambda and CloudWatch
HCL
1
star