Discover rab4u/spark_generic_file_formatter Open Source project

Stars
3
Rank 3,963,521 (Top 79 %)
Language
Scala
Created over 5 years ago
Updated over 2 years ago

rab4u

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

This spark job is used to convert any file format (JSON, CSV, TXT, SEQ, PARQUET, ORC, AVRO) to any file format (JSON, CSV, TXT, SEQ, PARQUET, ORC, AVRO). One of the cool feature is you can exclude or select Columns in the Output (Supports Nested fields).

flume-s3-sink

flume-s3-sink is a custome sink for Flume NG to write events to AWS S3. It uses AWS SDK to interact with S3 and supports both static and temporary credentials

Java

aws-glue-schema-registry-learning

This repository helps to understand how to access the glue schema registry and perform basic schema validations.

datadoorway

DataDoorway is your doorway to send / receive data from various data sources

Python

apicurio-registry-learning

This repo contains docker-compose yaml to run apicurio locally with basic validation example in Java and Python

Python

aws_ecs_zookeeper