Discover MrPowers/mrpowers.github.io Open Source project

pyspark methods to enhance developer productivity 📣 👯 🎉

743

quinn

Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)

582

spark-fast-tests

PySpark test helper methods with beautiful error messages

418

chispa

Delta Lake helper methods in PySpark

351

mack

spark-style-guide

code_quizzer

Programming practice questions with Ruby, JavaScript, Rails, and Bash.

HTML

201

frontend-generators

Rake tasks to add Bootstrap, Font Awesome, and Start Bootstrap Landing Pages to a Rails app

CSS

spark-sbt.g8

A giter8 template for Spark SBT projects

Spark functions to run popular phonetic and string matching algorithms

spark-stringmetric

Filling in the Spark function gaps across APIs

bebe

Delta lake and filesystem helper methods

jodie

Fake Pandas / PySpark DataFrame creator

farsante

Rust

beavis

Pandas helper functions

tic_tac_toe

Ruby tic tac toe game

PySpark phonetic and string matching algorithms

ceja

Spark DataFrame transformation and UDF test examples

spark-test-example

Test suite to document the behavior of Spark

spark-spec

An example PySpark project with pytest

gill

Modeling directed acyclic graphs (DAG) for topological sorting, shortest path, longest path, etc.

directed_graph

Speak Slack notifications and process Slack slash commands

spark-slack

Testing Scala code with scalatest

scalatest-example

Using the Parquet file format with Python

python-parquet-examples

Delta Lake helper methods. No Spark dependency.

levi

DAGs on DAGs! Smart PySpark custom transformation runner

unicron

PySpark testing example project

pysparktestingexample

Example Spark project with Java API

JavaSpark

Java

spark-pika

Demo how to set up Spark with SBT

Lightweight Spark ETL framework

spark-etl

Helping Slack and Trello play together nicely

slack_trello

A curated list of awesome Apache Spark packages and resources.

awesome-spark

mill_spark_example

Apache Spark project with the Mill build tool

MrPowers benchmarks for Dask, Polars, DataFusion, and pandas

mrpowers-benchmarks

Style for the PyData stack

pydata-style-guide

walle

Compression algorithms for different file formats

PySpark on Poetry example

angelou

Curated collection of Spark libraries and example applications

great-spark

appa

Data lake metadata / transaction log store

Set application variables for the development, test, and production environments

turf

PySpark Hive helper methods

eren

Prawn gem to develop vector line charts

prawn_charts

Tutorial on running faster tests with Spark

spark-bulba

Introduction to Machine Learning with Python Book

ml-book

Great Pandas and Jupyter workflow with Poetry

blake

Model cmap exports as a directed graph and generate SQL

cmap

Using the Redshift UNLOAD/COPY commands to move data from one Redshift cluster/database to another

redshift_extractor

Delta Lake powered by Dask

deltadask

A Spark playground to help me write blog posts

spark-examples

Typed Datasets with Spark

spark-frameless

Making it easier to work with the slack_notifier gem

slack_notifier_wrapper

Templates in Scala with Scalate

scalate-example

Creative by Start Bootstrap - Rails Version

rails-startbootstrap-creative

Easily manage clone Git repos in Ruby applications

repo_tools

Simple example of the MUnit testing library

munit-example

Implementing HyperLogLog functions in Spark

hll-example

Documents the behavior of pyspark

pyspark-spec

Integration tests to demonstrate Dask's interoperability with other systems

dask-interop

Code to support MungingData blog posts: https://mungingdata.com/

mungingdata

dask-fun

Dask examples with tests

Practicing Vim after completing the vimtutor

vimtraining

technical_writing

Elements of style for blogs, books, and presentations

scala-design

Core Scala language features and design patterns

This is a GameBoard class with methods to help analyze the grid.

GameBoard

Guide to provision a Mac for developers

cali

Vim Script

data-scrapbook

A collection of images and captions to explain core data concepts

sapo

Data store validator for sqlite, Parquet

learn_spanish

Logically learn Spanish

Some Project Euler solutions

project_euler

yellow-taxi

Data lake fun!

Creating a sqlite db and writing it to files

sqlite-example

http_validator

Example notebooks for how to use pola.rs

polars-fun

Print colorful tables with nice diffs in the Terminal

mesita

Example of how to use uTest with Spark

spark-utest

Book on MrPowers OSS projects, blogs, and other assets

mrpowers-book

doctor_scrabble

dotfiles

mini_yelp

eli5_ruby_cs

explain like I'm 5: computer science with ruby

PySpark example notebooks

pyspark-examples

A tic tac toe game, written in JS, with DOM crap isolated out of the way

tic_tac_toe_js

JavaScript

go-example

Simple Go project

custom_tableau

Using JavaScript to create Tableu-like dashboards

JavaScript

ansible_playbooks

Ansible playbooks