gabfl/pg_dump-to-s3

Stars
121
Rank 292,254 (Top 6 %)
Language
Shell
Created about 8 years ago
Updated about 1 year ago

gabfl/pg_dump-to-s3

gabfl

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Automatically dump and archive PostgreSQL backups to Amazon S3

pg_dump-to-s3

Automatically dump and archive PostgreSQL backups to Amazon S3.

Requirements

AWS cli: pip install awscli

Setup

Use aws configure to store your AWS credentials in ~/.aws (read documentation)
Edit .conf and set your PostgreSQL's credentials and the list of databases to back up
If your PostgreSQL connection uses a password, you will need to store it in ~/.pgpass (read documentation)

Usage

./pg_dump-to-s3.sh

#  * Backup in progress.,.
#    -> backing up test...
# upload: ../../../tmp/2023-06-28-at-22-20-08_test.dump to s3://*****/backups/2023-06-28-at-22-20-08_test.dump
#       ...database test has been backed up
#  * Deleting old backups...

# ...done!

Restore a backup

# USAGE: pg_restore-from-s3.sh [db target] [s3 object]

./pg_restore-from-s3.sh my_database_1 2023-06-28-at-10-29-44_my_database_1.dump

# download: s3://your_bucket/folder/2023-06-28-at-22-17-15_my_database_1.dump to /tmp/2023-06-28-at-22-17-15_my_database_1.dump
# Database my_database_1 already exists, skipping creation
# 2023-06-28-at-22-17-15_my_database_1.dump restored to database my_database_1

vault

Python password manager

bigquery_fdw

BigQuery Foreign Data Wrapper for PostgreSQL

mysql_generate_series

mysql_generate_series is a MySQL replica of PostgreSQL's generate_series functions

redis-priority-queue

Simple Redis work queue with added features (priorities, pop multiple items at once)

sql2csv

Run MySQL and PostgreSQL queries and store result in CSV

password-generator-py

Passwords easy for humans, hard for computers

mysql-batch

Run large MySQL UPDATE and DELETE queries with small batches to prevent table/row-level locks

webhook

Easily test webhooks and other HTTP requests

dbschema

MySQL/PostgreSQL schema migrations made easy

pg-batch

Run large PostgreSQL UPDATE and DELETE queries with small batches to prevent locks

roulette

Python roulette game

PHP-PSR-2-Fixer-Coda-Plugin

Coda Plugin that uses PHP_CodeSniffer to automatically fix PHP coding standards issues

sitecrawl

Simple Python module to crawl a website and extract URLs

shachange

Change an image sha signature

pg_IdFromDate

PostgreSQL methods to efficiently find the ID associated to a date in a big data table

http-sqlite

Rest API built on top of SQLite

cloudssh

EC2 SSH connections helper

redis-expire-least-used

Redis 2 step expiration tool to pre-expire least used items on a Redis server

sample-django

Django basic example

sitemap_maker

Python tool to generate sitemap XML files

sample-flask

Flask basic example

deb