Discover seamusabshere/py-upsert Open Source project

Stars
18
Rank 1,208,065 (Top 24 %)
Language
Python
Created about 12 years ago
Updated over 1 year ago

seamusabshere

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Python library to make it easy to upsert on MySQL, PostgreSQL, and SQLite3.

fuzzy_match

Find a needle (a document or record) in a haystack using string similarity and (optionally) regular expression rules. Uses Dice's Coefficient (aka Pair Similiarity) and Levenshtein Distance internally.

Ruby

668

upsert

Upsert on MySQL, PostgreSQL, and SQLite3. Transparently creates functions (UDF) for MySQL and PostgreSQL; on SQLite3, uses INSERT OR IGNORE.

Ruby

652

data_miner

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

Ruby

301

unix_utils

Like FileUtils, but provides zip, unzip, bzip2, bunzip2, tar, untar, sed, du, md5sum, shasum, cut, head, tail, wc, unix2dos, dos2unix, iconv, curl, perl, etc.

Ruby

227

remote_table

Open local or remote XLSX, XLS, ODS, CSV (comma separated), TSV (tab separated), other delimited, fixed-width files, and Google Docs. Returns an enumerator of Arrays or Hashes, depending on whether there are headers.

HTML

225

cache_method

Cache based on arguments AND object state; store in memcached, redis, or in-process. Like alias_method, but it's cache_method! One step beyond memoization.

Ruby

136

lock_and_cache

Most caching libraries don't do locking, meaning that >1 process can be calculating a cached value at the same time. Since you presumably cache things because they cost CPU, database reads, or money, doesn't it make sense to lock while caching?

Ruby

134

mysql2xxxx

Gives you binaries like mysql2csv, mysql2json, and mysql2xml, and Ruby classes to match.

Ruby

cache

Defines a simple interface to multiple cache-like storage engines by wrapping common Ruby client libraries like memcached, redis, memcache-client, dalli. Handles each underlying library's weirdnesses, including forking.

Ruby

eat

A (better?) replacement for open-uri. Lets you open local and remote files by immediately returning their contents as a string.

Ruby

to_regexp

Provides String#to_regexp

Ruby

errata

Define an errata in table format (CSV) and then apply it to an arbitrary source. Inspired by RFC Errata, lets you keep your own errata in a transparent way.

Ruby

cacheable

DEPRECATED. Use cache_method instead.

Ruby

report

DSL for creating clean CSV, XLSX, and PDF reports in Ruby. Uses xlsx_writer, prawn and pdftk internally.

Ruby

database_url

Convert back and forth between Heroku-style ENV['DATABASE_URL'] and Rails/ActiveRecord-style config/database.yml hashes.

Ruby

lock_method

Like alias_method, but it's lock_method! (lockfiles)

Ruby

common_name

Helps you stop using chains of humanize/downcase/underscore/pluralize/to_sym/etc everywhere in your models, your views, your controllers, etc.

Ruby

engineyard-metadata

Presents a simple, unchanging interface to get metadata about your EngineYard AppCloud instances running on Amazon EC2.

Ruby

cohort_analysis

TBD

Ruby

create_table

Analyze and inspect CREATE TABLE SQL statements and translate across databases. Uses Ragel internally for parsing.

Ruby

ruby_ragel_examples

Examples of using ragel and ruby together

Ruby

fuzzy_infer

Fuzzy set analysis - predicts one or more unknown characteristics of an input case by comparing its known characteristics to a reference dataset whose records contain both the known and unknown characteristics.

Ruby

the_geom_geojson

For PostGIS/PostgreSQL and ActiveRecord, provides "the_geom_geojson" getter and setter that update "the_geom" and "the_geom_webmercator" columns.

Ruby

validates_decency_of

Rails plugin that uses George Carlin's list of seven dirty words (aka swear words, aka cuss words, aka bad words) to check for "decency" on ActiveRecord model attributes.

Ruby

weighted_average

Aircraft.average(:seats) versus Aircraft.weighted_average(:seats, :weighted_by => :takeoffs)

Ruby

loose_tight_dictionary

DEPRECATED: use fuzzy_match. Find a needle in a haystack using string similarity and (optionally) regexp rules.

Ruby

pg_trgm

Ruby

redirect_routing

Ruby

hash_digest

Generates non-cryptographic digests of Hashes (and Arrays) indifferent to key type (string or symbol) and ordering.

Ruby

ey_cloud_awareness

DEPRECATED: use engineyard-metadata. Make your EngineYard cloud instances aware of each other.

Ruby

xml_split

Split XML files on an element, yielding (streaming, so constant memory usage) each node in turn. Uses sgrep2 internally; future versions should use a pure-Ruby SAX parser.

Ruby

characterizable

DEPRECATED. Use charisma instead.

Ruby

has_handle_fallback

Make it easy to use handles (callsigns/monikers/usernames) in URLs, even if they might be blank.

Ruby

table_warnings

Warn yourself of problems with your ActiveRecord tables.

Ruby

to_json_fix

TODO: one-line summary of your gem

Ruby

honeypot

TODO: one-line summary of your gem

Ruby

cohort_scope

DEPRECATED. Use cohort_analysis. Provides cohorts (in the form of ActiveRecord scopes) that dynamically widen until they contain a certain number of records.

Ruby

vector_embed

Vector embedding of strings, booleans, numerics, and arrays into LIBSVM / LIBLINEAR format.

Ruby

switches

Turn on and off parts of your code based on yaml files.

Ruby

flights1percent

1% flights

JavaScript

zmq

Drop-in replacement for zmq gem with included binaries

Ruby

json_to_csv_to_json

csv_to_json and json_to_csv

Ruby

cvg

Like jq or grep for csv. Combine one or more CSVs while filtering on fields with regular expressions, whitelists, presence, missing, etc.

Ruby

has_timestamps

Rails plugin to add named timestamps to ActiveRecord models.

Ruby

zip5

Convert United States zip codes to their correct Zip5 representation, even if they're missing a leading zero and/or they have the +4 suffix.

Ruby

string_enumerator

Given a string containing placeholders (like [color]), enumerate all of the possible strings resulting from filling those placeholders with replacements (like red, blue).

Ruby

nonrandomapp

Ruby

mini_record

mini_record-compat is DEPRECATED. Use original mini_record OR active_record_inline_schema instead.

Ruby

string_replacer

DEPRECATED/POINTLESS - use sed or augeas. Replace text in a file without disturbing the rest of the file.

Ruby

geocode_records

As long as you do very specific things... quickly re-geocode tables.

Ruby

force_schema

[DEPRECATED - use mini_record] Declare a table structure like an ActiveRecord migration and run 'force_schema!' whenever you want. For when you don't need up and down migrations.

Ruby

fast_timestamp

Rapidly and arbitrarily timestamp ActiveRecord records.

Ruby

seamusabshere/py-upsert

seamusabshere

Reviews

Repository Details

More Repositories