• Stars
    star
    256
  • Rank 159,219 (Top 4 %)
  • Language
    Python
  • License
    BSD 3-Clause "New...
  • Created about 12 years ago
  • Updated about 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

PostgreSQL database access simplified

Queries: PostgreSQL Simplified

Queries is a BSD licensed opinionated wrapper of the psycopg2 library for interacting with PostgreSQL.

The popular psycopg2 package is a full-featured python client. Unfortunately as a developer, you're often repeating the same steps to get started with your applications that use it. Queries aims to reduce the complexity of psycopg2 while adding additional features to make writing PostgreSQL client applications both fast and easy. Check out the Usage section below to see how easy it can be.

Key features include:

  • Simplified API
  • Support of Python 2.7+ and 3.4+
  • PyPy support via psycopg2cffi
  • Asynchronous support for Tornado
  • Connection information provided by URI
  • Query results delivered as a generator based iterators
  • Automatically registered data-type support for UUIDs, Unicode and Unicode Arrays
  • Ability to directly access psycopg2 connection and cursor objects
  • Internal connection pooling

Version Status Coverage License

Documentation

Documentation is available at https://queries.readthedocs.org

Installation

Queries is available via pypi and can be installed with easy_install or pip:

pip install queries

Usage

Queries provides a session based API for interacting with PostgreSQL. Simply pass in the URI of the PostgreSQL server to connect to when creating a session:

session = queries.Session("postgresql://postgres@localhost:5432/postgres")

Queries built-in connection pooling will re-use connections when possible, lowering the overhead of connecting and reconnecting.

When specifying a URI, if you omit the username and database name to connect with, Queries will use the current OS username for both. You can also omit the URI when connecting to connect to localhost on port 5432 as the current OS user, connecting to a database named for the current user. For example, if your username is fred and you omit the URI when issuing queries.query the URI that is constructed would be postgresql://fred@localhost:5432/fred.

If you'd rather use individual values for the connection, the queries.uri() method provides a quick and easy way to create a URI to pass into the various methods.

>>> queries.uri("server-name", 5432, "dbname", "user", "pass")
'postgresql://user:pass@server-name:5432/dbname'

Environment Variables

Currently Queries uses the following environment variables for tweaking various configuration values. The supported ones are:

  • QUERIES_MAX_POOL_SIZE - Modify the maximum size of the connection pool (default: 1)

Using the queries.Session class

To execute queries or call stored procedures, you start by creating an instance of the queries.Session class. It can act as a context manager, meaning you can use it with the with keyword and it will take care of cleaning up after itself. For more information on the with keyword and context managers, see PEP343.

In addition to both the queries.Session.query and queries.Session.callproc methods that are similar to the simple API methods, the queries.Session class provides access to the psycopg2 connection and cursor objects.

Using queries.Session.query

The following example shows how a queries.Session object can be used as a context manager to query the database table:

>>> import pprint
>>> import queries
>>>
>>> with queries.Session() as session:
...     for row in session.query('SELECT * FROM names'):
...         pprint.pprint(row)
...
{'id': 1, 'name': u'Jacob'}
{'id': 2, 'name': u'Mason'}
{'id': 3, 'name': u'Ethan'}

Using queries.Session.callproc

This example uses queries.Session.callproc to execute a stored procedure and then pretty-prints the single row results as a dictionary:

>>> import pprint
>>> import queries
>>> with queries.Session() as session:
...   results = session.callproc('chr', [65])
...   pprint.pprint(results.as_dict())
...
{'chr': u'A'}

Asynchronous Queries with Tornado

In addition to providing a Pythonic, synchronous client API for PostgreSQL, Queries provides a very similar asynchronous API for use with Tornado. The only major difference API difference between queries.TornadoSession and queries.Session is the TornadoSession.query and TornadoSession.callproc methods return the entire result set instead of acting as an iterator over the results. The following example uses TornadoSession.query in an asynchronous Tornado web application to send a JSON payload with the query result set.

from tornado import gen, ioloop, web
import queries

class MainHandler(web.RequestHandler):

    def initialize(self):
        self.session = queries.TornadoSession()

    @gen.coroutine
    def get(self):
        results = yield self.session.query('SELECT * FROM names')
        self.finish({'data': results.items()})
        results.free()

application = web.Application([
    (r"/", MainHandler),
])

if __name__ == "__main__":
    application.listen(8888)
    ioloop.IOLoop.instance().start()

Inspiration

Queries is inspired by Kenneth Reitz's awesome work on requests.

History

Queries is a fork and enhancement of pgsql_wrapper, which can be found in the main GitHub repository of Queries as tags prior to version 1.2.0.

More Repositories

1

consulate

Python client for the Consul HTTP API
Python
342
star
2

pgsql-listen-exchange

RabbitMQ Exchange that publishes messages received from PostgreSQL Notifications.
Erlang
282
star
3

rabbitpy

A pure python, thread-safe, minimalistic and pythonic RabbitMQ client library
Python
242
star
4

tinman

Tinman is a Tornado support package including an application wrapper/runner and a set of handy decorators.
Python
187
star
5

RabbitMQ-in-Depth

Examples and materials for RabbitMQ in Depth
Python
133
star
6

flatdict

Python module for interacting with nested dicts as a single level dict with delimited keys.
Python
111
star
7

env-aws-params

Inject AWS SSM Parameters as Environment Variables
Go
86
star
8

rejected

rejected is a consumer framework for RabbitMQ
Python
58
star
9

pamqp

Low level AMQP frame encoding and decoding library
Python
51
star
10

rabbitmq-flume-plugin

A Flume plugin that provides a RabbitMQ Source and Sink
Java
48
star
11

pycon2013-logging

Examples from my "Become a Logging Expert in 30 Minutes" talk at PyCon 2013
Python
48
star
12

glyphicons-sprite-generator

The GLYPHICONS Sprite Generator allows you to change the size and colors of the Bootstap icons using the Free or Pro versions of GLYPHICONS.
Python
47
star
13

rabbitmq-pulse

rabbitmq-pulse is a RabbitMQ plugin that publishes node and queue information, pushing stats instead of polling the Management API
Erlang
45
star
14

tornado-elasticsearch

Extends the official Elasticsearch Python API adding Tornado AsyncHTTPClient support
Python
38
star
15

influxdb-storage-exchange

Post JSON structured event messages to InfluxDB
Erlang
33
star
16

alpine-rabbitmq-autocluster

RabbitMQ image with the autocluster plugin
32
star
17

email-normalize

Return a normalized email-address stripping ISP specific behaviors
Python
30
star
18

pgdumplib

Python3 library for reading and writing pg_dump files using the custom format
Python
30
star
19

On-Rabbits-and-Elephants

Code for my lightning talk at pgCon 2011
Python
29
star
20

helper

Development library for quickly writing configurable applications and daemons.
Python
26
star
21

statelessd

statelessd is a stateless HTTP to AMQP publishing gateway prototype in Python
Python
25
star
22

kvpbench

Key-Value Store Benchmark Application
Python
24
star
23

srvlookup

A small wrapper for dnspython to return SRV records for a given host, protocol, and domain name as a list of namedtuples
Python
23
star
24

passport

Passport generates templated configuration files using Consul's Service Discovery and Key/Value database tools
Python
22
star
25

aiorabbit

An AsyncIO RabbitMQ client for Python 3
Python
21
star
26

tredis

An asynchronous Redis client for Tornado
Python
20
star
27

hockeyapp

Python Client for the HockeyApp.net API
Python
19
star
28

mredis

MRedis is a multi-server wrapper for the excellent Python Redis client.
Python
19
star
29

progrock

A multi-progressbar implementation to complement multiprocessing.Process.
Python
18
star
30

pg-statsd

pg_statsd is a set of PostgreSQL user-defined functions that provide and interface to statsd.
C
15
star
31

strftimerl

Erlang implementation of strftime
Erlang
13
star
32

urilib

A RFC-3986 URI Library for parsing and building URIs
Erlang
13
star
33

infoblox

A python library for interfacing with Infoblox NIOS.
Python
12
star
34

mikkoo

Mikkoo is a PgQ to RabbitMQ Relay
Python
12
star
35

huesos-de-vaquero

A skeleton project for web development using Cowboy (Erlang) for back-end development and Google Web-Starter-Kit for front-end development
Erlang
12
star
36

tornado-aws

A low-level Amazon Web Services API client for Tornado
Python
12
star
37

houston

Easy docker stack deployment to CoreOS clusters using Fleet and Consul
Python
11
star
38

framewerk

Framewerk is a PHP5 OO Framework
PHP
10
star
39

alpine-pgbouncer

Minimalistic Alpine Linux pgBouncer container
Shell
8
star
40

httpbl

Python client library for the project-honeypot.org Http:BL API
Python
8
star
41

cardiff

Cardiff is an extendible statsd clone written in Python that supports all statsd metric types and a variety of configurable stats destinations.
Python
8
star
42

fluent-plugin-json-in-json

Fluentd parser plugin that parses JSON attributes with JSON strings in them
Ruby
7
star
43

erlln

Example Erlang OTP application that listens for PostgreSQL NOTIFY messages
Erlang
7
star
44

pgpasslib

Library for getting passwords from a PostgreSQL Password file
Python
7
star
45

conserl

Erlang client for Consul
Erlang
6
star
46

mgmt-stress

A RabbitMQ stress-testing tool
Erlang
6
star
47

privatepaste

v2 of PrivatePaste.com
Erlang
5
star
48

python-tornado.tmbundle

A TextMate 2 Bundle for Tornado
5
star
49

ossicons

An icon font for popular Open Source project logos
HTML
5
star
50

alpine-postgres

gavinmroy/alpine-postgres is a Docker image with a small disk footprint for PostgreSQL
Shell
5
star
51

dynamodb-backup

CLI utility for backing up DynamoDB to Avro containers
Python
4
star
52

pgsql-utilities

A collection of PostgreSQL utilities
PHP
4
star
53

logging-config

A wrapper class for the Python standard logging module
Python
4
star
54

fleetpy

An opinionated fleet API client for Python
Python
4
star
55

rabbitmq-sns-plugin

RabbitMQ plugin for AWS SNS interoperability
Makefile
4
star
56

golconde

Golconde (gŏl-kŏn'dΙ™) is a queue based data-distribution solution for PostgreSQL written in Python 2.6.
Python
4
star
57

github-issue-autoresponder

Check GitHub projects periodically for new issues and automatically comment or label them based upon string matching rules
Python
4
star
58

rabbitstew

A command-line tool for publishing messages to RabbitMQ
Python
4
star
59

couchclient

A light-weight, read-only CouchDB client
Python
4
star
60

pglifecycle

A PostgreSQL schema management tool
Python
3
star
61

rmq-node-report

A small set of scripts to gather information from a local RabbitMQ node
Erlang
3
star
62

pgparse

Python wrapper for libpg_query
Python
3
star
63

alpine-rabbitmq

RabbitMQ image with a small disk footprint
3
star
64

PyCon-2012-Tornado-IOLoop-Talk-Examples

Examples from my 2012 PyCon talk: "More than just a pretty web framework, the Tornado IOLoop"
Python
3
star
65

menwith

Menwith is a memcache text protocol command utilization analysis tool.
Python
3
star
66

elmer

Elmer allows granular monitoring of RabbitMQ queues with Nagios and other monitoring tools using custom queue arguments.
Erlang
3
star
67

http-exchange

RabbitMQ exchange for stateless publishing over HTTP
Erlang
3
star
68

alpine-statsd

A statsd container with a small disk footprint
3
star
69

boxcar

An experimental OS/X app for interfacing with Vagrant
Objective-C
3
star
70

rmq-cluster-rebalance

CLI application for rebalancing queues in a RabbitMQ cluster
Python
3
star
71

rabbitmq-in-depth-cookbook

Cookbook used by Vagrant when setting up the RabbitMQ in Depth VM
Python
2
star
72

httpc-aws

A light-weight, relatively unopinionated AWS API client for Erlang
Erlang
2
star
73

cowboy-bones-gutenberg-generator

An #Erlang Cowboy web app Gutenberg generator with i18n, templating, and more
Erlang
2
star
74

tdns

An asynchronous Tornado pycares DNS client wrapper
Python
2
star
75

rabbitmq-management-ui-prototype

A ReactJS RabbitMQ Management UI Prototype
TypeScript
2
star
76

pgdumpreport

Create reports from pg_dump archives
JavaScript
2
star
77

hashicorp-packaging

Makefile and templates for creating deb packages for HashiCorp (and ecosystem) projects.
Makefile
2
star
78

formulary

Easy management of AWS Cloud Formation stacks
Python
2
star
79

alpine-redis

Redis image with a small disk footprint
Dockerfile
2
star
80

ecs-pipeline-deploy

An opinionated deployment application for ECS services
Python
2
star
81

rabbitmq-pagerduty

A RabbitMQ monitoring plugin for triggering PagerDuty alerts
Erlang
2
star
82

pg-graph-dump

Dependency graph based dump and reload of PostgreSQL DDL
Erlang
2
star
83

ppp-loan-data

PPP FOIA Data
2
star
84

mdnserl

Erlang library for providing RFC-6762 multicast-DNS announcement and discovery
Erlang
1
star
85

rmq-definitions

Deterministically sort and format RabbitMQ definition backups
Python
1
star
86

telegraf-pgbouncer

pgBouncer Stats Collector for Telegraf
Python
1
star
87

ssm-ps-template

Command line application to render templates with data from SSM Parameter Store
Python
1
star
88

s3-config-overlay

Fetch config from amazon S3 for other containers
Shell
1
star
89

estuary

Accumulates Avro datum received via RabbitMQ messages and stores them in Avro container files
Erlang
1
star
90

remy

Remy is a tool for managing Chef repositories and automating the addition and management of cookbooks with a chef server using Jenkins and GitHub.
Python
1
star
91

topological

Python3 library of topological sorting algorithms
Python
1
star
92

google-apps-for-domains-utilities

Small python utilities for managing Google Apps for Domains accounts
Python
1
star
93

apiary

JavaScript
1
star
94

tornado-template-gen

This application uses the Tornado Template module to generate static HTML files while still making available a subset of the Tornado template methods like static_url.
Python
1
star
95

rabbitmq-aws

A RabbitMQ Plugin fork of httpc-aws - Now @ rabbitmq/rabbitmq-aws
Erlang
1
star
96

tornado-dynamodb

Deprecated for sprockets.clients.dynamodb
Python
1
star