• Stars
    star
    15,199
  • Rank 1,806 (Top 0.04 %)
  • Language
    Go
  • License
    MIT License
  • Created over 11 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

general purpose extensions to golang's database/sql

sqlx

CircleCI Coverage Status Godoc license

sqlx is a library which provides a set of extensions on go's standard database/sql library. The sqlx versions of sql.DB, sql.TX, sql.Stmt, et al. all leave the underlying interfaces untouched, so that their interfaces are a superset on the standard ones. This makes it relatively painless to integrate existing codebases using database/sql with sqlx.

Major additional concepts are:

  • Marshal rows into structs (with embedded struct support), maps, and slices
  • Named parameter support including prepared statements
  • Get and Select to go quickly from query to struct/slice

In addition to the godoc API documentation, there is also some user documentation that explains how to use database/sql along with sqlx.

Recent Changes

1.3.0:

  • sqlx.DB.Connx(context.Context) *sqlx.Conn
  • sqlx.BindDriver(driverName, bindType)
  • support for []map[string]interface{} to do "batch" insertions
  • allocation & perf improvements for sqlx.In

DB.Connx returns an sqlx.Conn, which is an sql.Conn-alike consistent with sqlx's wrapping of other types.

BindDriver allows users to control the bindvars that sqlx will use for drivers, and add new drivers at runtime. This results in a very slight performance hit when resolving the driver into a bind type (~40ns per call), but it allows users to specify what bindtype their driver uses even when sqlx has not been updated to know about it by default.

Backwards Compatibility

Compatibility with the most recent two versions of Go is a requirement for any new changes. Compatibility beyond that is not guaranteed.

Versioning is done with Go modules. Breaking changes (eg. removing deprecated API) will get major version number bumps.

install

go get github.com/jmoiron/sqlx

issues

Row headers can be ambiguous (SELECT 1 AS a, 2 AS a), and the result of Columns() does not fully qualify column names in queries like:

SELECT a.id, a.name, b.id, b.name FROM foos AS a JOIN foos AS b ON a.parent = b.id;

making a struct or map destination ambiguous. Use AS in your queries to give columns distinct names, rows.Scan to scan them manually, or SliceScan to get a slice of results.

usage

Below is an example which shows some common use cases for sqlx. Check sqlx_test.go for more usage.

package main

import (
    "database/sql"
    "fmt"
    "log"
    
    _ "github.com/lib/pq"
    "github.com/jmoiron/sqlx"
)

var schema = `
CREATE TABLE person (
    first_name text,
    last_name text,
    email text
);

CREATE TABLE place (
    country text,
    city text NULL,
    telcode integer
)`

type Person struct {
    FirstName string `db:"first_name"`
    LastName  string `db:"last_name"`
    Email     string
}

type Place struct {
    Country string
    City    sql.NullString
    TelCode int
}

func main() {
    // this Pings the database trying to connect
    // use sqlx.Open() for sql.Open() semantics
    db, err := sqlx.Connect("postgres", "user=foo dbname=bar sslmode=disable")
    if err != nil {
        log.Fatalln(err)
    }

    // exec the schema or fail; multi-statement Exec behavior varies between
    // database drivers;  pq will exec them all, sqlite3 won't, ymmv
    db.MustExec(schema)
    
    tx := db.MustBegin()
    tx.MustExec("INSERT INTO person (first_name, last_name, email) VALUES ($1, $2, $3)", "Jason", "Moiron", "[email protected]")
    tx.MustExec("INSERT INTO person (first_name, last_name, email) VALUES ($1, $2, $3)", "John", "Doe", "[email protected]")
    tx.MustExec("INSERT INTO place (country, city, telcode) VALUES ($1, $2, $3)", "United States", "New York", "1")
    tx.MustExec("INSERT INTO place (country, telcode) VALUES ($1, $2)", "Hong Kong", "852")
    tx.MustExec("INSERT INTO place (country, telcode) VALUES ($1, $2)", "Singapore", "65")
    // Named queries can use structs, so if you have an existing struct (i.e. person := &Person{}) that you have populated, you can pass it in as &person
    tx.NamedExec("INSERT INTO person (first_name, last_name, email) VALUES (:first_name, :last_name, :email)", &Person{"Jane", "Citizen", "[email protected]"})
    tx.Commit()

    // Query the database, storing results in a []Person (wrapped in []interface{})
    people := []Person{}
    db.Select(&people, "SELECT * FROM person ORDER BY first_name ASC")
    jason, john := people[0], people[1]

    fmt.Printf("%#v\n%#v", jason, john)
    // Person{FirstName:"Jason", LastName:"Moiron", Email:"[email protected]"}
    // Person{FirstName:"John", LastName:"Doe", Email:"[email protected]"}

    // You can also get a single result, a la QueryRow
    jason = Person{}
    err = db.Get(&jason, "SELECT * FROM person WHERE first_name=$1", "Jason")
    fmt.Printf("%#v\n", jason)
    // Person{FirstName:"Jason", LastName:"Moiron", Email:"[email protected]"}

    // if you have null fields and use SELECT *, you must use sql.Null* in your struct
    places := []Place{}
    err = db.Select(&places, "SELECT * FROM place ORDER BY telcode ASC")
    if err != nil {
        fmt.Println(err)
        return
    }
    usa, singsing, honkers := places[0], places[1], places[2]
    
    fmt.Printf("%#v\n%#v\n%#v\n", usa, singsing, honkers)
    // Place{Country:"United States", City:sql.NullString{String:"New York", Valid:true}, TelCode:1}
    // Place{Country:"Singapore", City:sql.NullString{String:"", Valid:false}, TelCode:65}
    // Place{Country:"Hong Kong", City:sql.NullString{String:"", Valid:false}, TelCode:852}

    // Loop through rows using only one struct
    place := Place{}
    rows, err := db.Queryx("SELECT * FROM place")
    for rows.Next() {
        err := rows.StructScan(&place)
        if err != nil {
            log.Fatalln(err)
        } 
        fmt.Printf("%#v\n", place)
    }
    // Place{Country:"United States", City:sql.NullString{String:"New York", Valid:true}, TelCode:1}
    // Place{Country:"Hong Kong", City:sql.NullString{String:"", Valid:false}, TelCode:852}
    // Place{Country:"Singapore", City:sql.NullString{String:"", Valid:false}, TelCode:65}

    // Named queries, using `:name` as the bindvar.  Automatic bindvar support
    // which takes into account the dbtype based on the driverName on sqlx.Open/Connect
    _, err = db.NamedExec(`INSERT INTO person (first_name,last_name,email) VALUES (:first,:last,:email)`, 
        map[string]interface{}{
            "first": "Bin",
            "last": "Smuth",
            "email": "[email protected]",
    })

    // Selects Mr. Smith from the database
    rows, err = db.NamedQuery(`SELECT * FROM person WHERE first_name=:fn`, map[string]interface{}{"fn": "Bin"})

    // Named queries can also use structs.  Their bind names follow the same rules
    // as the name -> db mapping, so struct fields are lowercased and the `db` tag
    // is taken into consideration.
    rows, err = db.NamedQuery(`SELECT * FROM person WHERE first_name=:first_name`, jason)
    
    
    // batch insert
    
    // batch insert with structs
    personStructs := []Person{
        {FirstName: "Ardie", LastName: "Savea", Email: "[email protected]"},
        {FirstName: "Sonny Bill", LastName: "Williams", Email: "[email protected]"},
        {FirstName: "Ngani", LastName: "Laumape", Email: "[email protected]"},
    }

    _, err = db.NamedExec(`INSERT INTO person (first_name, last_name, email)
        VALUES (:first_name, :last_name, :email)`, personStructs)

    // batch insert with maps
    personMaps := []map[string]interface{}{
        {"first_name": "Ardie", "last_name": "Savea", "email": "[email protected]"},
        {"first_name": "Sonny Bill", "last_name": "Williams", "email": "[email protected]"},
        {"first_name": "Ngani", "last_name": "Laumape", "email": "[email protected]"},
    }

    _, err = db.NamedExec(`INSERT INTO person (first_name, last_name, email)
        VALUES (:first_name, :last_name, :email)`, personMaps)
}

More Repositories

1

humanize

python humanize functions
Python
1,672
star
2

jsonq

simple json field access for golang
Go
586
star
3

modl

golang database modelling library
Go
478
star
4

johnny-cache

johnny cache django caching framework
Python
306
star
5

monet

golang blog
Go
194
star
6

speedparser

feedparser but faster and worse
Python
101
star
7

gowiki

single-file single-executable wiki written in golang
Go
76
star
8

jigo

an attempt at a jinja2 implementation in go
Go
72
star
9

dmc

dmc runs commands via ssh on multiple machines
Go
34
star
10

django-slow-log

django slow request log
Python
31
star
11

mandira

language agnostic logic-light template system
Go
27
star
12

micromongo

tiny fast python orm-ish tools for mongodb
Python
26
star
13

arachne

a complex but scalable web spider
Python
25
star
14

aranha

simple python gevent web spider
Python
23
star
15

python-github

simple python github api2 library
Python
16
star
16

cm

very simple config manager
Go
15
star
17

par2ools

par2 tools (par2ls, par2mv, par2unrar)
Python
12
star
18

what-the-sql

online sql test
Go
11
star
19

contact-form

server that implements an email contact form for an otherwise static website
Go
10
star
20

euler.go

project euler solutions in go
Go
9
star
21

jmoiron.net

personal blog + website written with flask
Python
8
star
22

dotfiles

dotfiles + cm
Vim Script
8
star
23

jinkies

jinkies is a simple jenkins cli
Python
6
star
24

ongaku

http music player for local files streamable to a chromecast via chrome tab streaming
Go
6
star
25

gaspar

generic eventlet+zmq worker library
Python
5
star
26

kokuen

django-app performance tracking w/ statsd + graphite
Python
5
star
27

pdf2zip

converts image-based pdfs to a zipfile full of extracted images
Python
5
star
28

gevent-memcache-bench

python memcached speed tests w/ and w/o gevent
Python
4
star
29

etod

epoch to date
Go
3
star
30

redtape

simple program to create simple html docs from simple markdown files
Python
3
star
31

uromkan

python unicode romaji to kana conversion
Python
3
star
32

iris

python command-line photo management thing
Python
2
star
33

daneel

Python
2
star
34

chapman

chapman python backup nonsense
Python
2
star
35

nvidia-gpu-temp

Python
1
star
36

terminal-schemer

portable style scheme application for mate-terminal and gnome-terminal
Go
1
star
37

cs101

simplistic explorations of basic CS data structures & algorithms
Go
1
star
38

pctilde

pctilde emulates the behavior of the zsh PS1 var '%~'
Go
1
star
39

golang-sdk

Go
1
star
40

ulv-covers-modern

ulv covers for gregtech ceu modern
Java
1
star