• Stars
    star
    1,488
  • Rank 31,571 (Top 0.7 %)
  • Language
    Go
  • License
    MIT License
  • Created over 3 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

HTTP requests for Gophers

Requests GoDoc Go Report Card Coverage Status Mentioned in Awesome Go

Requests logo

HTTP requests for Gophers.

The problem: Go's net/http is powerful and versatile, but using it correctly for client requests can be extremely verbose.

The solution: The requests.Builder type is a convenient way to build, send, and handle HTTP requests. Builder has a fluent API with methods returning a pointer to the same struct, which allows for declaratively describing a request by method chaining.

Requests also comes with tools for building custom http transports, include a request recorder and replayer for testing.

See this note on the canonical project URL.

Features

  • Simplifies HTTP client usage compared to net/http
  • Can't forget to close response body
  • Checks status codes by default
  • Supports context.Context
  • JSON serialization and deserialization helpers
  • Easily manipulate URLs and query parameters
  • Request recording and replaying for tests
  • Customizable transports and validators that are compatible with the standard library and third party libraries
  • No third party dependencies
  • Good test coverage

Examples

Simple GET into a string

code with net/http code with requests
req, err := http.NewRequestWithContext(ctx,
	http.MethodGet, "http://example.com", nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
s := string(b)
var s string
err := requests.
	URL("http://example.com").
	ToString(&s).
	Fetch(ctx)
11+ lines5 lines

POST a raw body

code with net/http code with requests
body := bytes.NewReader(([]byte(`hello, world`))
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
	"https://postman-echo.com/post", body)
if err != nil {
	// ...
}
req.Header.Set("Content-Type", "text/plain")
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
_, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
err := requests.
	URL("https://postman-echo.com/post").
	BodyBytes([]byte(`hello, world`)).
	ContentType("text/plain").
	Fetch(ctx)
12+ lines5 lines

GET a JSON object

code with net/http code with requests
var post placeholder
u, err := url.Parse("https://jsonplaceholder.typicode.com")
if err != nil {
	// ...
}
u.Path = fmt.Sprintf("/posts/%d", 1)
req, err := http.NewRequestWithContext(ctx,
	http.MethodGet, u.String(), nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
err := json.Unmarshal(b, &post)
if err != nil {
	// ...
}
var post placeholder
err := requests.
	URL("https://jsonplaceholder.typicode.com").
	Pathf("/posts/%d", 1).
	ToJSON(&post).
	Fetch(ctx)
18+ lines7 lines

POST a JSON object and parse the response

var res placeholder
req := placeholder{
	Title:  "foo",
	Body:   "baz",
	UserID: 1,
}
err := requests.
	URL("/posts").
	Host("jsonplaceholder.typicode.com").
	BodyJSON(&req).
	ToJSON(&res).
	Fetch(ctx)
// net/http equivalent left as an exercise for the reader

Set custom headers for a request

// Set headers
var headers postman
err := requests.
	URL("https://postman-echo.com/get").
	UserAgent("bond/james-bond").
	ContentType("secret").
	Header("martini", "shaken").
	Fetch(ctx)

Easily manipulate URLs and query parameters

u, err := requests.
	URL("https://prod.example.com/get?a=1&b=2").
	Hostf("%s.example.com", "dev1").
	Param("b", "3").
	ParamInt("c", 4).
	URL()
if err != nil { /* ... */ }
fmt.Println(u.String()) // https://dev1.example.com/get?a=1&b=3&c=4

Record and replay responses

// record a request to the file system
var s1, s2 string
err := requests.URL("http://example.com").
	Transport(requests.Record(nil, "somedir")).
	ToString(&s1).
	Fetch(ctx)
check(err)

// now replay the request in tests
err = requests.URL("http://example.com").
	Transport(requests.Replay("somedir")).
	ToString(&s2).
	Fetch(ctx)
check(err)
assert(s1 == s2) // true

FAQs

See wiki for more details.

Why not just use the standard library HTTP client?

Brad Fitzpatrick, long time maintainer of the net/http package, wrote an extensive list of problems with the standard library HTTP client. His four main points (ignoring issues that can't be resolved by a wrapper around the standard library) are:

  • Too easy to not call Response.Body.Close.
  • Too easy to not check return status codes
  • Context support is oddly bolted on
  • Proper usage is too many lines of boilerplate

Requests solves these issues by always closing the response body, checking status codes by default, always requiring a context.Context, and simplifying the boilerplate with a descriptive UI based on fluent method chaining.

Why requests and not some other helper library?

There are two major flaws in other libraries as I see it. One is that in other libraries support for context.Context tends to be bolted on if it exists at all. Two, many hide the underlying http.Client in such a way that it is difficult or impossible to replace or mock out. Beyond that, I believe that none have achieved the same core simplicity that the requests library has.

How do I just get some JSON?

var data SomeDataType
err := requests.
	URL("https://example.com/my-json").
	ToJSON(&data).
	Fetch(ctx)

How do I post JSON and read the response JSON?

body := MyRequestType{}
var resp MyResponseType
err := requests.
	URL("https://example.com/my-json").
	BodyJSON(&body).
	ToJSON(&resp).
	Fetch(ctx)

How do I just save a file to disk?

It depends on exactly what you need in terms of file atomicity and buffering, but this will work for most cases:

err := requests.
	URL("http://example.com").
	ToFile("myfile.txt").
	Fetch(ctx)

For more advanced use case, use ToWriter.

How do I save a response to a string?

var s string
err := requests.
	URL("http://example.com").
	ToString(&s).
	Fetch(ctx)

How do I validate the response status?

By default, if no other validators are added to a builder, requests will check that the response is in the 2XX range. If you add another validator, you can add builder.CheckStatus(200) or builder.AddValidator(requests.DefaultValidator) to the validation stack.

To disable all response validation, run builder.AddValidator(nil).

Contributing

Please create a discussion before submitting a pull request for a new feature.

More Repositories

1

flowmatic

Structured concurrency made easy
Go
366
star
2

versioninfo

Importable package that parses version info from debug.ReadBuildInfo().
Go
248
star
3

heffalump

Heffalump is an endless honeypot
Go
199
star
4

pomodoro

Command line pomodoro timer
Go
173
star
5

be

The Go test helper for minimalists
Go
103
star
6

go-cli

Template for creating Go CLIs
Go
57
star
7

new

A helper function to create a pointer to a new object in Go 1.18+
Go
55
star
8

workgroup

Structured concurrency manager for Go
Go
46
star
9

certinfo

Get information about the certificate used at a domain
Go
34
star
10

truthy

Package truthy provides truthy condition testing with Go generics
Go
33
star
11

exembed

Go Embed experiments
Go
30
star
12

get-headers

Tool that shows headers and stats from GET-ing a URL
Go
28
star
13

deque

Generic deque container
Go
27
star
14

netlify-go-function-demo

https://blog.carlmjohnson.net/post/2020/how-to-host-golang-on-netlify-for-free/
HTML
26
star
15

shitpic

Recompresses JPEGs to make shitpics
JavaScript
25
star
16

resperr

Go package to associate status codes and messages with errors
Go
23
star
17

csv

Go CSV reader like Python's DictReader
Go
21
star
18

scattered

Command line tool for asset hashing
Go
20
star
19

tumblr-importr

An importer that uses the Tumblr API to create a Hugo static site
Go
20
star
20

decoder-ring

CLI tool for decoding/encoding from common formats
Go
16
star
21

springerle

A cookiecutter tool written in Go
Go
16
star
22

errorx

Error helpers for Go
Go
16
star
23

syncx

Go sync utility functions using generics
Go
14
star
24

bytemap

Bytemap contains types for making maps from bytes to bool, integer, or float using a backing array
Go
12
star
25

pointer

Generic pointer helpers
Go
11
star
26

opensesame

Trivial password generator
Go
11
star
27

exitcode

Go package to convert errors to exit codes
Go
10
star
28

flagext

Implementations of the flag.Value interface to extend the flag package
Go
9
star
29

go-run

Shebang line equivalent for Go
Go
9
star
30

json-tidy

Pretty prints JSON from stdin, files, or URLs
Go
9
star
31

tsrproxy

Simple Tailscale reverse proxy
Go
7
star
32

flagx

Extensions to the Go flag package
Go
7
star
33

collections

Go
6
star
34

portfor

Deterministic hash from name to a port number
Go
6
star
35

sudoku

Sudoku solver in Go
Go
6
star
36

simple-reverse-proxy

A simple reverse proxy written in Go
Go
5
star
37

monterey-jack

A friend to zippers
Go
4
star
38

haystack

Pinboard search CLI
Go
4
star
39

crockford

Go implementation of Crockford base 32 encoding
Go
4
star
40

go-utils

Implementation of the Fisherโ€“Yates shuffle (Knuth shuffle) in Go
Go
4
star
41

feed2json

Given an Atom or RSS feed, creates a comparable JSON feed
Go
3
star
42

whatsit

Looks at a file and guess its content type
Go
3
star
43

webarchive

Go
3
star
44

stringutil

Some Go string utilities
Go
3
star
45

gsize

Utility that tells how large a file will be after gzip compression
Go
3
star
46

kawatte

Recursive bulk find and replace
Go
3
star
47

bletchley

Simple command line application for basic public key crypto
Go
2
star
48

Watsuji-and-Aesthetics

Dissertation, Carl M. Johnson
HTML
2
star
49

errutil

Helpful tools for errors in Go
Go
2
star
50

randline

Chooses random line(s) from a file
Go
2
star
51

nfspampurge

Delete all spam from Netlify spam page
Go
2
star
52

richpaste

Paste as rich text in Mac OS
Go
2
star
53

walker

Walker, File Ranger
Go
2
star
54

slumber

Like Unix sleep but takes minutes, hours, etc.
Go
1
star
55

lich

A port of Wolf Rentzsch's Lich binary file format to Go (golang)
Go
1
star
56

rootdown

The internet demanded another Go router
Go
1
star
57

echo-request

Simple server that echoes requests back as text
Go
1
star
58

loggo

Logs requests to files for debugging webhooks
Go
1
star
59

luhn

Yet another Luhn algorithm implementation
Go
1
star
60

django-context-variables

Simple utility to help make declarative class-based views in Django
Python
1
star
61

junix

A re-implementation of common Unix commands with output as JSON instead of plain text
Go
1
star
62

rank-em

CLI tool for making ranking lists
Go
1
star
63

errors

Some Go error helpers
Go
1
star
64

jax

JavaScript for Automation scripts
AppleScript
1
star
65

python-defunct

Defunct. See https://github.com/carlmjohnson/python-tools instead.
Python
1
star
66

352-interactive-design-development

HTML
1
star
67

randpwd

Randomly generated passwords
HTML
1
star
68

slackhook

Simple client for Slack web hook URLs
Go
1
star
69

dotfiles-public

Public repo for configuration files (git, Sublime Text, etc.)
Python
1
star
70

gracefulserver

Boilerplate for starting an HTTP server in Go with graceful shutdown
Go
1
star
71

carljohnson-jour352

HTML
1
star
72

scooter

Scoot files into directories by date added
Go
1
star
73

carlana-redirect

1
star
74

jamtalk

Repo for SRCCON 2019 talk
HTML
1
star