• Stars
    star
    3,904
  • Rank 10,667 (Top 0.3 %)
  • Language
    Go
  • License
    MIT License
  • Created about 7 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Simple Go HTTP client with Black Magic

req

Simple Go HTTP client with Black Magic

Build Status Go Report Card License GitHub Releases Mentioned in Awesome Go

Documentation

Full documentation is available on the official website: https://req.cool.

  • Simple and Powerful: Simple and easy to use, providing rich client-level and request-level settings, all of which are intuitive and chainable methods.
  • Easy Debugging: Powerful and convenient debug utilities, including debug logs, performance traces, and even dump the complete request and response content (see Debugging).
  • Easy API Testing: API testing can be done with minimal code, no need to explicitly create any Request or Client, or even to handle errors (See Quick HTTP Test)
  • Smart by Default: Detect and decode to utf-8 automatically if possible to avoid garbled characters (See Auto Decode), marshal request body and unmarshal response body automatically according to the Content-Type.
  • Support Multiple HTTP Versions: Support HTTP/1.1, HTTP/2, and HTTP/3, and can automatically detect the server side and select the optimal HTTP version for requests, you can also force the protocol if you want (See Force HTTP version).
  • Support Retry: Support automatic request retry and is fully customizable (See Retry).
  • HTTP Fingerprinting: Support http fingerprint impersonation, so that we can access websites that prohibit crawler programs by identifying http fingerprints (See HTTP Fingerprint).
  • Multiple Authentication Methods: You can use HTTP Basic Auth, Bearer Auth Token and Digest Auth out of box (see Authentication).
  • Easy Download and Upload: You can download and upload files with simple request settings, and even set a callback to show real-time progress (See Download and Upload).
  • Exportable: req.Transport is exportable. Compared with http.Transport, it also supports HTTP3, dump content, middleware, etc. It can directly replace the Transport of http.Client in existing projects, and obtain more powerful functions with minimal code change.
  • Extensible: Support Middleware for Request, Response, Client and Transport (See Request and Response Middleware) and Client and Transport Middleware).

Install

You first need Go installed (version 1.20+ is required), then you can use the below Go command to install req:

go get github.com/imroc/req/v3

Import

Import req to your code:

import "github.com/imroc/req/v3"

Basic Usage

# assume the following codes in main.go file
$ cat main.go
package main

import (
    "github.com/imroc/req/v3"
)

func main() {
    req.DevMode() // Treat the package name as a Client, enable development mode
    req.MustGet("https://httpbin.org/uuid") // Treat the package name as a Request, send GET request.

    req.EnableForceHTTP1() // Force using HTTP/1.1
    req.MustGet("https://httpbin.org/uuid")
}
$ go run main.go
2022/05/19 10:05:07.920113 DEBUG [req] HTTP/2 GET https://httpbin.org/uuid
:authority: httpbin.org
:method: GET
:path: /uuid
:scheme: https
user-agent: req/v3 (https://github.com/imroc/req/v3)
accept-encoding: gzip

:status: 200
date: Thu, 19 May 2022 02:05:08 GMT
content-type: application/json
content-length: 53
server: gunicorn/19.9.0
access-control-allow-origin: *
access-control-allow-credentials: true

{
  "uuid": "bd519208-35d1-4483-ad9f-e1555ae108ba"
}

2022/05/19 10:05:09.340974 DEBUG [req] HTTP/1.1 GET https://httpbin.org/uuid
GET /uuid HTTP/1.1
Host: httpbin.org
User-Agent: req/v3 (https://github.com/imroc/req/v3)
Accept-Encoding: gzip

HTTP/1.1 200 OK
Date: Thu, 19 May 2022 02:05:09 GMT
Content-Type: application/json
Content-Length: 53
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "uuid": "49b7f916-c6f3-49d4-a6d4-22ae93b71969"
}

The sample code above is good for quick testing purposes, which use DevMode() to see request details, and send requests using global wrapper methods that use the default client behind the scenes to initiate the request.

In production, it is recommended to explicitly create a client, and then use the same client to send all requests, please see other examples below.

Videos

The following is a series of video tutorials for req:

More

Check more introduction, tutorials, examples, best practices and API references on the official website.

package main

import (
	"fmt"
	"github.com/imroc/req/v3"
	"log"
)

func main() {
	client := req.C() // Use C() to create a client.
	resp, err := client.R(). // Use R() to create a request.
		Get("https://httpbin.org/uuid")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(resp)
}
{
  "uuid": "a4d4430d-0e5f-412f-88f5-722d84bc2a62"
}
package main

import (
  "fmt"
  "github.com/imroc/req/v3"
  "log"
  "time"
)

type ErrorMessage struct {
  Message string `json:"message"`
}

type UserInfo struct {
  Name string `json:"name"`
  Blog string `json:"blog"`
}

func main() {
  client := req.C().
    SetUserAgent("my-custom-client"). // Chainable client settings.
    SetTimeout(5 * time.Second)

  var userInfo UserInfo
  var errMsg ErrorMessage
  resp, err := client.R().
    SetHeader("Accept", "application/vnd.github.v3+json"). // Chainable request settings.
    SetPathParam("username", "imroc"). // Replace path variable in url.
    SetSuccessResult(&userInfo). // Unmarshal response body into userInfo automatically if status code is between 200 and 299.
    SetErrorResult(&errMsg). // Unmarshal response body into errMsg automatically if status code >= 400.
    EnableDump(). // Enable dump at request level, only print dump content if there is an error or some unknown situation occurs to help troubleshoot.
    Get("https://api.github.com/users/{username}")

  if err != nil { // Error handling.
    log.Println("error:", err)
    log.Println("raw content:")
    log.Println(resp.Dump()) // Record raw content when error occurs.
    return
  }

  if resp.IsErrorState() { // Status code >= 400.
    fmt.Println(errMsg.Message) // Record error message returned.
    return
  }

  if resp.IsSuccessState() { // Status code is between 200 and 299.
    fmt.Printf("%s (%s)\n", userInfo.Name, userInfo.Blog)
    return
  }

  // Unknown status code.
  log.Println("unknown status", resp.Status)
  log.Println("raw content:")
  log.Println(resp.Dump()) // Record raw content when server returned unknown status code.
}

Normally it will output (SuccessState):

roc (https://imroc.cc)

You can set up a unified logic for error handling on the client, so that each time you send a request you only need to focus on the success situation, reducing duplicate code.

package main

import (
	"fmt"
	"github.com/imroc/req/v3"
	"log"
	"time"
)

type ErrorMessage struct {
	Message string `json:"message"`
}

func (msg *ErrorMessage) Error() string {
	return fmt.Sprintf("API Error: %s", msg.Message)
}

type UserInfo struct {
	Name string `json:"name"`
	Blog string `json:"blog"`
}

var client = req.C().
	SetUserAgent("my-custom-client"). // Chainable client settings.
	SetTimeout(5 * time.Second).
	EnableDumpEachRequest().
	SetCommonErrorResult(&ErrorMessage{}).
	OnAfterResponse(func(client *req.Client, resp *req.Response) error {
		if resp.Err != nil { // There is an underlying error, e.g. network error or unmarshal error.
			return nil
		}
		if errMsg, ok := resp.ErrorResult().(*ErrorMessage); ok {
			resp.Err = errMsg // Convert api error into go error
			return nil
		}
		if !resp.IsSuccessState() {
			// Neither a success response nor a error response, record details to help troubleshooting
			resp.Err = fmt.Errorf("bad status: %s\nraw content:\n%s", resp.Status, resp.Dump())
		}
		return nil
	})

func main() {
	var userInfo UserInfo
	resp, err := client.R().
		SetHeader("Accept", "application/vnd.github.v3+json"). // Chainable request settings
		SetPathParam("username", "imroc").
		SetSuccessResult(&userInfo). // Unmarshal response body into userInfo automatically if status code is between 200 and 299.
		Get("https://api.github.com/users/{username}")

	if err != nil { // Error handling.
		log.Println("error:", err)
		return
	}

	if resp.IsSuccessState() { // Status code is between 200 and 299.
		fmt.Printf("%s (%s)\n", userInfo.Name, userInfo.Blog)
	}
}
package main

import (
  "fmt"
  "github.com/imroc/req/v3"
  "log"
)

type Repo struct {
  Name string `json:"name"`
  Url  string `json:"url"`
}

type Result struct {
  Data string `json:"data"`
}

func main() {
  client := req.C().DevMode()
  var result Result

  resp, err := client.R().
    SetBody(&Repo{Name: "req", Url: "https://github.com/imroc/req"}).
    SetSuccessResult(&result).
    Post("https://httpbin.org/post")
  if err != nil {
    log.Fatal(err)
  }

  if !resp.IsSuccessState() {
    fmt.Println("bad response status:", resp.Status)
    return
  }
  fmt.Println("++++++++++++++++++++++++++++++++++++++++++++++++")
  fmt.Println("data:", result.Data)
  fmt.Println("++++++++++++++++++++++++++++++++++++++++++++++++")
}
2022/05/19 20:11:00.151171 DEBUG [req] HTTP/2 POST https://httpbin.org/post
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
user-agent: req/v3 (https://github.com/imroc/req/v3)
content-type: application/json; charset=utf-8
content-length: 55
accept-encoding: gzip

{"name":"req","website":"https://github.com/imroc/req"}

:status: 200
date: Thu, 19 May 2022 12:11:00 GMT
content-type: application/json
content-length: 651
server: gunicorn/19.9.0
access-control-allow-origin: *
access-control-allow-credentials: true

{
  "args": {},
  "data": "{\"name\":\"req\",\"website\":\"https://github.com/imroc/req\"}",
  "files": {},
  "form": {},
  "headers": {
    "Accept-Encoding": "gzip",
    "Content-Length": "55",
    "Content-Type": "application/json; charset=utf-8",
    "Host": "httpbin.org",
    "User-Agent": "req/v3 (https://github.com/imroc/req/v3)",
    "X-Amzn-Trace-Id": "Root=1-628633d4-7559d633152b4307288ead2e"
  },
  "json": {
    "name": "req",
    "website": "https://github.com/imroc/req"
  },
  "origin": "103.7.29.30",
  "url": "https://httpbin.org/post"
}

++++++++++++++++++++++++++++++++++++++++++++++++
data: {"name":"req","url":"https://github.com/imroc/req"}
++++++++++++++++++++++++++++++++++++++++++++++++

If you like, you can also use a Do API style like the following to make requests:

package main

import (
	"fmt"
	"github.com/imroc/req/v3"
)

type APIResponse struct {
	Origin string `json:"origin"`
	Url    string `json:"url"`
}

func main() {
	var resp APIResponse
	c := req.C().SetBaseURL("https://httpbin.org/post")
	err := c.Post().
		SetBody("hello").
		Do().
		Into(&resp)
	if err != nil {
		panic(err)
	}
	fmt.Println("My IP is", resp.Origin)
}
My IP is 182.138.155.113
  • The order of chain calls is more intuitive: first call Client to create a request with a specified Method, then use chain calls to set the request, then use Do() to fire the request, return Response, and finally call Response.Into to unmarshal response body into specified object.
  • Response.Into will return an error if an error occurs during sending the request or during unmarshalling.
  • The url of some APIs is fixed, and different types of requests are implemented by passing different bodies. In this scenario, Client.SetBaseURL can be used to set a unified url, and there is no need to set the url for each request when initiating a request. Of course, you can also call Request.SetURL to set it if you need it.

Here is an example of building GitHub's SDK with req, using two styles (GetUserProfile_Style1, GetUserProfile_Style2).

import (
	"context"
	"fmt"
	"github.com/imroc/req/v3"
)

type ErrorMessage struct {
	Message string `json:"message"`
}

// Error implements go error interface.
func (msg *ErrorMessage) Error() string {
	return fmt.Sprintf("API Error: %s", msg.Message)
}

type GithubClient struct {
	*req.Client
}

func NewGithubClient() *GithubClient {
	return &GithubClient{
		Client: req.C().
			SetBaseURL("https://api.github.com").
			SetCommonErrorResult(&ErrorMessage{}).
			EnableDumpEachRequest().
			OnAfterResponse(func(client *req.Client, resp *req.Response) error {
				if resp.Err != nil { // There is an underlying error, e.g. network error or unmarshal error.
					return nil
				}
				if errMsg, ok := resp.ErrorResult().(*ErrorMessage); ok {
					resp.Err = errMsg // Convert api error into go error
					return nil
				}
				if !resp.IsSuccessState() {
					// Neither a success response nor a error response, record details to help troubleshooting
					resp.Err = fmt.Errorf("bad status: %s\nraw content:\n%s", resp.Status, resp.Dump())
				}
				return nil
			}),
	}
}

type UserProfile struct {
	Name string `json:"name"`
	Blog string `json:"blog"`
}

// GetUserProfile_Style1 returns the user profile for the specified user.
// Github API doc: https://docs.github.com/en/rest/users/users#get-a-user
func (c *GithubClient) GetUserProfile_Style1(ctx context.Context, username string) (user *UserProfile, err error) {
	_, err = c.R().
		SetContext(ctx).
		SetPathParam("username", username).
		SetSuccessResult(&user).
		Get("/users/{username}")
	return
}

// GetUserProfile_Style2 returns the user profile for the specified user.
// Github API doc: https://docs.github.com/en/rest/users/users#get-a-user
func (c *GithubClient) GetUserProfile_Style2(ctx context.Context, username string) (user *UserProfile, err error) {
	err = c.Get("/users/{username}").
		SetPathParam("username", username).
		Do(ctx).
		Into(&user)
	return
}

Contributing

If you have a bug report or feature request, you can open an issue, and pull requests are also welcome.

Contact

If you have questions, feel free to reach out to us in the following ways:

Sponsors

If you like req and it really helps you, feel free to reward me with a cup of coffee, and don't forget to mention your github id.


Wechat

Alipay

Many thanks to the following sponsors:


M-Cosmosss 🥇

aadog 🥈

Req released under MIT license, refer LICENSE file.

More Repositories

1

kubernetes-guide

Kubernetes 实践指南
TypeScript
271
star
2

biu

a binary util of Go
Go
176
star
3

kubernetes-practice-guide

Kubernetes 实践指南(已过时,最新请看 https://imroc.cc/kubernetes/)
Shell
176
star
4

ontts

科大讯飞语音linux在线语音合成后台服务
Go
167
star
5

imgo

imgo is a push server written by golang based on goim
Go
105
star
6

istio-guide

istio 实践指南
TypeScript
72
star
7

learning-kubernetes

Kubernetes 学习笔记
49
star
8

cert-manager-webhook-dnspod

cert-manager webhook resolver for DNSPod
Go
41
star
9

log

simple and useful logging tool of Go
Go
24
star
10

zk2etcd

zk2etcd 是一款同步 zookeeper 数据到 etcd 的工具
Go
24
star
11

learning-istio

记录 istio 相关知识与实践
14
star
12

scifi-page

personal web page for science fiction effect,very cool!
JavaScript
11
star
13

helm-cos

Helm plugin for manage helm repositories on Tencent COS
Go
8
star
14

tls

OpenSSL bindings for Go, compatible with crypto/tls,faster and more features.
Go
8
star
15

kubernetes-the-gradual-way

learn kubernetes from the primer to the mastery
Shell
5
star
16

airflow

Run Apache Airflow on kubernetes
Python
5
star
17

sh

sh is a light weight tool for using golang to execute shell script.
Go
5
star
18

imgo-java-sdk

java sdk for imgo
Java
4
star
19

tclient

灵活的腾讯云SDK
Go
4
star
20

helm-push

a helm plugin that allows you to push chart package to TencentHub
Go
4
star
21

kafka-monitor

docker image for KafkaOffsetMonitor
3
star
22

imroc.github.io

personal web page for roc chan
HTML
3
star
23

wechatbot

tuling wechat bot
Python
3
star
24

xhugo

A Extremly beautiful theme for Hugo
CSS
3
star
25

domain

a domain parser, like ruby's domainatrix
Go
3
star
26

httpcli

httcli is a http client library, making http request much easier
Go
3
star
27

md5x

golang md5 utility
Go
3
star
28

learning-tke

记录腾讯云容器服务相关知识与实践经验
3
star
29

webot

企业微信机器人SDK
Go
2
star
30

vim-go-ide

Fully vim configuration for best practice of Go development,and very easy to use.
Vim Script
2
star
31

k8s_baiduyun_uploader

下载kubernetes相关文件并上传到百度云
PHP
2
star
32

blog

my personal blog
2
star
33

conv

a golang util for convert data to the type you want
Go
2
star
34

system

common system call across different os with single API (open link in brower, etc)
Go
2
star
35

debugk8s

Go
2
star
36

learning-docker

分享 docker 相关知识与实践
Shell
2
star
37

imroc.cc

my personal website
TypeScript
2
star
38

kube-monitoring

monitoring stack for kubernetes
1
star
39

gitcli

git client abstraction for github,gitlab...
1
star
40

req-website

req's website
HTML
1
star
41

cyd

Shell
1
star
42

kubernetes-manifest

my kuberntes manifest
Makefile
1
star
43

knative-deploy

One-click deployment for knative
Makefile
1
star
44

ooui.js

Object Oriented web User Interface,making web more powerful
JavaScript
1
star
45

kube-apps

Deploy well known apps on kubernetes with best practices and less configuration
Smarty
1
star
46

scifi

my personal science fiction web page
1
star
47

maven-docker-example

Example repository of java maven project, build and run in container environment.
Java
1
star
48

assets

static resources
Shell
1
star
49

cos-go-sdk-v5

Tencent COS v5 SDK for Go
Go
1
star
50

beautifulhugo

Theme for the Hugo
HTML
1
star
51

learning-network

网络学习笔记
1
star
52

tke-guide

TKE 实践指南 (https://imroc.cc/tke)
TypeScript
1
star
53

docker-debian

useful debian docker images
1
star
54

types

handy type conversion tool based on reflection
Go
1
star
55

docker-golang

docker image for golnag
Go
1
star
56

istio-test

Go
1
star
57

helm-values

Shell
1
star
58

helm-charts

useful helm charts
Smarty
1
star
59

tech-note

My technical notes
TypeScript
1
star
60

kubernetes-troubleshooting-guide

Kubernetes 排错指南
Makefile
1
star
61

image-porter

A tool for syncing container images
Go
1
star
62

grpc-demo

Go
1
star