• Stars
    star
    155
  • Rank 240,864 (Top 5 %)
  • Language
    Go
  • License
    GNU General Publi...
  • Created about 5 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Simple golang mitm proxy implementation

Code Coverage Go Report Card GolangCI Go Doc

gomitmproxy

This is a customizable HTTP proxy with TLS interception support. It was created as a part of AdGuard Home. However, it can be used for different purposes so we decided to make it a separate project.

Features

  • HTTP proxy
  • HTTP over TLS (HTTPS) proxy
  • Proxy authorization
  • TLS termination

How to use gomitmproxy

Simple HTTP proxy

package main

import (
	"log"
	"net"
	"os"
	"os/signal"
	"syscall"

	"github.com/AdguardTeam/gomitmproxy"
)

func main() {
	proxy := gomitmproxy.NewProxy(gomitmproxy.Config{
		ListenAddr: &net.TCPAddr{
			IP:   net.IPv4(0, 0, 0, 0),
			Port: 8080,
		},
	})
	err := proxy.Start()
	if err != nil {
		log.Fatal(err)
	}

	signalChannel := make(chan os.Signal, 1)
	signal.Notify(signalChannel, syscall.SIGINT, syscall.SIGTERM)
	<-signalChannel

	// Clean up
	proxy.Close()
}

Modifying requests and responses

You can modify requests and responses using OnRequest and OnResponse handlers.

The example below will block requests to example.net and add a short comment to the end of every HTML response.

proxy := gomitmproxy.NewProxy(gomitmproxy.Config{
    ListenAddr: &net.TCPAddr{
        IP:   net.IPv4(0, 0, 0, 0),
        Port: 8080,
    },
    OnRequest: func(session *gomitmproxy.Session) (request *http.Request, response *http.Response) {
        req := session.Request()

        log.Printf("onRequest: %s %s", req.Method, req.URL.String())

        if req.URL.Host == "example.net" {
            body := strings.NewReader("<html><body><h1>Replaced response</h1></body></html>")
            res := proxyutil.NewResponse(http.StatusOK, body, req)
            res.Header.Set("Content-Type", "text/html")

            // Use session props to pass the information about request being blocked
            session.SetProp("blocked", true)
            return nil, res
        }

        return nil, nil
    },
    OnResponse: func(session *gomitmproxy.Session) *http.Response {
        log.Printf("onResponse: %s", session.Request().URL.String())

        if _, ok := session.GetProp("blocked"); ok {
            log.Printf("onResponse: was blocked")
        }

        res := session.Response()
        req := session.Request()
    
        if strings.Index(res.Header.Get("Content-Type"), "text/html") != 0 {
            // Do nothing with non-HTML responses
            return nil
        }
    
        b, err := proxyutil.ReadDecompressedBody(res)
        // Close the original body
        _ = res.Body.Close()
        if err != nil {
            return proxyutil.NewErrorResponse(req, err)
        }
    
        // Use latin1 before modifying the body
        // Using this 1-byte encoding will let us preserve all original characters
        // regardless of what exactly is the encoding
        body, err := proxyutil.DecodeLatin1(bytes.NewReader(b))
        if err != nil {
            return proxyutil.NewErrorResponse(session.Request(), err)
        }
    
        // Modifying the original body
        modifiedBody, err := proxyutil.EncodeLatin1(body + "<!-- EDITED -->")
        if err != nil {
            return proxyutil.NewErrorResponse(session.Request(), err)
        }
    
        res.Body = ioutil.NopCloser(bytes.NewReader(modifiedBody))
        res.Header.Del("Content-Encoding")
        res.ContentLength = int64(len(modifiedBody))
        return res
    },
})

Proxy authorization

If you want to protect your proxy with Basic authentication, set Username and Password fields in the proxy configuration.

proxy := gomitmproxy.NewProxy(gomitmproxy.Config{
    ListenAddr: &net.TCPAddr{
        IP:   net.IPv4(0, 0, 0, 0),
        Port: 8080,
    },
    Username: "user",
    Password: "pass",
})

HTTP over TLS (HTTPS) proxy

If you want to protect yourself from eavesdropping on your traffic to proxy, you can configure it to work over a TLS tunnel. This is really simple to do, just set a *tls.Config instance in your proxy configuration.

tlsConfig := &tls.Config{
    Certificates: []tls.Certificate{*proxyCert},
}
proxy := gomitmproxy.NewProxy(gomitmproxy.Config{
    ListenAddr: addr,
    TLSConfig:  tlsConfig,
})

TLS interception

If you want to do TLS termination, you first need to prepare a self-signed certificate that will be used as a certificates authority. Use the following openssl commands to do this.

openssl genrsa -out demo.key 2048
openssl req -new -x509 -key demo.key -out demo.crt -days 3650 -addext subjectAltName=DNS:<hostname>,IP:<ip>

Now you can use it to initialize MITMConfig:

tlsCert, err := tls.LoadX509KeyPair("demo.crt", "demo.key")
if err != nil {
    log.Fatal(err)
}
privateKey := tlsCert.PrivateKey.(*rsa.PrivateKey)

x509c, err := x509.ParseCertificate(tlsCert.Certificate[0])
if err != nil {
    log.Fatal(err)
}

mitmConfig, err := mitm.NewConfig(x509c, privateKey, nil)
if err != nil {
    log.Fatal(err)
}

mitmConfig.SetValidity(time.Hour * 24 * 7) // generate certs valid for 7 days
mitmConfig.SetOrganization("gomitmproxy")  // cert organization

Please note that you can set MITMExceptions to a list of hostnames, which will be excluded from TLS interception.

proxy := gomitmproxy.NewProxy(gomitmproxy.Config{
    ListenAddr: &net.TCPAddr{
        IP:   net.IPv4(0, 0, 0, 0),
        Port: 3333,
    },
    MITMConfig:     mitmConfig,
    MITMExceptions: []string{"example.com"},
})

If you configure the APIHost, you'll be able to download the CA certificate from http://[APIHost]/cert.crt when the proxy is configured.

// Navigate to http://gomitmproxy/cert.crt to download the CA certificate
proxy.APIHost = "gomitmproxy"

Custom certs storage

By default, gomitmproxy uses an in-memory map-based storage for the certificates, generated while doing TLS interception. It is often necessary to use a different kind of certificates storage. If this is your case, you can supply your own implementation of the CertsStorage interface.

// CustomCertsStorage - an example of a custom cert storage
type CustomCertsStorage struct {
	certsCache map[string]*tls.Certificate // cache with the generated certificates
}

// Get gets the certificate from the storage
func (c *CustomCertsStorage) Get(key string) (*tls.Certificate, bool) {
	v, ok := c.certsCache[key]
	return v, ok
}

// Set saves the certificate to the storage
func (c *CustomCertsStorage) Set(key string, cert *tls.Certificate) {
	c.certsCache[key] = cert
}

Then pass it to the NewConfig function.

mitmConfig, err := mitm.NewConfig(x509c, privateKey, &CustomCertsStorage{
    certsCache: map[string]*tls.Certificate{}},
)

Notable alternatives

  • martian - an awesome debugging proxy with TLS interception support.
  • goproxy - also supports TLS interception and requests.

TODO

  • Basic HTTP proxy without MITM
  • Proxy
    • Expose APIs for the library users
    • How-to doc
    • Travis configuration
    • Proxy-Authorization
    • WebSockets support (see this)
    • certsCache -- allow custom implementations
    • Support HTTP CONNECT over TLS
    • Test plain HTTP requests inside HTTP CONNECT
    • Test memory leaks
    • Editing response body in a callback
    • Handle unknown content-encoding values
    • Handle CONNECT to APIHost properly (without trying to actually connect anywhere)
    • Allow hijacking connections (!)
    • Multiple listeners
    • Unit tests
    • Check & fix TODOs
    • Allow specifying net.Dialer
    • Specify timeouts for http.Transport
  • MITM
    • Basic MITM
    • MITM exceptions
    • Handle invalid server certificates properly (not just reset connections)
    • Pass the most important tests on badssl.com/dashboard
    • Handle certificate authentication
    • Allow configuring minimum supported TLS version
    • OCSP check (see example)
    • (?) HPKP (see example)
    • (?) CT logs (see example)
    • (?) CRLSets (see example)

More Repositories

1

AdGuardHome

Network-wide ads & trackers blocking DNS server
Go
18,381
star
2

AdguardBrowserExtension

AdGuard browser extension
JavaScript
2,427
star
3

AdguardFilters

AdGuard Content Blocking Filters
Adblock Filter List
2,331
star
4

dnsproxy

Simple DNS proxy with DoH, DoT, DoQ and DNSCrypt support
Go
1,791
star
5

AdguardForiOS

The most advanced ad blocker for iOS
JavaScript
1,289
star
6

AdguardForAndroid

Open bug tracker for Android version of AdGuard.
1,060
star
7

AdGuardForSafari

AdGuard for Safari app extension
JavaScript
875
star
8

AdGuardDNS

Public DNS resolver that protects you from ad trackers
Go
634
star
9

AdguardForWindows

AdGuard for Windows open bug tracker
568
star
10

AdGuardSDNSFilter

AdGuard Simplified Domain names filter
Adblock Filter List
491
star
11

ContentBlocker

Content blocking extension for Samsung Internet and Yandex Browser
Java
402
star
12

cname-trackers

This repository contains a list of popular CNAME trackers
JavaScript
334
star
13

AdGuardExtra

AdGuard Extra is designed to solve complicated cases when regular ad blocking rules aren't enough.
270
star
14

AdguardForMac

Open bug tracker for Mac version of AdGuard
269
star
15

PopupBlocker

Popup blocking userscript
TypeScript
220
star
16

FiltersRegistry

Known filters subscriptions transformed for better compatibility with AdGuard
Adblock Filter List
152
star
17

AdguardKnowledgeBase

Adguard Knowledge Base
140
star
18

AdGuardMV3

AdGuard browser extension prototype based on the new Manifest V3
TypeScript
121
star
19

HostlistsRegistry

Known hosts blocklists that are made available to the users of AdGuard products
JavaScript
113
star
20

AdguardAssistant

Adguard Assistant userscript
JavaScript
110
star
21

Scriptlets

AdGuard scriptlets library
JavaScript
109
star
22

HostlistCompiler

A simple tool that compiles hosts blocklists from multiple sources
JavaScript
95
star
23

BlockYouTubeAdsShortcut

This repo contains the code for blocking YouTube ads that is supposed to be run by an iOS shortcut
JavaScript
85
star
24

AdGuardVPNForAndroid

AdGuard VPN Android app open bug tracker
81
star
25

DisableAMP

Disable AMP userscript
JavaScript
76
star
26

DnsLibs

DNS filtering library
C++
75
star
27

urlfilter

AdGuard content blocking library in golang
Go
71
star
28

AdGuardVPNExtension

AdGuard VPN Chrome and Firefox extension
TypeScript
57
star
29

AnonymousRedirect

Very simple HTML page that is used for anonymous redirect
HTML
55
star
30

ExtendedCss

A TypeScript library for non-standard element selecting — :contains(), :matches-css(), etc., and applying CSS styles with extended properties.
TypeScript
54
star
31

HttpsExclusions

Centralized repo for HTTPS exclusions
JavaScript
48
star
32

KnowledgeBaseDNS

AdGuard DNS knowledge base
JavaScript
47
star
33

StealthMode

JavaScript
46
star
34

Userscripts

Userscripts made by our team
JavaScript
42
star
35

AdGuardVPNForWindows

AdGuard VPN Windows app open bug tracker
39
star
36

VscodeAdblockSyntax

TM language plugin with ad blocking rules syntax
37
star
37

FiltersCompiler

A tool that compiles & validates filters
JavaScript
36
star
38

tsurlfilter

AdGuard content blocking library
TypeScript
35
star
39

CoreLibs

Core Adguard libraries
33
star
40

BrowserAssistant

AdGuard Browser Assistant
JavaScript
33
star
41

AGLint

Universal adblock filter list parser, linter and converter
TypeScript
28
star
42

KnowledgeBase

AdGuard knowledge base
JavaScript
26
star
43

AdGuardVPNForiOS

AdGuard VPN iOS app open bug tracker
25
star
44

SafariConverterLib

Swift library that converts AdGuard rules to Safari content blocking rules
Swift
21
star
45

AdGuardVPNForMac

AdGuard VPN Mac app open bug tracker
21
star
46

AdguardTeam.github.io

HTML
17
star
47

companiesdb

This is a companies DB that we use in AdGuard Home and AdGuard DNS.
JavaScript
16
star
48

ProxiesSetup

A simple script that sets up an HTTP and a SOCKS5 proxy (squid and danted)
Shell
16
star
49

WebBatteryTester

This application allows you to test how quick is your battery drained by web-browsing.
Java
14
star
50

golibs

Small helper Go libraries
Go
13
star
51

SafariConverter

Converter: ad blocking rules -> safari content blocker
JavaScript
12
star
52

deep-override

TypeScript
10
star
53

VerificationLibrary

Certificates verification library
C
9
star
54

go-webext

Automation for working with extension stores
Go
9
star
55

FiltersDownloader

Pre-processing library for filters subscriptions
JavaScript
9
star
56

translate

Simple internationalization library with react integration
TypeScript
8
star
57

AdGuardFiltersStats

The repo where we collect AdguardFilters statistics
JavaScript
8
star
58

ecsstree

Adblock Extended CSS supplement for CSSTree
JavaScript
7
star
59

ReportsWebApp

Allows users to report a problem with Adguard filters
JavaScript
7
star
60

NativeLibsCommon

Native libs common
C++
6
star
61

Recovery

JavaScript
5
star
62

VpnLibs

Open bug tracker for AdGuard VPN core library
5
star
63

KnowledgeBaseVPN

AdGuard VPN knowledge base
JavaScript
5
star
64

AdGuardVpnKnowledgeBase

AdGuard VPN Knowledge Base
5
star
65

LegalDocs

AdGuard's legal documents
4
star
66

SafariContentBlockerTester

Utility Safari extension for content blocker format testing.
JavaScript
4
star
67

TestCases

Used for testing puproses
JavaScript
3
star
68

SafariContentBlockerConverterCompiler

Utility scripts for building safari content blocker converter.
JavaScript
3
star
69

github-stats

Tool for calculating AdguardFilters contributors statistics
JavaScript
3
star
70

DeadDomainsLinter

Simple tool to check adblock filtering rules for dead domains.
JavaScript
2
star
71

AdGuardVPNCLI

AdGuard VPN command-line version
2
star
72

AdGuardDNSClient

Client tool for AdGuard DNS
Go
2
star
73

closure-tools-helper

TypeScript
1
star
74

dns-sde-extension

Proof-of-concept browser extension for Structured DNS errors
1
star