• Stars
    star
    686
  • Rank 65,892 (Top 2 %)
  • Language
    Go
  • License
    MIT License
  • Created over 7 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language.

cdp

Build Status Coverage Status Go Report Card GoDoc

Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language. The bindings are generated (by cdpgen) from the latest tip-of-tree (tot) protocol definitions and are mainly intended for use with Google Chrome or Chromium, however, they can be used with any debug target (Node.js, Edge DevTools Protocol, Safari, etc.) that implement the protocol.

This package can be used for any kind of browser automation, scripting or debugging via the Chrome DevTools Protocol.

A big motivation for cdp is to expose the full functionality of the Chrome DevTools Protocol and provide it in a discoverable and self-documenting manner.

Providing high-level browser automation is a non-goal for this project. That being said, cdp hopes to improve the ergonomics of working with the protocol by providing primitives better suited for Go and automating repetitive tasks.

Features

  • Discoverable API for the Chrome DevTools Protocol (GoDoc, autocomplete friendly)
  • Contexts as a first-class citizen (for timeouts and cancellation)
  • Simple and synchronous event handling (no callbacks)
  • Concurrently safe
  • No silent or hidden errors
  • Do what the user expects
  • Match CDP types to Go types wherever possible
  • Separation of concerns (avoid mixing CDP and RPC)

Installation

$ go get -u github.com/mafredri/cdp

Documentation

See API documentation for package, API descriptions and examples. Examples can also be found in this repository, see the simple, advanced, logging and incognito examples.

Usage

The main packages are cdp and rpcc, the former provides the CDP bindings and the latter handles the RPC communication with the debugging target.

To connect to a debug target, a WebSocket debugger URL is needed. For example, if Chrome is running with --remote-debugging-port=9222 the debugger URL can be found at localhost:9222/json. The devtool package can also be used to query the DevTools JSON API (see example below).

Here is an example of using cdp:

package main

import (
	"bufio"
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"time"

	"github.com/mafredri/cdp"
	"github.com/mafredri/cdp/devtool"
	"github.com/mafredri/cdp/protocol/dom"
	"github.com/mafredri/cdp/protocol/page"
	"github.com/mafredri/cdp/rpcc"
)

func main() {
	err := run(5 * time.Second)
	if err != nil {
		log.Fatal(err)
	}
}

func run(timeout time.Duration) error {
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	// Use the DevTools HTTP/JSON API to manage targets (e.g. pages, webworkers).
	devt := devtool.New("http://127.0.0.1:9222")
	pt, err := devt.Get(ctx, devtool.Page)
	if err != nil {
		pt, err = devt.Create(ctx)
		if err != nil {
			return err
		}
	}

	// Initiate a new RPC connection to the Chrome DevTools Protocol target.
	conn, err := rpcc.DialContext(ctx, pt.WebSocketDebuggerURL)
	if err != nil {
		return err
	}
	defer conn.Close() // Leaving connections open will leak memory.

	c := cdp.NewClient(conn)

	// Open a DOMContentEventFired client to buffer this event.
	domContent, err := c.Page.DOMContentEventFired(ctx)
	if err != nil {
		return err
	}
	defer domContent.Close()

	// Enable events on the Page domain, it's often preferrable to create
	// event clients before enabling events so that we don't miss any.
	if err = c.Page.Enable(ctx); err != nil {
		return err
	}

	// Create the Navigate arguments with the optional Referrer field set.
	navArgs := page.NewNavigateArgs("https://www.google.com").
		SetReferrer("https://duckduckgo.com")
	nav, err := c.Page.Navigate(ctx, navArgs)
	if err != nil {
		return err
	}

	// Wait until we have a DOMContentEventFired event.
	if _, err = domContent.Recv(); err != nil {
		return err
	}

	fmt.Printf("Page loaded with frame ID: %s\n", nav.FrameID)

	// Fetch the document root node. We can pass nil here
	// since this method only takes optional arguments.
	doc, err := c.DOM.GetDocument(ctx, nil)
	if err != nil {
		return err
	}

	// Get the outer HTML for the page.
	result, err := c.DOM.GetOuterHTML(ctx, &dom.GetOuterHTMLArgs{
		NodeID: &doc.Root.NodeID,
	})
	if err != nil {
		return err
	}

	fmt.Printf("HTML: %s\n", result.OuterHTML)

	// Capture a screenshot of the current page.
	screenshotName := "screenshot.jpg"
	screenshotArgs := page.NewCaptureScreenshotArgs().
		SetFormat("jpeg").
		SetQuality(80)
	screenshot, err := c.Page.CaptureScreenshot(ctx, screenshotArgs)
	if err != nil {
		return err
	}
	if err = ioutil.WriteFile(screenshotName, screenshot.Data, 0o644); err != nil {
		return err
	}

	fmt.Printf("Saved screenshot: %s\n", screenshotName)

	pdfName := "page.pdf"
	f, err := os.Create(pdfName)
	if err != nil {
		return err
	}

	pdfArgs := page.NewPrintToPDFArgs().
		SetTransferMode("ReturnAsStream") // Request stream.
	pdfData, err := c.Page.PrintToPDF(ctx, pdfArgs)
	if err != nil {
		return err
	}

	sr := c.NewIOStreamReader(ctx, *pdfData.Stream)
	r := bufio.NewReader(sr)

	// Write to file in ~r.Size() chunks.
	_, err = r.WriteTo(f)
	if err != nil {
		return err
	}

	err = f.Close()
	if err != nil {
		return err
	}

	fmt.Printf("Saved PDF: %s\n", pdfName)

	return nil
}

For more information, consult the documentation.

Acknowledgements

The Go implementation of gRPC (grpc-go) has been a source of inspiration for some of the design decisions made in the cdp and rpcc packages. Some ideas have also been borrowed from the net/rpc package from the standard library.

Resources

More Repositories

1

zsh-async

Because your terminal should be able to perform tasks asynchronously without external tools!
Shell
710
star
2

vyatta-wireguard-installer

Install, upgrade or remove WireGuard (WireGuard/wireguard-vyatta-ubnt) on Ubiquiti hardware
Shell
142
star
3

phoenix-config

My personal Phoenix (kasper/phoenix) configuration, written in TypeScript
TypeScript
38
star
4

phoenix-typings

TypeScript typings for Phoenix (kasper/phoenix)
29
star
5

macos-display-overrides

Display overrides to make (my) monitors work better on macOS
Ruby
22
star
6

go-trueskill

An implementation of the TrueSkillâ„¢ ranking system (by Microsoft) in Go
Go
22
star
7

macos-brightness

Simple CLI for changing brightness on macOS
Go
20
star
8

asustor-platform-driver

Linux kernel platform driver for ASUSTOR NAS hardware (leds, buttons)
C
19
star
9

zeet

My personal Zsh setup
Shell
16
star
10

openbttn

Open source firmware for bt.tn buttons
C
16
star
11

asustor_as-6xxt

All things ASUSTOR AS-6XXT sans ADM
Shell
13
star
12

asustor-packages

Collection of packages for ASUSTOR ADM
Shell
9
star
13

lcm

Implementation of the ASUSTOR NAS LCD serial port communication protocol
Go
8
star
14

asdev

ASUSTOR development tool, upload apps, etc.
Go
7
star
15

safari-bookmarklets

Bookmarklets for Safari (macOS / iOS)
JavaScript
7
star
16

gpg-notify

Notify about GPG events (pinentry, smartcard)
Go
6
star
17

fineli-sql

Convert Fineli, the Finnish Food Composition Database, into sqlite3 / SQL
Shell
5
star
18

macos-darkmode

Simple CLI for controlling macOS Dark Mode via Apples private API
Go
5
star
19

debian-my-live-build

Set up a custom debian live image (CD / USB)
Shell
5
star
20

goodspeaker

This package speaks the LG speaker wire protocol which is used by both LG Music Flow Player and LG Wi-Fi Speaker
Go
4
star
21

asustor-radarr

Radarr for ASUSTOR ADM
Shell
4
star
22

magic4linux

Allows you to use the magic remote on your webOS LG TV as a keyboard/mouse for your Linux machine
Go
4
star
23

asustor-deluge

Build Deluge package for ASUSTOR ADM
Shell
4
star
24

asustor-libcec

Shell
3
star
25

go-gaussian

Normal (gaussian) distribution for Go
Go
3
star
26

musicflow

This package and its commands can be used to control LG speakers that use the Music Flow app
Go
3
star
27

kodi-watchedlist-sync

Sync Kodi watched status without watchedlist plugin (MySQL)
PLpgSQL
2
star
28

vyatta-dhcp-option-6rd-reporter

Add IPv6 6RD reporting for UniFi Security Gateway
Shell
2
star
29

svenska-yle-rss-content-fixer

Attach content to Svenska Yle RSS feeds
Go
2
star
30

go-trueskilld

A simple http API for mafredri/go-trueskill (https://github.com/mafredri/go-trueskill).
Go
2
star
31

asustor-qbittorrent

qBittorrent for ASUSTOR ADM
Shell
2
star
32

kinesis-advantage2

Configuration for my Kinesis Advantage2
Shell
1
star
33

asustor-sonarr

Sonarr for ASUSTOR ADM
Shell
1
star
34

asustor-go

The Go Programming Language for ASUSTOR ADM
Shell
1
star
35

asustor-jackett

Jackett for ASUSTOR ADM
Shell
1
star
36

vyatta-dhcp-global-option

Add support for configuring dhclient global options on UniFi Security Gateway
Perl
1
star
37

asustor-mono-latest

An up-to-date version of Mono for ASUSTOR ADM
Shell
1
star
38

go-mathextra

Extra math functions for Go
Go
1
star
39

asustor-python-cec

Python
1
star
40

as-search

Search from ASUSTOR NAS using Searchligt
Python
1
star
41

mafredri.github.io

HTML
1
star
42

asustor-rtorrent

rTorrent for Asustor ADM
Shell
1
star
43

playgoround

My Golang playground (playGoRound)
Go
1
star