• Stars
    star
    12,859
  • Rank 2,476 (Top 0.05 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 11 years ago
  • Updated 11 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

groupcache

Summary

groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many cases.

For API docs and examples, see http://godoc.org/github.com/golang/groupcache

Comparison to memcached

Like memcached, groupcache:

  • shards by key to select which peer is responsible for that key

Unlike memcached, groupcache:

  • does not require running a separate set of servers, thus massively reducing deployment/configuration pain. groupcache is a client library as well as a server. It connects to its own peers, forming a distributed cache.

  • comes with a cache filling mechanism. Whereas memcached just says "Sorry, cache miss", often resulting in a thundering herd of database (or whatever) loads from an unbounded number of clients (which has resulted in several fun outages), groupcache coordinates cache fills such that only one load in one process of an entire replicated set of processes populates the cache, then multiplexes the loaded value to all callers.

  • does not support versioned values. If key "foo" is value "bar", key "foo" must always be "bar". There are neither cache expiration times, nor explicit cache evictions. Thus there is also no CAS, nor Increment/Decrement. This also means that groupcache....

  • ... supports automatic mirroring of super-hot items to multiple processes. This prevents memcached hot spotting where a machine's CPU and/or NIC are overloaded by very popular keys/values.

  • is currently only available for Go. It's very unlikely that I (bradfitz@) will port the code to any other language.

Loading process

In a nutshell, a groupcache lookup of Get("foo") looks like:

(On machine #5 of a set of N machines running the same code)

  1. Is the value of "foo" in local memory because it's super hot? If so, use it.

  2. Is the value of "foo" in local memory because peer #5 (the current peer) is the owner of it? If so, use it.

  3. Amongst all the peers in my set of N, am I the owner of the key "foo"? (e.g. does it consistent hash to 5?) If so, load it. If other callers come in, via the same process or via RPC requests from peers, they block waiting for the load to finish and get the same answer. If not, RPC to the peer that's the owner and get the answer. If the RPC fails, just load it locally (still with local dup suppression).

Users

groupcache is in production use by dl.google.com (its original user), parts of Blogger, parts of Google Code, parts of Google Fiber, parts of Google production monitoring systems, etc.

Presentations

See http://talks.golang.org/2013/oscon-dl.slide

Help

Use the golang-nuts mailing list for any discussion or questions.

More Repositories

1

go

The Go programming language
Go
121,832
star
2

dep

Go dependency management tool experiment (deprecated)
Go
12,857
star
3

protobuf

Go support for Google's protocol buffers
Go
9,676
star
4

mock

GoMock is a mocking framework for the Go programming language.
Go
9,286
star
5

tools

[mirror] Go Tools
Go
7,321
star
6

mobile

[mirror] Go on Mobile
Go
5,768
star
7

oauth2

Go OAuth2
Go
5,270
star
8

lint

[mirror] This is a linter for Go source code. (deprecated)
Go
3,974
star
9

vscode-go

Go extension for Visual Studio Code
TypeScript
3,810
star
10

glog

Leveled execution logs for Go
Go
3,520
star
11

proposal

Go Project Design Documents
Go
3,312
star
12

crypto

[mirror] Go supplementary cryptography libraries
Go
2,982
star
13

net

[mirror] Go supplementary network libraries
Go
2,846
star
14

example

Go example projects
Go
2,526
star
15

geo

S2 geometry library in Go
Go
1,655
star
16

tour

[mirror] A Tour of Go
Go
1,543
star
17

vgo

[mirror] Versioned Go Prototype
Go
1,519
star
18

snappy

The Snappy compression format in the Go programming language.
Go
1,512
star
19

sys

[mirror] Go packages for low-level interaction with the operating system
Go
1,253
star
20

pkgsite

[mirror] Home of the pkg.go.dev website
Go
1,148
star
21

leveldb

The LevelDB key-value database in the Go programming language.
Go
1,145
star
22

gddo

Go Doc Dot Org
Go
1,104
star
23

gofrontend

Go compiler frontend (gccgo)
Go
852
star
24

sync

[mirror] concurrency primitives
Go
848
star
25

exp

[mirror] Experimental and deprecated packages
Go
838
star
26

freetype

The Freetype font rasterizer in the Go programming language.
Go
783
star
27

text

[mirror] Go text processing support
Go
745
star
28

playground

[mirror] The Go Playground
Go
741
star
29

talks

Go talks
694
star
30

appengine

Go App Engine packages
Go
669
star
31

build

[mirror] Go's continuous build and release infrastructure (no stability promises)
Go
627
star
32

vulndb

[mirror] The Go Vulnerability Database
Go
555
star
33

image

[mirror] Go supplementary image libraries
Go
522
star
34

blog

[mirror] Go Blog (obsolete)
389
star
35

time

[mirror] Go supplementary time packages
Go
383
star
36

vuln

[mirror] the database client and tools for the Go vulnerability database
Go
376
star
37

perf

[mirror] Performance measurement, storage, and analysis.
Go
375
star
38

website

[mirror] Home of the go.dev and golang.org websites
HTML
373
star
39

sublime-build

The official Sublime Text package for Go build system integration.
Python
343
star
40

xerrors

Go
277
star
41

term

Go terminal and console support
Go
272
star
42

debug

[mirror] debugging tools
Go
230
star
43

dl

[mirror] go install golang.org/dl/go1.N@latest
Go
189
star
44

mod

[mirror] Go module mechanics libraries
Go
187
star
45

benchmarks

Benchmarks for the perf dashboard
Go
181
star
46

arch

[mirror] architecture code
Go
145
star
47

review

[mirror] Tool for working with Gerrit code reviews
Go
143
star
48

cwg

Community outreach Working Group
132
star
49

oscar

[mirror] Open source contributor agent architecture repo.
Go
90
star
50

govulncheck-action

[mirror] GitHub action for govulncheck
89
star
51

sublime-config

A library for Go environment configuration in Sublime Text
Python
85
star
52

winstrap

Bootstrapping tools for windows builders
Go
48
star
53

telemetry

[mirror] Go Telemetry services and libraries
Go
36
star
54

wiki

[mirror] Go Wiki
29
star
55

scratch

[mirror] repository used for testing
Go
28
star
56

pkgsite-metrics

Code to serve pkg.go.dev/metrics [mirror]
Go
13
star
57

go-get-issue-15410

go-get-issue-15410
Go
4
star
58

.github

2
star
59

.allstar

2
star