• Stars
    star
    1,357
  • Rank 33,315 (Top 0.7 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A command-line tool to perform health-checks for gRPC applications in Kubernetes and elsewhere

grpc_health_probe(1)

ci GitHub all releases

The grpc_health_probe utility allows you to query health of gRPC services that expose service their status through the gRPC Health Checking Protocol.

grpc_health_probe is meant to be used for health checking gRPC applications in Kubernetes, using the exec probes.

⚠️ Kubernetes v1.23 has now introduced built-in gRPC health checking capability as an alpha feature. As a result, you might no longer need to use this tool and use the native Kubernetes feature instead.

This tool can still be useful if you are on older versions of Kubernetes, or using advanced configuration (such as custom metadata, TLS or finer timeout tuning), or not using Kubernetes at all.

This command-line utility makes a RPC to /grpc.health.v1.Health/Check. If it responds with a SERVING status, the grpc_health_probe will exit with success, otherwise it will exit with a non-zero exit code (documented below).

EXAMPLES

$ grpc_health_probe -addr=localhost:5000
healthy: SERVING
$ grpc_health_probe -addr=localhost:9999 -connect-timeout 250ms -rpc-timeout 100ms
failed to connect service at "localhost:9999": context deadline exceeded
exit status 2

Installation

It is recommended to use a version-stamped binary distribution:

  • Choose a binary from the Releases page.

Installing from source (not recommended):

  • Make sure you have git and go installed.
  • Run: go install github.com/grpc-ecosystem/grpc-health-probe@latest
  • This will compile the binary into your $GOPATH/bin (or $HOME/go/bin).

Using the gRPC Health Checking Protocol

To make use of the grpc_health_probe, your application must implement the gRPC Health Checking Protocol v1. This means you must to register the Health service and implement the rpc Check that returns a SERVING status.

Since the Health Checking protocol is part of the gRPC core, it has packages/libraries available for the languages supported by gRPC:

[health.proto] [Go] [Java] [Python] [C#/NuGet] [Ruby] ...

Most of the languages listed above provide helper functions that hides implementation details. This eliminates the need for you to implement the Check rpc yourself.

Example: gRPC health checking on Kubernetes

Kubernetes does not natively support gRPC health checking since it does not favor one RPC framework over another. Similarly, HTTP health probes Kubernetes has is not sufficient to craft a valid gRPC request. As a solution, grpc_health_probe can be used for Kubernetes to health-check gRPC servers running in the Pod.

You are recommended to use Kubernetes exec probes and define liveness and/or readiness checks for your gRPC server pods.

You can bundle the statically compiled grpc_health_probe in your container image. Choose a binary release and download it in your Dockerfile:

RUN GRPC_HEALTH_PROBE_VERSION=v0.4.13 && \
    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && \
    chmod +x /bin/grpc_health_probe

In your Kubernetes Pod specification manifest, specify a livenessProbe and/or readinessProbe for the container:

spec:
  containers:
  - name: server
    image: "[YOUR-DOCKER-IMAGE]"
    ports:
    - containerPort: 5000
    readinessProbe:
      exec:
        command: ["/bin/grpc_health_probe", "-addr=:5000"]
      initialDelaySeconds: 5
    livenessProbe:
      exec:
        command: ["/bin/grpc_health_probe", "-addr=:5000"]
      initialDelaySeconds: 10

This approach provide proper readiness/liveness checking to your applications that implement the gRPC Health Checking Protocol.

Health Checking TLS Servers

If a gRPC server is serving traffic over TLS, or uses TLS client authentication to authorize clients, you can still use grpc_health_probe to check health with command-line options:

Option Description
-tls use TLS (default: false)
-tls-ca-cert path to file containing CA certificates (to override system root CAs)
-tls-client-cert client certificate for authenticating to the server
-tls-client-key private key for for authenticating to the server
-tls-no-verify use TLS, but do not verify the certificate presented by the server (INSECURE) (default: false)
-tls-server-name override the hostname used to verify the server certificate

Health checking TLS Servers with SPIFFE issued credentials

If your gRPC server requires authentication, you can use the following command line options and set the SPIFFE_ENDPOINT_SOCKET environment variable.

Option Description
-spiffe use SPIFFE Workload API to retrieve TLS credentials (default: false)

Other Available Flags

Option Description
-v verbose logs (default: false)
-connect-timeout timeout for establishing connection
-rpc-timeout timeout for health check rpc
-rpc-header sends metadata in the RPC request context (default: empty map)
-user-agent user-agent header value of health check requests (default: grpc_health_probe)
-service service name to check (default: "") - empty string is convention for server health
-gzip use GZIPCompressor for requests and GZIPDecompressor for response (default: false)

Example:

  1. Start the route_guide example server with TLS by running:

    go run server/server.go -tls
    
  2. Run grpc_client_probe with the CA certificate (in the testdata/ directory) and hostname override the cert is signed for:

    $ grpc_health_probe -addr 127.0.0.1:10000 \
        -tls \
        -tls-ca-cert /path/to/testdata/ca.pem \
        -tls-server-name=example.com \
        -rpc-header=foo:bar \
        -rpc-header=foo2:bar2
    
    status: SERVING

Exit codes

It is not recommended to rely on specific exit statuses. Any failure will be a non-zero exit code.

Exit Code Description
0 success: rpc response is SERVING.
1 failure: invalid command-line arguments
2 failure: connection failed or timed out
3 failure: rpc failed or timed out
4 failure: rpc successful, but the response is not SERVING
20 failure: could not retrieve TLS credentials using the SPIFFE Workload API

This is not an official Google project.

More Repositories

1

grpc-gateway

gRPC to JSON proxy generator following the gRPC HTTP spec
Go
17,198
star
2

awesome-grpc

A curated list of useful resources for gRPC
7,163
star
3

go-grpc-middleware

Golang gRPC Middlewares: interceptor chaining, auth, logging, retries and more.
Go
6,058
star
4

grpc-spring

Spring Boot starter module for gRPC framework.
Java
3,328
star
5

go-grpc-prometheus

Prometheus monitoring for your gRPC Go servers.
Go
1,328
star
6

polyglot

A universal grpc command line client
Java
525
star
7

grpc-opentracing

OpenTracing is a set of consistent, expressive, vendor-neutral APIs for distributed tracing and context propagation
Python
465
star
8

java-grpc-prometheus

Java interceptors which can be used to monitor Grpc services using Prometheus.
Java
220
star
9

grpc-httpjson-transcoding

Transcoding to provide HTTP/JSON interface for gRPC Service
C++
154
star
10

protoc-gen-grpc-gateway-ts

protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript clients that connect the web frontend and golang backend fronted by grpc-gateway.
Go
136
star
11

grpc-simon-says

Multiplayer Simon Says game using bidirectional gRPC streaming
Go
132
star
12

grpcdebug

grpcdebug is a command line interface focusing on simplifying the debugging process of gRPC applications.
Go
124
star
13

grpc-cloud-run-example

Go
114
star
14

meetup-kit

gRPC meet up kit in a box
37
star
15

grift

This repository hosts gRPC's support for Thrift IDL and protocol
Java
10
star
16

grpc-exchange-o-gram

Exchange-o-gram demo
C#
8
star
17

grpcz-stackdriver

grpcz-monitoring - description: standalone agent for monitoring statistics from grpc library and publishing to various sinks (like stackdriver, serial port etc).
4
star
18

proto-converter

C++
2
star
19

proto-field-extraction

C++
1
star