• Stars
    star
    195
  • Rank 199,374 (Top 4 %)
  • Language
    Go
  • License
    BSD 2-Clause "Sim...
  • Created about 7 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Manages a virtual IP based on state kept in etcd or Consul

License: MIT Go Report Card Release Github All Releases

vip-manager

Manages a virtual IP based on state kept in etcd or Consul

Table of Contents

Prerequisites

  • go >= 1.19
  • make (optional)
  • goreleaser (optional)

Building

  1. clone this repo
git clone https://github.com/cybertec-postgresql/vip-manager.git
  1. Build the binary using make or go build.
  2. To build your own packages (.deb, .rpm, .zip, etc.), run
make package

or

goreleaser release --snapshot --skip-publish --rm-dist

Installing from package

You can download .rpm or .deb packages here, on the Releases page. On Debian and Ubuntu, the universe repositories should provide you with vip-manager, though the version may be not as recent.

Warning
Our packages are probably not compatible with the one from those repositories, do not try to install them side-by-side.

Installing from source

  • Follow the steps to build vip-manager.
  • Run DESTDIR=/tmp make install to copy the binary, service files and config file into the destination of your choice.
  • Edit config to your needs, then run systemctl daemon-reload, then systemctl start vip-manager.

Note
systemd will only pick the service files up if you chose a DESTDIR so that it can find it. Usually DESTDIR='' should work.

Environment prerequisites

When vip-manager is in charge of registering and deregistering the VIP locally, it needs superuser privileges to do so. This is not required when vip-manager is used to manage a VIP through some API, e.g. Hetzner Robot API or Hetzner Cloud API.

Note
At some point it would be great to reduce this requirement to only the CAP_NET_RAW and CAP_NET_ADMIN capabilities, which could be added by a superuser to the vip-manager binary once. Right now, this is not possible since vip-manager launches plain shell commands to register and deregister virtual IP addresses locally (at least on linux), so the whole user would need these privileges. When vip-manager is eventually taught to directly use a library that directly uses the Linux kernel's API to register/deregister the VIP, the capabilities set for the binary will suffice.

PostgreSQL prerequisites

For any virtual IP based solutions to work in general with Postgres you need to make sure that it is configured to automatically scan and bind to all found network interfaces. So something like * or 0.0.0.0 (IPv4 only) is needed for the listen_addresses parameter to activate the automatic binding. This again might not be suitable for all use cases where security is paramount for example.

nonlocal bind

If you can't set listen_addresses to a wildcard address, you can explicitly specify only those adresses that you want to listen to. However, if you add the virtual IP to those addresses, PostgreSQL will fail to start when that address is not yet registered on one of the interfaces of the machine. You need to configure the kernel to allow "nonlocal bind" of IP (v4) addresses:

  • temporarily:
sysctl -w net.ipv4.ip_nonlocal_bind=1
  • permanently:
echo "net.ipv4.ip_nonlocal_bind = 1"  >> /etc/sysctl.conf
sysctl -p

Configuration

The configuration can be passed to the executable through argument flags, environment variables or through a YAML config file. Run vip-manager --help to see the available flags.

Note
The location of the YAML config file can be specified with the --config flag. An exemplary config file is installed into /etc/default/vip-manager.yml or is available in the vipconfig directory in the repository of the software.

Configuration is now (from release v1.0 on) handled using the viper library. This means that environment variables, command line flags, and config files can be used to configure vip-manager. When using different configuration sources simultaneously, this is the precedence order:

  • flag
  • env
  • config

Note
So flags always overwrite env variables and entries from the config file. Env variables overwrite the config file entries.

All flags and file entries are written in lower case. To make longer multi-word flags and entries readable, they are separated by dashes, e.g. retry-num.

If you put a flag or file entry into uppercase and replace dashes with underscores, you end up with the format of environment variables. To avoid overlapping configuration with other applications, the env variables are additionall prefixed with VIP_, e.g. VIP_RETRY_NUM.

This is a list of all avaiable configuration items:

flag/yaml key env notation required example description
ip VIP_IP yes 10.10.10.123 The virtual IP address that will be managed.
netmask VIP_NETMASK yes 24 The netmask that is associated with the subnet that the virtual IP vip is part of.
interface VIP_INTERFACE yes eth0 A local network interface on the machine that runs vip-manager. Required when using manager-type=basic. The vip will be added to and removed from this interface.
trigger-key VIP_TRIGGER_KEY yes /service/pgcluster/leader The key in the DCS that will be monitored by vip-manager. Must match <namespace>/<scope>/leader from Patroni config. When the value returned by the DCS equals trigger-value, vip-manager will make sure that the virtual IP is registered to this machine. If it does not match, vip-manager makes sure that the virtual IP is not registered to this machine.
trigger-value VIP_TRIGGER_VALUE no pgcluster_member_1 The value that the DCS' answer for trigger-key will be matched to. Must match <name> from Patroni config. This is usually set to the name of the patroni cluster member that this vip-manager instance is associated with. Defaults to the machine's hostname.
manager-type VIP_MANAGER_TYPE no basic Either basic or hetzner. This describes the mechanism that is used to manage the virtual IP. Defaults to basic.
dcs-type VIP_DCS_TYPE no etcd The type of DCS that vip-manager will use to monitor the trigger-key. Defaults to etcd.
dcs-endpoints VIP_DCS_ENDPOINTS no http://10.10.11.1:2379 A url that defines where to reach the DCS. Multiple endpoints can be passed to the flag or env variable using a comma-separated-list. In the config file, a list can be specified, see the sample config for an example. Defaults to http://127.0.0.1:2379 for dcs-type=etcd and http://127.0.0.1:8500 for dcs-type=consul.
etcd-user VIP_ETCD_USER no patroni A username that is allowed to look at the trigger-key in an etcd DCS. Optional when using dcs-type=etcd .
etcd-password VIP_ETCD_PASSWORD no snakeoil The password for etcd-user. Optional when using dcs-type=etcd . Requires that etcd-user is also set.
consul-token VIP_CONSUL_TOKEN no snakeoil A token that can be used with the consul-API for authentication. Optional when using dcs-type=consul .
interval VIP_INTERVAL no 1000 The time vip-manager main loop sleeps before checking for changes. Measured in ms. Defaults to 1000.
retry-after VIP_RETRY_AFTER no 250 The time to wait before retrying interactions with components outside of vip-manager. Measured in ms. Defaults to 250.
retry-num VIP_RETRY_NUM no 3 The number of times interactions with components outside of vip-manager are retried. Defaults to 3.
etcd-ca-file VIP_ETCD_CA_FILE no /etc/etcd/ca.cert.pem A certificate authority file that can be used to verify the certificate provided by etcd endpoints. Make sure to change dcs-endpoints to reflect that https is used.
etcd-cert-file VIP_ETCD_CERT_FILE no /etc/etcd/client.cert.pem A client certificate that is used to authenticate against etcd endpoints. Requires etcd-ca-file to be set as well.
etcd-key-file VIP_ETCD_KEY_FILE no /etc/etcd/client.key.pem A private key for the client certificate, used to decrypt messages sent by etcd endpoints. Required when etcd-cert-file is specified.
verbose VIP_VERBOSE no true Enable more verbose logging. Currently only the manager-type=hetzner provides additional logs.

Configuration - Hetzner

To use vip-manager with Hetzner Robot API you need a Credential file, set hosting_type to hetzner in /etc/default/vip-manager.yml and your Floating-IP must be added on all Servers. The Floating-IP (VIP) will not be added or removed on the current Master node interface, Hetzner will route it to the current one.

Credential File - Hetzner

Add the File /etc/hetzner with your Username and Password

user="myUsername"
pass="myPassword"

Debugging

Either:

  • run vip-manager with --verbose flag or
  • set verbose to true in /etc/default/vip-manager.yml
  • set VIP_VERBOSE=true

Note
Currently only supported for hetzner

Author

Cybertec Schönig & Schönig GmbH, https://www.cybertec-postgresql.com

More Repositories

1

pgwatch2

PostgreSQL metrics monitor/dashboard
PLpgSQL
1,792
star
2

pg_timetable

pg_timetable: Advanced scheduling for PostgreSQL
Go
1,072
star
3

pg_squeeze

A PostgreSQL extension for automatic bloat cleanup
C
462
star
4

pg_show_plans

Show query plans of all currently running SQL statements
C
186
star
5

postgres-showcase

Postgres features showcase (commented SQL samples) for beginners
164
star
6

pgfaceting

Faceted query acceleration for PostgreSQL using roaring bitmaps
PLpgSQL
145
star
7

pg_permissions

A simple set of views to see ALL permissions in a PostgreSQL database
PLpgSQL
143
star
8

ora_migrator

Tools for Oracle to PostgreSQL migration
PLpgSQL
109
star
9

zheap

⚡️ Development status and progress reporting.
HTML
93
star
10

rjsf-material-ui

[Archived] Material UI theme for react-jsonschema-form.
TypeScript
62
star
11

pgwatch

🔬PGWATCH: PostgreSQL metrics monitor/dashboard
Go
59
star
12

postgres

PostgreSQL with Transparent Data Encryption (TDE)
C
49
star
13

pg_cgroups

PostgreSQL extension to manage Linux Control Groups
C
36
star
14

pg_rewrite

Perform maintenance tasks which require a table to be rewritten (i.e. the table data to be copied to a new storage) and which are expected to limit the access to the table as little as possible
C
34
star
15

pg_crash

Periodically or randomly crash your database
C
26
star
16

safe-backup

Pre- and post-backup scripts for a safe PostgreSQL online file system backup
Shell
25
star
17

cybertec_migrator

CYBERTEC Migrator 🗄🔀🗄
Shell
22
star
18

patroni-packaging

Shell
20
star
19

db_migrator

PLpgSQL
18
star
20

react-database-diagram

A react component to render nice database diagram using storm-react-diagrams
TypeScript
17
star
21

patroni-windows-packaging

Automate installing and launching of Patroni under Windows
Batchfile
15
star
22

pg_timetable_gui

GUI for pg_timetable
Pascal
13
star
23

CYBERTEC-pg-operator

Go
12
star
24

pg_sequence_fixer

Fixing PostgreSQL sequences which got out of sync with the data
PLpgSQL
12
star
25

walbouncer

A proxy server for PostgreSQL replication connections with the capability to filter out user defined subsets of data
C
12
star
26

scripts

Various scripts around PostgreSQL management
Python
10
star
27

generic-plan

generate a generic plan for a parameterized SQL statement
PLpgSQL
10
star
28

pg_remote_exec

Run shell commands in SQL prompt
C
8
star
29

drop_role_helper

Generate SQL to revoke all privileges of a role in the current database
PLpgSQL
7
star
30

poc-plpgsql-analyzer

Proof of concept for tooling to migrate PL/SQL code to PL/pgSQL written in Rust
Rust
7
star
31

PES

Patroni Environment Setup
Pascal
6
star
32

walbouncer-companion

A tool to perform a selective Postgres basebackup
Python
6
star
33

essence

☕ breaks during database dump extraction? Not on our watch!
Rust
4
star
34

merge_ips

PLpgSQL
3
star
35

today-i-learned

TypeScript
3
star
36

python_template

Template repository for Python command line apps.
Python
3
star
37

.github

Repository to share actions and workflows
3
star
38

CYBERTEC-operator-tutorials

3
star
39

yaim

yaim - yet another ip manager
Go
2
star
40

postgresql-action

:octocat: GitHub Action for PostgreSQL
2
star
41

layman

fuse-overlayfs within Docker for volume mounts.
Shell
2
star
42

debezium2postgres

Application to apply CDC log from Debezium to the destination PostgreSQL
Go
2
star
43

pgbackrest-tde

pgbackrest fork, patched for PostgreSQL-TDE
C
2
star
44

patroni-infoblox-integration

Patroni callbacks for integrating into Infoblox DNS services
Python
2
star
45

CYBERTEC-pg-container

Python
1
star
46

today-i-learned-content

This repository contains all the content for Today I learned @Cybertec-postgresql.
Shell
1
star
47

postgresql

A fork of https://salsa.debian.org/postgresql/postgresql/
1
star