github.com/tiredofit/docker-restic

About

This will build a Docker Image for Restic, A deduplicating, compressing backup tool capable of backing up to many different remote locatoins.

Features:

• Multiple backup snapshot support
• Schedule times to take snapshots
• Cleanup/Snapshot retention support
• Repository check support (multiple)
• Repository prune support (multiple)
• Pre and Post Hooks for all operations
• Restic REST Server included
• RClone included for REST Server functionality/connecting to different backends
• Multiple repository support
• Metrics shipping to Zabbix server

Maintainer

• Dave Conroy

Table of Contents

• About
• Maintainer
• Table of Contents
• Installation
  • Build from Source
  • Prebuilt Images
    • Multi Architecture
• Configuration
  • Quick Start
  • Persistent Storage
  • Environment Variables
    • Base Images used
    • Container Options
    • Job Defaults
    • Backup Options
      • Default Backup Options
      • Job Backup Options
      • Hooks
    • Check Options
      • Default Check Options
      • Job Check Options
      • Hooks
    • Cleanup Options
      • Default Cleanup Options
      • Job Cleanup Options
      • Hooks
    • Inventory Options
      • Default Inventory Options
      • Hooks
    • Prune Options
      • Default Prune Options
      • Hooks
    • Server Options
    • RClone Options
    • Unlock Options
      • Hooks
    • Notifications
      • Custom Notifications
      • Email Notifications
      • Matrix Notifications
      • Mattermost Notifications
      • Rocketchat Notifications
  • Networking
• Maintenance
  • Shell Access
  • Manual Backups
  • Manual Cleanup
  • Manual Checks
  • Manual Inventory
  • Manual Prune
  • Creating Server Users and Passwords
• Support
  • Usage
  • Bugfixes
  • Feature Requests
  • Updates
• License
• References

Installation

Build from Source

Clone this repository and build the image with docker build -t (imagename) .

Prebuilt Images

Builds of the image are available on Docker Hub and is the recommended method of installation.

docker pull tiredofit/restic:(imagetag)

The following image tags are available along with their tagged release based on what's written in the Changelog:

Multi Architecture

Images are built primarily for amd64 architecture, and may also include builds for arm/v7, arm64 and others. These variants are all unsupported. Consider sponsoring my work so that I can work with various hardware. To see if this image supports multiple architecures, type docker manifest (image):(tag)

Configuration

Quick Start

• The quickest way to get started is using docker-compose. See the examples folder for a working compose.yml that can be modified for development or production use.

• Set various environment variables to understand the capabilities of this image.

• Map persistent storage for access to configuration and data files for backup.

Persistent Storage

The following directories are used for configuration and can be mapped for persistent storage.

Environment Variables

Base Images used

This image relies on an Alpine Linux base image that relies on an init system for added capabilities. Outgoing SMTP capabilities are handlded via msmtp. Individual container performance monitoring is performed by zabbix-agent. Additional tools include: bash,curl,less,logrotate,nano,vim.

Be sure to view the following repositories to understand all the customizable options:

Container Options

Job Defaults

If these are set and no other defaults or variables are set explicitly, they will be added to any of the BACKUP, CHECK, CLEANUP, INVENTORY or PRUNE jobs.

Backup Options

This allows restic to take periodical snapshots to your repository. Multiple Backup Jobs can be scheduled at once. Be careful not so schedule jobs so that they bump up against CHECK, CLEANUP, or PRUNE jobs.

Default Backup Options

If set, these variables will be passed to each backup job, unless each job explicitly sets otherwise.

Job Backup Options

If DEFAULT_BACKUP_ variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value of unset. Additional backup jobs can be scheduled by using BACKUP02_,BACKUP03_,BACKUP04_ ... prefixes.

Hooks

The following will be sent to the snapshot job hook script:

Pre: HOSTNAME CONTAINER_NAME BACKUP INSTANCE_NUMBER[XX] BACKUP[XX]_NAME BACKUP[XX]_REPOSITORY_PATH ROUTINE_START_EPOCH BACKUP[XX]_SNAPSHOT_PATH BACKUP[XX]_SNAPSHOT_PATH_FILE

Example:

server container_name BACKUP 01 backupjobname rest:username:password@http://repo.url 1677953980 /etc /backup-location-file.if_set

Post: HOSTNAME CONTAINER_NAME BACKUP INSTANCE_NUMBER[XX] BACKUP[XX]NAME BACKUP[XX]REPOSITORY_PATH ROUTINE_START_EPOCH PROCESS_START_EPOCH PROCESS_FINISH_EPOCH PROCESS_TOTAL_EPOCH EXITCODE LOGFILE FILES_NEW FILES_CHANGED FILES_UNMODIFIED DIRS_NEW DIRS_CHANGED DIRS_UNMODIFIED SIZE_BYTES_ADDED SIZE_BYTES_STORED SIZE_BYTES_PROCESSED ERROR_COUNT

Example:

server container_name BACKUP 01 backupjobname rest:username:password@http://repo.url 1677953980 1677953981 1677953991 10 0 /logs/20230304/20230304_100501-backup-backupjobname.log 123 100 1024 2 3 2048 1204 1536 65535 0`

Check Options

This allows restic to check your repository for errors. There is functionality to check minimally, a subset of the data, or all data. A Check job requires exclusive access to the Restic Repository, therefore no other jobs should be running on them at any time.

Default Check Options

If set, these variables will be passed to each prune job, unless each job explicitly sets otherwise.

Job Check Options

If DEFAULT_CHECK_ variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value of unset. Additional check jobs can be scheduled by using CHECK02_,CHECK03_,CHECK04_ ... prefixes.

Hooks

The following will be sent to the hook script:

Pre: HOSTNAME CONTAINER_NAME CHECK INSTANCE_NUMBER[XX] CHECK[XX]_NAME CHECK[XX]_REPOSITORY_PATH ROUTINE_START_EPOCH

Example:

server container_name CHECK 01 checkjobname rest:username:password@http://repo.url 1677953980

Post: HOSTNAME CONTAINER_NAME CHECK INSTANCE_NUMBER[XX] CHECK[XX]NAME CHECK[XX]REPOSITORY_PATH ROUTINE_START_EPOCH PROCESS_START_EPOCH PROCESS_FINISH_EPOCH PROCESS_TOTAL_EPOCH EXITCODE LOGFILE PACKS_UNREFERENCED SNAPSHOTS_PROCESSED

Example:

server container_name CHECK 01 checkjobname rest:username:password@http://repo.url 1677953980 1677953981 1677953991 10 0 /logs/20230304/20230304_100501-check-checkjobname.log 0 205 205

Cleanup Options

This allows restic to cleanup old backups from your repository, only retaining snapshots that have a certain criteria. By default this does not actually delete the files from your repository, only the snapshot references. You can run a seperate PRUNE job, or use the included AUTO_PRUNE environment variable. A Cleanup job requires exclusive access to the Restic Repository, therefore no other jobs should be running on them at any time.

Default Cleanup Options

If set, these variables will be passed to each cleanup job, unless each job explicitly sets otherwise.

Job Cleanup Options

If DEFAULT_CLEANUP_ variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value of unset. Additional backup jobs can be scheduled by using CLEANUP02_,CLEANUP03_,CLEANUP04_ ... prefixes.

Hooks

The following will be sent to the hooks script :

Pre: HOSTNAME CONTAINER_NAME CLEANUP INSTANCE_NUMBER[XX] CLEANUP[XX]_NAME CLEANUP[XX]_HOST CLEANUP[XX]_REPOSITORY_PATH ROUTINE_START_EPOCH

Example:

server container_name CLEANUP 01 cleanupname repository rest:username:password@http://repo.url 1677953980

Post: HOSTNAME CONTAINER_NAME CLEANUP INSTANCE_NUMBER[XX] CLEANUP[XX]NAME CLEANUP[XX]_HOST CLEANUP[XX]REPOSITORY_PATH ROUTINE_START_EPOCH PROCESS_START_EPOCH PROCESS_FINISH_EPOCH PROCESS_TOTAL_EPOCH EXITCODE LOGFILE SNAPSHOTS_REMOVED PRUNE_PACKS_PROCESSED PRUNE_PACKS_KEEP PRUNE_PACKS_REPACK PRUNE_PACKS_DELETE PRUNE_PACKS_DELETE_UNREFERENCED PRUNE_PACKS_DELETE_OLD

server container_name CLEANUP 01 cleaupname repository rest:username:password@http://repo.url 1677953980 1677953981 1677953991 10 0 /logs/20230304/20230304_100501-cleanup-cleanupname.log 23 6266 3921 372 1973 82 2345

Inventory Options

This allows restic to take inventory of what backups have been taken on the repository. It creates a log file detailing the snapshot id, date / time, hostname, tags, and backup paths. An Inventory job takes lots of time if working with remote repositories.

Default Inventory Options

If set, these variables will be passed to each inventory job, unless each job explicitly sets otherwise.

If DEFAULT_INVENTORY_ variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value of unset. Additional inventory jobs can be scheduled by using INVENTORY02_,INVENTORY03_,INVENTORY04_ ... prefixes.

Hooks

The following will be sent to the hooks script :

Pre: HOSTNAME CONTAINER_NAME INVENTORY INSTANCE_NUMBER[XX] INVENTORY[XX]_NAME INVENTORY[XX]_REPOSITORY_PATH ROUTINE_START_EPOCH

Example:

server container_name INVENTORY 01 cleanupname rest:username:password@http://repo.url 1677953980

Post: HOSTNAME CONTAINER_NAME INVENTORY INSTANCE_NUMBER[XX] INVENTORY[XX]NAME INVENTORY[XX]REPOSITORY_PATH ROUTINE_START_EPOCH PROCESS_START_EPOCH PROCESS_FINISH_EPOCH PROCESS_TOTAL_EPOCH EXITCODE LOGFILE SNAPSHOTS_TOTAL

server container_name INVENTORY 01 cleaupname rest:username:password@http://repo.url 1677953980 1677953981 1677953991 10 0 /logs/20230304/20230304_100501-cleanup-cleanupname.log 23

Prune Options

This allows restic to delete from the repository filesystem the snapshots that have been marked as "cleaned up". A Prune job requires exclusive access to the Restic Repository, therefore no other jobs should be running on them at any time.

Default Prune Options

If set, these variables will be passed to each prune job, unless each job explicitly sets otherwise.

If DEFAULT_PRUNE_ variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value of unset. Additional prune jobs can be scheduled by using PRUNE02_,PRUNE03_,PRUNE04_ ... prefixes.

Hooks

The following information will be sent to the hook script: Pre: HOSTNAME CONTAINER_NAME PRUNE INSTANCE_NUMBER[XX] PRUNE[XX]_NAME PRUNE[XX]_REPOSITORY_PATH ROUTINE_START_EPOCH

Example:

server container_name PRUNE 01 prunereponame rest:username:password@http://repo.url 1677953980

Post: HOSTNAME CONTAINER_NAME PRUNE INSTANCE_NUMBER[XX] PRUNE[XX]NAME PRUNE[XX]REPOSITORY_PATH ROUTINE_START_EPOCH PROCESS_START_EPOCH PROCESS_FINISH_EPOCH PROCESS_TOTAL_EPOCH EXITCODE LOGFILE PACKS_REVIEWED PACKS_DELETED_UNREFERENCED PACKS_REPACKED PACKS_REINDEXED PACKS_DELETED_OLD PACKS_INDEX_DELETED_OBSOLETE

Example:

server container_name PRUNE 01 prunereponame rest:username:password@http://repo.url 1677953980 1677953981 1677953991 10 0 /logs/20230304/20230304_100501-prune-prunereponame.log 203 37 10 110 41 5

Server Options

This will spawn a REST Server either running with the RESTIC built service, or by using the inbuilt feature of RClone, allowing one to take advantage of backing up to one of the many services it supports. See the maintenance section to create users and passwords.

RClone Options

If set in MODE this will spawn an RClone instance

Unlock Options

Sometimes repositories will get stuck and in a locked state. The image attempts to perform automatic unlocking if it detects errors. These settings shouldn't need to be touched.

Hooks

The following information will be sent to the hook script:

Pre: HOSTNAME CONTAINER_NAME UNLOCK REPOSITORY_PATH ROUTINE_START_EPOCH

Example:

server container_name UNLOCK rest:username:password@http://repo.url 1677953980

Post: `HOSTNAME CONTAINER_NAME TYPE REPOSITORY_PATH ROUTINE_START_EPOCH PROCESS_START_EPOCH PROCESS_FINISH_EPOCH PROCESS_TOTAL_SECONDS EXITCODE LOGFILE

Example:

server container_name UNLOCK rest:username:password@http://repo.url 1677953980 1677953981 1677953991 10 0 20230304/20230304_090351-unlock.log

Notifications

This image has capabilities on sending notifications via a handful of services when a restic process fails.

Custom Notifications

The following is sent to the custom script. Use how you wish:

$1 unix timestamp
$2 logfile
$3 errorcode
$4 subject
$5 body/error message

Email Notifications

See base image for additional SMTP environment variables

Matrix Notifications

Fetch a MATRIX_ACCESS_TOKEN:

curl -XPOST -d '{"type":"m.login.password", "user":"myuserid", "password":"mypass"}' "https://matrix.org/_matrix/client/r0/login"

Copy the JSON response access_token that will look something like this:

{"access_token":"MDAxO...blahblah","refresh_token":"MDAxO...blahblah","home_server":"matrix.org","user_id":"@myuserid:matrix.org"}

Mattermost Notifications

Rocketchat Notifications

Networking

Maintenance

Shell Access

For debugging and maintenance purposes you may want access the containers shell.

docker exec -it (whatever your container name is) bash

Manual Backups

Manual Backups can be performed by entering the container and typing backup-now. This will execute all the backup tasks that are scheduled by means of the BACKUPXX_ variables. Alternatively if you wanted to execute a job on its own you could simply type backup01-now (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.

Manual Cleanup

Manual Cleanup can be performed by entering the container and typing cleanup-now. This will execute all the cleanup tasks that are scheduled by means of the CLEANUPXX_ variables. Alternatively if you wanted to execute a job on its own you could simply type cleanup01-now (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.

Manual Checks

Manual Checks can be performed by entering the container and typing check-now. This will execute all the check tasks that are scheduled by means of the CHECKXX_ variables. Alternatively if you wanted to execute a job on its own you could simply type check01-now (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.

Manual Inventory

Manual Inventory can be performed by entering the container and typing inventory-now. This will execute all the prune tasks that are scheduled by means of the INVENTORYXX_ variables. Alternatively if you wanted to execute a job on its own you could simply type inventory01-now (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.

Manual Prune

Manual Pruning can be performed by entering the container and typing prune-now. This will execute all the prune tasks that are scheduled by means of the PRUNEXX_ variables. Alternatively if you wanted to execute a job on its own you could simply type prune01-now (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.

Creating Server Users and Passwords

Use the server-user command:

• Create: server-user create <username> <password>
• Delete: server-user <delete>

Support

These images were built to serve a specific need in a production environment and gradually have had more functionality added based on requests from the community.

Usage

• The Discussions board is a great place for working with the community on tips and tricks of using this image.
• Consider sponsoring me personalized support.

Bugfixes

• Please, submit a Bug Report if something isn't working as expected. I'll do my best to issue a fix in short order.

Feature Requests

• Feel free to submit a feature request, however there is no guarantee that it will be added, or at what timeline.
• Consider sponsoring me regarding development of features.

Updates

• Best effort to track upstream changes, More priority if I am actively using the image in a production environment.
• Consider sponsoring me for up to date releases.

License

MIT. See LICENSE for more details.

References

• https://restic.readthedocs.io/en/latest/020_installation.html