Restic is a modern backup program with support for encryption and deduplication. As we saw in this tutorial, Restic is easy to use, but it doesn’t make use of a configuration file, and doesn’t natively include a method to orchestrate and organize multiple backups. That is where autorestic comes in handy: it is a wrapper around Restic which let us configure and orchestrate backups in a single yaml configuration file, specifying multiple sources (locations) and destinations (backends).
In this tutorial we learn how to install autorestic on Linux, how its configuration file is structured, and how to create a basic setup.
In this tutorial you will learn:
- How to install autorestic on Linux
- How the autorestic configuration file is structured
- How to configure Restic backups with autorestic
- How to execute custom commands on autorestic hooks
| Category | Requirements, Conventions or Software Version Used |
|---|---|
| System | Distribution-agnostic |
| Software | Restic, autorestic |
| Other | None |
| Conventions | # – requires given linux-commands to be executed with root privileges either directly as a root user or by use of sudo command$ – requires given linux-commands to be executed as a regular non-privileged user |
Installing autorestic
In this article I will assume you alread installed Restic; if you are not familiar with it, you may want to take a look at our introductive tutorial, first.
At the moment of writing, of all the major Linux distributions, only Fedora includes autorestic in its repositories. To install the software, we can either download a pre-compiled binary from the project GitHub repository, or use the dedicated Docker image. Let’s explore these options.
Manual installation
The easiest way to install autorestic is to download the appropriate version of the binary for our operating system, and place it in a directory in our PATH. In the example below, we download the latest available x86_64 Linux version (1.8.2 at the moment of writing), and install it in the /usr/local/bin directory. First, we download the “.bz2” archive:
$ curl -LO https://github.com/cupcakearmy/autorestic/releases/download/v1.8.2/autorestic_1.8.2_linux_amd64.bz2
Once the download is complete we extract the file:
$ bzip2 -d autorestic_1.8.2_linux_amd64.bz2
Finally, we move the file in the target directory, and we make it executable:
$ sudo mv autorestic_1.8.2_linux_amd64 /usr/local/bin/autorestic && sudo chmod +x /usr/local/bin/autorestic
Installation via package manager (Fedora-only)
If we are using Fedora, the easiest and most convenient way to install autorestic, is by using the dnf package manager, since the tool is included in the official repositories of the distribution:
$ sudo dnf install autorestic
Using the autorestic Docker image
An official Docker autorestic image is available; when running a container based on it, we need to bind-mount the directories we want to backup, the autorestic configuration file, and the target repository (if performing a local backup), somewhere inside the container, and point autorestic to the path of configuration file with the -c option. Here is an example in which we backup the content of the host /data directory:
$ sudo docker run --rm -v /data:/data:Z -v /etc/autorestic.yml:/etc/autorestic.yml:Z -v /mnt/restic_repo:/mnt/restic_repo:Z cupcakearmy/autorestic autorestic backup -va -c /etc/autorestic.yml
We bind-mounted the
/etc/autorestic.yaml configuration file, the /data and the /mnt/restic_repo directories (which are the source and destination of our backup, respectively) in their respective positions inside the container. For each bind mount we used the Z option: this is only necessary when SELinux is active, since it changes the label of the referenced files to object_r:container_file_t, so that they are accessible inside the container.
The autorestic configuration file
The core of autorestic is its yaml configuration file. In this file, we set the sources and destinations of our backups, the restic options we want to use, and many other things. For obvious reasons, we can’t cover all possible options here, so we will see only the essential stuff.
Autorestic looks for the configuration file in the following positions, in order of priority:
- .autorestic.yml
- ~/.autorestic.yml
- ~/.config/autorestic/.autorestic.yml
Inside the configuration file, we define the sources of our backups in the “location” section, and their destinations in the “backends” section. Let’s start with a basic example. Suppose we want to backup the content of the /data directory to the /mnt/restic_repo local repository. Here is how we would populate the autorestic configuration file, which we will save as .autorestic.yml in our current working directory:
version: 2
locations:
data:
from:
- /data
to:
- localrepo
backends:
localrepo:
type: local
path: /mnt/restic_repo
key: repositorypassword
Each location is identified by a lowercase name; in this case we used “data” for the one we created. With the from key, we define one or multiple directories we want to backup: the paths can be specified as absolute or relative to the path of the configuration file. With the to key, instead, we provide a single or multiple targets or backends for the location, again, referencing them by name. We define each destination, or “backend”, in the “backends” section.
In this example, we defined just one backend, which we called “localrepo” (just like locations, backends names must be defined in lowercase form). By setting the value of the type key to “local”, we specified the backend is a restic repository which exists on the local filesystem (as we know, restic supports a lot of storage platforms; among the others: Backblaze, S3 and SFTP). With the path key, we specified the path of the restic repository, and, finally, with key we reported the repository password (putting a cleartext password in a configuration file can be dangerous, we will see some alternatives to this behavior later).
Launching a backup
Now, if it is the first time we run the backup, we can use the autorestic
check command, to be sure everyhing is ready. The command will automatically initialize the defined backends for us, therefore there is no need to create the restic repositories beforehand:
$ sudo autorestic check
To start the backup we can now use the backup command:
$ sudo autorestic backup -va
Above, we invoked autorestic with the -v option, to make it run in verbose mode, and with -a to instruct it to perform the backup of all existing locations. In case we defined multiple locations, and we want to backup only a subset of them, we can pass the comma-separated list of their names as argument to the -l option instead, e.g:
$ autorestic backup -v -l data
Specifying restic options
In the autorestic configuration file, we can specify the options we want to pass to restic, for each location, or globally. Options restic should be invoked with, are provided as value of the options key. We can specify the options we want to pass for the backup and forget commands, or for both of them, using the backup, forget, and all keys, respectively. Here is an example. Suppose we want to exclude all files with the “.txt” extension when performing a backup of the “data” location. We would write:
version: 2
locations:
data:
from:
- /data
to:
- localrepo
options:
backup:
exclude:
- '*.txt'
backends:
localrepo:
type: local
path: /mnt/restic_repo
key: repositorypassword
The restic “forget” command, is used to remove old snapshots from a repository, according to the policy we specify with the --keep options. To provide those options, we specify them under the forget key. To always keep all the most recent 10 snapshots and delete the older ones, for example, we would use --keep-last=10:
version: 2
locations:
data:
from:
- /data
to:
- localrepo
options:
backup:
exclude:
- '*.txt'
forget:
keep-last: 10
backends:
localrepo:
type: local
path: /mnt/restic_repo
key: repositorypassword
The “forget” options are applied when we invoke autorestic with the forget command:
$ autorestic forget -va
We can configure autorestic so that the “forget” command is automatically executed after each backup, by using the
forget key directly in the location definition. The value of this key can be set to true if we just want the snapshots to be forgotten, or to prune if we also want to remove data related to the forgotten snapshots from the repository. Here is an example:
version: 2
locations:
data:
from:
- /data
to:
- localrepo
forget: prune
options:
backup:
exclude:
- '*.txt'
forget:
keep-last: 10
backends:
localrepo:
type: local
path: /mnt/restic_repo
key: repositorypassword
The options we set in the configuration file until now, are valid only for the “data” location. If we have multiple locations, and we want to apply certain options to all of them, we can populate the global section, e.g:
version: 2
global:
backup:
exclude:
- '*.txt'
forget:
keep-last: 10
locations:
data:
from:
- /data
to:
- localrepo
backends:
localrepo:
type: local
path: /mnt/restic_repo
key: repositorypassword
Location-specific options overiddes those set in the global section. This strategy is useful if we want to just specify exceptions.
Alternative ways to provide a backend password
In the previous examples, we directly specified the password for the “localrepo” backend in the configuration file. As an alternative, we can set the password as the value of an environment variable we can define when launching autorestic, or in a dedicated file called .autorestic.env, which must be located in the same directory as the configuration file. In order for the variable to be automatically associated to a specific backend, we define it using the following syntax:
AUTORESTIC_<BACKEND-NAME>_<VARIABLE_NAME>
So, to assign the password for the “localrepo” backend, we would write:
AUTORESTIC_LOCALREPO_RESTIC_PASSWORD="repositorypassword"
We can use this strategy also to provide additional information. When using a remote Backblaze backend, for example, we can provide both the account id and account key. Suppose we defined a backup called “backblaze”, we would write:
AUTORESTIC_BACKBLAZE_B2_ACCOUNT_ID="123" AUTORESTIC_BACKBLAZE_B2_ACCOUNT_KEY="456"
Another alternative consists into using the restic native --password-command option. With this option we can provide a command which must return the password to stdout. With this strategy, for example, we can read the password from an external file, as we did below:
version: 2
locations:
data:
from:
- /data
to:
- localrepo
options:
all:
password-command: cat /root/.password
backup:
exclude:
- '*.txt'
forget:
keep-last: 10
backends:
localrepo:
type: local
path: /mnt/restic_repo
We used the option in the “all” section, so that it is always passed, no matter if we run the backup or the forget command.
Autorestic hooks
Autorestic allows us to execute custom commands in specific moments, using the following hooks:
- prevalidate
- before
- after
- failure
- success
The prevalidate hook is run before everything else, even before various checks are executed (autorestic checks if source directories associated to a location exist, for example); the before hook, instead, is executed after the checks, but before the backup is launched. The after hook is always executed after the backup is finished, independently of the exit status (it is instead skipped if the backup can’t be started at all, due to errors in the “prevalidate” or “before” hooks). Finally, commands associated with the failure and success hooks, are invoked after the backup is executed successfully, or after it fails, respectively.
In the example below, we send a push notification using the ntfy.sh service if the backup is successful:
version: 2
locations:
data:
from:
- /data
to:
- localrepo
options:
all:
password-command: cat /root/.password
backup:
exclude:
- '*.txt'
forget:
keep-last: 10
hooks:
success:
- curl -d "backup executed succesfully!" ntfy.sh/lxcfg
backends:
localrepo:
type: local
path: /mnt/restic_repo
During a hook, we can access additional information stored in some predefined variables. Here are some examples:
- AUTORESTIC_SNAPSHOT_ID
- AUTORESTIC_PARENT_SNAPSHOT_ID
- AUTORESTIC_FILES_ADDED
- AUTORESTIC_FILES_CHANGED
- AUTORESTIC_FILES_UNMODIFIED
- AUTORESTIC_LOCATION
Conclusions
In this tutorial we saw how to orchestrate and organize Restic backups with the autorestic wrapper. Autorestic let us backup multiple locations to multiple backends by creating and populating a yaml configuration file. We saw how to install autorestic on the most used Linux distributions, and how to configure locations and backends. We also learned how to launch a backup and how to attach custom commands to the hooks provided by autorestic. To know more about autorestic you can take a look at the official documentation.
