Sean Hammond, Apr 2022

How I Use Restic to Back up My Home Folders to Backblaze B2

My guide to using the open source backup program restic to back up your home folders to Backblaze B2 cloud storage.

See also:

• restic’s documentation, including the section on B2
• Backblaze B2’s own restic guide

• Backing up a new machine for the first time
  • 1. Create a B2 bucket
  • 2. Create an application key for the B2 bucket
  • 3. Install pass
  • 4. Insert restic settings into pass
  • 5. Generate a RESTIC_PASSWORD and save it in Bitwarden and pass
  • 6. Create an env script
  • 7. Source the env script
  • 8. Install restic
  • 9. Initialize a restic repo in the B2 bucket
  • 10. Create a backup script
  • 11. Make the backup script executable
  • 12. Do the initial backup
• Updating a backup
• Interrupting and resuming a backup
• Limiting bandwidth usage
• Backing up just one file to the same repo
• Doing a dry run with restic backup --dry-run
• Comparing two snapshots with restic diff
• Restoring files from your backup
  • Listing snapshots with restic snapshots
  • Listing files in snapshots with restic ls
  • Restoring files from a snapshot with restic restore
    • Restoring from the “latest” snapshot
• Maintaining your backup
  • Deleting snapshots with restic forget and restic prune
  • Verifying a backup with restic check
  • Create a maintenance script
• Accessing another host’s backups
• restic’s built-in help
• Troubleshooting
  • “Please specify repository location” error
  • “unable to create lock in backend” error
• TODO
  • restic find
  • restic mount
  • What if restic check finds a problem?

Backing up a new machine for the first time

1. Create a B2 bucket

All files in B2 are contained in buckets, which are like the top-level folders in your B2 account. Before you can back anything up to B2 you need to create a bucket: Sign in to the Backblaze website, navigate to the B2 Cloud Storage Buckets page, and click the Create a Bucket button:

Creating a B2 bucket.

Bucket names must be unique across all of B2, including the names of other user’s buckets! Hence the random numbers at the end of the bucket name.

Make sure the bucket is set to Private.

You can leave B2’s default encryption disabled: restic will encrypt your files for you.

One-to-one mapping between buckets and repositories

You’ll see below that I create one restic repository in each B2 bucket, at the root of the bucket. That’s why my RESTIC_REPOSITORY environment variable contains just a bucket name with no path (b2:restic-seanh-laptop-79539).

Alternatively you could create the restic repository in a subdirectory within the bucket. Just add a path to the end of your RESTIC_REPOSITORY like this: b2:bucketname:path/to/repo. This would allow a single B2 bucket to contain multiple restic repositories at different paths, and I think B2 application keys can be given access restricted to subpaths of buckets so different hosts don’t have access to each other’s backups.

I find it simpler to create a separate bucket for each restic repo. A B2 account is limited to 100 buckets so if you have a lot of repos you might want to try sharing buckets.

Shared versus separate repos

When backing up multiple computers they can all back up to the same restic repo or each computer can back up to its own separate repo.

Most of the time I create a separate B2 bucket and restic repo for each machine because it’s more secure: when machines share a restic repo they have full read write access to each other’s backups. Using separate repos also avoids locking: machines can back up to the same repo simultaneously but if one machine is running certain maintenance commands like restic prune or restic check then other machines can’t back up to (or run maintenance commands on) the same repo at the same time.

My main desktop and laptop do share the same repo however. This is more storage efficient because restic will deduplicate files across the two different hosts: if the desktop and laptop both contain copies of the same file only one copy of that data will actually be stored in the restic repository. In this case I think the security risk is acceptable: these machines have a lot of files in common so the deduplication might be significant. There’s already all sorts of shared sensitive files that I have on both of these machines anyway (my GPG and SSH keys, log ins to my Bitwarden account, etc). Both machines use full disk encryption. To avoid locking I run all my maintenance commands from one host.

When sharing a single restic repo between multiple hosts I create a separate application key (in Backblaze) and a separate RESTIC_PASSWORD (using the restic key command, see restic help key) for each machine. This means each machine’s access can be revoked individually if that machine is compromised or is no longer using the repo.

2. Create an application key for the B2 bucket

Restic needs a B2 application key to read and write to the bucket. Navigate to the Application Keys page and click the Add a New Application Key button:

Creating an application key.

In the Allow access to Bucket(s) dropdown select only the bucket you just created.

Type of Access must be set to Read and Write.

Copy the applicationKey that’s shown after you click Create New Key, you’ll need it later:

Copy the application key for later.

3. Install pass

We’re going to use pass to save encrypted copies of the application key and other restic settings on the host machine that’s going to be backed up. On the host machine:

1. Install pass.

   On Ubuntu:

   $ sudo apt install pass
   

   On macOS install Homebrew and then run:

   $ brew install pass
   

2. Create a GPG keypair for pass to encrypt things with:

   $ gpg --full-generate-key
   ...
   public and secret key created and signed.
   
   pub   rsa3072 2022-04-03 [SC]
         6346C69761B1155E228D488F8C23BC2681C72FBA
   uid                      Sean Hammond (GPG key for `pass` on seanh-laptop)
   sub   rsa3072 2022-04-03 [E]
   

   Take note of the key’s UID (the part beginning 6346... in my example), you’ll need it for the next step. (You can always retrieve it with gpg --list-secret-keys).

3. Initialize the password store with the GPG key:

   $ pass init <GPG_KEY_ID>
   

4. Insert restic settings into pass

Insert the B2 bucket name, application key ID, and application key value into pass:

$ pass insert "$(hostname)"/RESTIC_REPOSITORY
<When prompted enter "b2:" followed by the name of the B2 bucket you created.>
<Example: b2:restic-seanh-laptop-79539>

$ pass insert "$(hostname)"/B2_ACCOUNT_ID
<When prompted paste in the keyID of the application key that you created.>

$ pass insert "$(hostname)"/B2_ACCOUNT_KEY
<When prompted paste in the secret value of the application key you created.>

We’re inserting the settings into a password store subfolder named after your hostname so that the password store can potentially be extended to hold settings for multiple restic repos, which we’ll be doing later.

5. Generate a RESTIC_PASSWORD and save it in Bitwarden and pass

restic also needs a RESTIC_PASSWORD setting which is the password it’ll use to encrypt your backups. I use the cloud password manager Bitwarden to back up the RESTIC_PASSWORD:

1. Use pass to generate a password and save it in the local password store:

   $ pass generate "$(hostname)"/RESTIC_PASSWORD
   

2. Also copy-paste the password into Bitwarden as a backup. pass can copy the password to the clipboard for you:

   $ pass show --clip "$(hostname)"/RESTIC_PASSWORD
   

Don’t lose your RESTIC_PASSWORD!

RESTIC_PASSWORD is a restic thing, Backblaze doesn’t know anything about it. While RESTIC_REPOSITORY, B2_ACCOUNT_ID and B2_ACCOUNT_KEY can all be retrieved or regenerated from the Backblaze website, RESTIC_PASSWORD can’t be. And since RESTIC_PASSWORD is what’s used to encrypt your backups, if you lose your RESTIC_PASSWORD you’ve lost your backups. This is why I save a copy of each RESTIC_PASSWORD in Bitwarden as well as in pass on the host machine.

6. Create an env script

Create a ~/.restic folder (mkdir -p ~/.restic) and create a ~/.restic/env.bash script to load the restic settings from pass into environment variables:

#!/bin/bash
set -euo pipefail

RESTIC_HOST="$(hostname)"
RESTIC_PATH="$HOME"
RESTIC_REPOSITORY="$(pass "$RESTIC_HOST"/RESTIC_REPOSITORY)"
B2_ACCOUNT_ID="$(pass "$RESTIC_HOST"/B2_ACCOUNT_ID)"
B2_ACCOUNT_KEY="$(pass "$RESTIC_HOST"/B2_ACCOUNT_KEY)"
RESTIC_PASSWORD_COMMAND=pass\ "$RESTIC_HOST"/RESTIC_PASSWORD

export RESTIC_HOST RESTIC_PATH RESTIC_REPOSITORY B2_ACCOUNT_ID B2_ACCOUNT_KEY RESTIC_PASSWORD_COMMAND

RESTIC_HOST and RESTIC_PATH

RESTIC_HOST and RESTIC_PATH aren’t environment variables recognised by restic itself. We’re exporting them because we’ll use them in our backup and maintenance scripts later on.

fish-shell

If you use fish-shell you might also want to create a ~/.restic/env.fish script that you can source from fish:

#!/usr/bin/fish

set -x RESTIC_HOST (hostname)
set -x RESTIC_PATH $HOME
set -x RESTIC_REPOSITORY (pass $RESTIC_HOST/RESTIC_REPOSITORY)
set -x B2_ACCOUNT_ID (pass $RESTIC_HOST/B2_ACCOUNT_ID)
set -x B2_ACCOUNT_KEY (pass $RESTIC_HOST/B2_ACCOUNT_KEY)
set -x RESTIC_PASSWORD_COMMAND pass $RESTIC_HOST/RESTIC_PASSWORD

7. Source the env script

Now you can source the env.bash script from a bash shell and any subsequent restic commands in that shell session will use your B2 bucket, application key, and restic password without having to pass them as options to each command.

The rest of the commands in this post will assume you’ve sourced the env.bash script, so do that now:

$ source ~/.restic/env.bash

The script can also be sourced from other bash scripts, which we’ll do below.

8. Install restic

To install restic on Ubuntu:

$ sudo apt install restic
$ sudo restic self-update

$ # Install bash autocompletions for restic subcommands and options.
$ mkdir -p ~/.local/share/bash-completion
$ restic generate --bash-completion ~/.local/share/bash-completion/restic

The autocompletions enable you to press Tab after typing restic to autocomplete restic subcommands, or press Tab after -- to autocomplete restic options.

On macOS install Homebrew and then run:

$ brew install restic

fish-shell

If you use fish-shell you might also want to install fish-shell autocompletions:

$ mkdir -p ~/.config/fish/completions
$ restic generate --fish-completion ~/.config/fish/completions/restic.fish

9. Initialize a restic repo in the B2 bucket

Initialize a restic repository (repo) in the B2 bucket:

$ restic init
created restic repository 191d720a00 at b2:restic-seanh-laptop-79539

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

If you browse your bucket in Backblaze you should see that restic has created some housekeeping files in it:

10. Create a backup script

Create a ~/.restic/backup script that runs a restic backup command to back up your home directory.

You’ll want to pass some command line options to restic backup, consider:

--compression max

Increase compression to save storage space on your backup location, at the cost of using more CPU.

--exclude-caches

Excludes directories that are marked as caches by having a CACHEDIR.TAG file.

--one-file-system

Excludes any other file systems that’re mounted within your home directory.

--cleanup-cache

Removes old cache directories after running.

--exclude <FILE>

Excludes unwanted files from the backup. --exclude can be given multiple times, and <FILE> can be a pattern. There are other options for excluding or including files as well, see Excluding Files in the restic docs for details. You might want to use Disk Usage Analyzer (baobab) to scan your home directory for large files or folders that you don’t want to back up.

See restic help backup for the rest of the available options.

Here’s an example backup script with lots of --exclude’s for Linux and macOS directories that you probably don’t want to back up:

#!/bin/bash
set -euo pipefail

source ~/.restic/env.bash

restic backup \
    --compression max \
    --exclude-caches \
    --one-file-system \
    --cleanup-cache \
    --exclude "$HOME/Downloads" \
    --exclude "$HOME/Library" \
    --exclude "$HOME/snap" \
    --exclude "$HOME/.Trash" \
    --exclude "$HOME/.bundle" \
    --exclude "$HOME/.cache" \
    --exclude "$HOME/.dbus" \
    --exclude "$HOME/.dropbox" \
    --exclude "$HOME/.dropbox-dist" \
    --exclude "$HOME/.local/pipx" \
    --exclude "$HOME/.local/share/Trash" \
    --exclude "$HOME/.npm" \
    --exclude "$HOME/.pyenv" \
    --exclude "$HOME/.thumbnails" \
    --exclude "$HOME/.virtualenvs" \
    --exclude "node_modules" \
    --exclude ".tox" \
    "$RESTIC_PATH"

11. Make the backup script executable

$ chmod u+x ~/.restic/backup

12. Do the initial backup

Finally, to backup your home directory to B2 just run:

$ ~/.restic/backup

This will take a long time to run. Once it’s finished you’ll see more files in your B2 bucket:

Updating a backup

To incrementally update a machine’s backup, uploading only new or modified files, just re-run the same backup script:

$ ~/.restic/backup

Interrupting and resuming a backup

You can interrupt a backup with Ctrl + c and later resume it by re-running the same ~/.restic/backup command. See Will restic resume an interrupted backup? in restic’s FAQ.

Limiting bandwidth usage

You can limit restic’s speed so it doesn’t hog all your bandwidth when you’re updating a backup, just add the --limit-download and --limit-upload options. You might not want to do this for your initial backup, but once your first backup has completed consider adding these to your backup script:

$ restic backup --limit-download 5600 --limit-upload 1800 ...

The values are the maximum rates in KiB/s.

Other commands like restic check (for verifying a backup) and restic restore (for restoring a backup) also support the the --limit-download and --limit-upload options.

Backing up just one file to the same repo

You can run a one-off command to quickly back up one or more specific files or directories to the same repo, without doing an incremental backup of the whole machine. Just pass the file path(s) to restic backup. This while create a separate snapshot that contains only the given file(s):

$ restic backup --verbose=3 foo.txt
open repository
repository d59c3d15 opened successfully, password is correct
lock repository
load index files
no parent snapshot found, will read all files
start scan on [foo.txt]
start backup on [foo.txt]
scan finished in 28.574s: 1 files, 4 B
new       /foo.txt, saved in 0.004s (4 B added)

Files:           1 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Data Blobs:      1 new
Tree Blobs:      1 new
Added to the repo: 384 B

processed 1 files, 4 B in 0:35
snapshot c6d56f86 saved

Doing a dry run with restic backup --dry-run

You can always test what a restic backup command would do by passing the --dry-run and --verbose arguments. This can be good for testing whether your exclude options will do what you expect, or checking how much data will be added to the backup before actually running it:

$ restic backup --dry-run --verbose=3 foo.txt
open repository
repository d59c3d15 opened successfully, password is correct
lock repository
load index files
no parent snapshot found, will read all files
start scan on [foo.txt]
start backup on [foo.txt]
scan finished in 8.994s: 1 files, 4 B
new       /foo.txt, saved in 0.003s (0 B added)

Files:           1 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Data Blobs:      0 new
Tree Blobs:      1 new
Would add to the repo: 380 B

processed 1 files, 4 B in 0:09

Comparing two snapshots with restic diff

You can use restic diff to compare the contents of two snapshots. Comparing a snapshot to the previous snapshot can be like a retrospective --dry-run, showing you what a restic backup did.

First use restic snapshots to get the IDs of two snapshots that you want to compare, then pass them to restic diff:

$ restic diff 9eb356c4 b072e1ac
-    foo.txt
+    bar.txt
M    gar.txt

Files:           1 new, 1 removed,     1 changed
Dirs:            0 new, 0 removed
Others:          0 new,    0 removed
Data Blobs:      1 new, 1 removed
Tree Blobs:      1 new, 1 removed
  Added:   384 B
  Removed: 51.780 GiB

In the list of files + denotes a file that was added, - is a file that was removed, M means the file’s contents were modified, U means the file’s metadata was changed and T mean’s the file’s type was changed (for example it changed from a file to a symlink).

Restoring files from your backup

Listing snapshots with restic snapshots

restic snapshots prints out a list of all the snapshots in the repo (each completed restic backup command creates a new snapshot). This is mostly useful for getting snapshot IDs to pass to commands like restic restore (see below).

$ restic snapshots
repository d59c3d15 opened successfully, password is correct
ID        Time                 Host          Tags        Paths
--------------------------------------------------------------------
1f5cfe20  2020-04-28 09:04:47  beatsworking              /home/seanh
81754433  2020-12-24 17:43:24  chamlis                   /home/seanh
27ca05c7  2021-04-29 18:20:00  chamlis                   /home/seanh
f362f638  2021-05-28 14:13:38  chamlis                   /home/seanh
ad4bf831  2021-06-10 18:32:55  chamlis                   /home/seanh
1fa5c822  2021-07-29 17:26:37  chamlis                   /home/seanh
50d3f0c2  2021-08-28 18:10:00  chamlis                   /home/seanh
2a62a5ed  2021-09-23 17:18:26  chamlis                   /home/seanh
9ae436fb  2021-10-28 17:01:38  chamlis                   /home/seanh
09f6ec02  2021-11-25 17:17:52  chamlis                   /home/seanh
053973eb  2021-12-25 19:43:58  chamlis                   /home/seanh
7939dbe7  2022-01-27 19:49:08  chamlis                   /home/seanh
43b899df  2022-02-26 16:55:29  chamlis                   /home/seanh
e377529b  2022-04-01 19:16:42  chamlis                   /home/seanh
9eb356c4  2022-04-01 20:37:36  beatsworking              /home/seanh
22a390d4  2022-04-02 23:41:36  beatsworking              /home/seanh
--------------------------------------------------------------------
16 snapshots

Listing files in snapshots with restic ls

restic ls lists files within snapshots. This is mostly useful for finding files to restore with restic restore (see below).

$ # List all files in the latest snapshot from any host.
$ restic ls latest
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in the latest snapshot from the current host.
$ restic ls --host "$(hostname)" latest
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in the latest snapshot of your home directory from the
$ # current host.
$ restic ls --host "$(hostname)" --path "$HOME" latest
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in a specific snapshot by snapshot ID.
$ restic ls 7518e078
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in the $HOME/Mail directory in the latest snapshot.
$ # When one or more directory arguments are given it only lists top-level files
$ # and folders in the given folder(s) unless you also give --recursive to tell
$ # it to recurse into subfolders.
$ restic ls --recursive latest $HOME/Mail
...

Restoring files from a snapshot with restic restore

Use restic snapshots to find the ID of the snapshot that you want to restore from. (f11c451b in the examples below). Then to restore the entire contents of the snapshot to a /tmp/restored directory:

$ mkdir /tmp/restored
$ restic restore f11c451b --target /tmp/restored

Add --verify to verify the contents of the files after restoring them.

Use --include to restore just a single file or folder from the snapshot. First use restic ls to find the paths of the files you want to restore and then run a restore command like:

$ restic restore f11c451b --target /tmp/restore-work --include /foo.txt

--include can be given multiple times to restore multiple specific files or folders at once and patterns can be used to restore all files that match a pattern. There’s also --exclude to exclude specific or matching files from the restore, restoring everything that isn’t excluded. See Restoring from backup in the restic docs or restic help restore for details.

Restoring from the “latest” snapshot

You can use latest instead of a snapshot ID to restore from the most recent snapshot:

$ restic restore latest --target /tmp/restored

latest on its own means the most recent snapshot from any host and of any path.

If you have multiple hosts backing up to the same repo and you want to restore the latest snapshot from the current host use --host "$(hostname)" with latest.

If you back up multiple paths to the same repo and you want to restore the latest snapshot of your home directory use --path "$HOME" with latest.

Here’s an example combining both:

$ restic restore --host "$(hostname)" --path "$HOME" latest --target /tmp/restored

Maintaining your backup

If your backup is large the restic prune and restic check commands can download a lot of data (potentially gigabytes) because they need to load all your indexes and snapshots. This can take a long time and cost you a lot of money in B2 download fees. For this reason you might not want to run prune and check too frequently. I run the maint script below once a year.

Deleting snapshots with restic forget and restic prune

You can delete old snapshots to save storage space. To delete a specific snapshot or snapshots get the snapshot ID(s) from restic snapshots and pass them to restic forget. For example:

$ restic forget f11c451b bea27147 f7e72151
repository d59c3d15 opened successfully, password is correct
[0:02] 100.00%  3 / 3 files deleted...

Just forgetting snapshots doesn’t actually save any space. You then have to run restic prune to find and delete the data that’s no longer used by any of the remaining snapshots:

$ restic prune
repository d59c3d15 opened successfully, password is correct
loading indexes...
loading all snapshots...
finding data that is still in use for 17 snapshots
[2:37] 100.00%  17 / 17 snapshots...
searching used packs...
collecting packs for deletion and repacking
[1:01] 100.00%  181618 / 181618 packs processed...

to repack:             0 blobs / 0 B
this removes:          0 blobs / 0 B
to delete:             5 blobs / 1.986 KiB
total prune:           5 blobs / 1.986 KiB
remaining:       5877282 blobs / 862.804 GiB
unused size after prune: 16.160 GiB (1.87% of remaining size)

rebuilding index
[7:50] 100.00%  181616 / 181616 packs processed...
deleting obsolete index files
[0:17] 100.00%  120 / 120 files deleted...
removing 2 old packs
[0:00] 100.00%  2 / 2 files deleted...
done

Restic’s docs advise you to run restic check (see below) after doing a restic prune.

To automatically select which snapshots to delete use restic forget with one or more --keep-* options. There are lots of --keep-* options, like --keep-last n to keep the last n snapshots or --keep-hourly n to keep only one snapshot per hour for the last n hours that have snapshots. See Removing snapshots according to a policy in restic’s docs for all the details. Any snapshots that don’t match one of the given --keep-* options will be deleted.

By default restic forget will delete snapshots from all hosts and of any path.

If you have multiple hosts backing up to the same repo add --host "$(hostname)" to limit restic to only deleting snapshots from the current host.

If you back up multiple paths to the same repo add --path "$HOME" to limit restic to only deleting snapshots of your home directory.

Use --dry-run to see which snapshots would be deleted before actually deleting them.

--tag <TAG> will limit restic to only deleting snapshots with the given tag(s). You can use --tag multiple times to delete snapshots with any of the given tags, for example restic forget --tag foo --tag bar ... will consider backups with either the tag foo or the tag bar for deletion. Or you can pass a comma-separated list of tags to --tag to only delete snapshots with all of the given tags, for example restic forget --tag foo,bar will only consider snapshots with both the tags foo and bar for deletion.

Use --tag '' to limit restic to only deleting untagged snapshots. This way you can protect certain snapshots from deletion by tagging them. Either use restic tag --add foo <SNAPSHOT_ID> to add a tag to an existing snapshot or pass --tag foo to the restic backup command when you first create the snapshot. For example to add the tag foo to the latest snapshot:

$ restic tag --add foo latest

See restic help tag for more documentation on managing tags. You can use restic snapshots to see which snapshots have which tags.

Here’s an example restic forget command using some of the --keep-* options:

$ restic forget --dry-run \
                --host "$(hostname)" \
                --path "$HOME" \
                --tag '' \
                --keep-within-daily 7d \
                --keep-within-weekly 1m \
                --keep-within-monthly 1y \
                --keep-within-yearly 100y

This will keep daily snapshots for the last week, weekly snapshots for the last month, monthly snapshots for the last year, and yearly snapshots for the last century. “Last week/month/year/century” are relative to the date of the latest snapshot, not to the current date. In addition it will keep all snapshots that are not from the current host, all snapshots that are not of your home directory, and all tagged snapshots. All other snapshots will be deleted (if the command is re-run without the --dry-run).

Again, you’d then need to run restic prune to actually free up the space.

Verifying a backup with restic check

The restic check command verifies the integrity of your restic repository:

$ restic check
using temporary cache in /tmp/restic-check-cache-423433534
repository d59c3d15 opened successfully, password is correct
created new cache in /tmp/restic-check-cache-423433534
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[2:02:10] 100.00%  130 / 130 snapshots
no errors were found

restic check just checks the “structural integrity and consistency” of the backup. If you want to check that the actual backed up files are correct you can use restic check --read-data but this will download all the files in the repository which can take a long time and add a lot of money to your Backblaze bill. Alternatively you can use restic check --read-data-subset=1G to check just a random one gigabyte subset of the data:

$ restic check --read-data-subset=1G
using temporary cache in /tmp/restic-check-cache-844156038
repository d59c3d15 opened successfully, password is correct
created new cache in /tmp/restic-check-cache-844156038
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[2:11:01] 100.00%  130 / 130 snapshots
read 1G of data packs
[0:06] 100.00%  1 / 1 packs
no errors were found

restic check with --read-data or --read-data-subset first checks the repo’s structural integrity and consistency (everything that a plain restic check would do) and then downloads and checks the files.

There are other ways to specify the subset to check as well, see Checking integrity and consistency in the restic docs.

Create a maintenance script

To make maintaining your backup easy create a ~/.restic/maint script that runs restic forget with some --keep-* options, then restic prune, then restic check. Here’s an example:

#!/bin/bash
set -euo pipefail

source ~/.restic/env.bash

restic forget \
    --host "$RESTIC_HOST" \
    --path "$RESTIC_PATH" \
    --tag '' \
    --keep-within-daily 7d \
    --keep-within-weekly 1m \
    --keep-within-monthly 1y \
    --keep-within-yearly 100y

restic prune

restic check --read-data-subset=1G

Make the maint script executable:

$ chmod u+x ~/.restic/maint

Now to weed out old snapshots and verify your backup you can just run:

$ ~/.restic/maint

Accessing another host’s backups

Sometimes you might want to access one host’s backups from a different host, for example to recover a different host’s files or to run maintenance commands for another host’s backups. I like to run the maintenance commands for all my backups from my main desktop machine, it saves having to run them on each laptop separately and saves the laptops from being occupied by long running commands.

An easy way to do this is to create a separate env.HOSTNAME.bash file for each host whose backups you want to access:

$ ls ~/.restic/*.bash
/home/seanh/.restic/env.bash
/home/seanh/.restic/env.beatsworking.bash
/home/seanh/.restic/env.xenocrat.bash

The env.bash file is for this host’s backups, the env.beatsworking.bash and env.xenocrat.bash files are for beatsworking and xenocrat’s backups (the hostnames of two laptops that I back up). If you use fish-shell you’ll want to create env.HOSTNAME.fish files for each host as well.

The contents of each env.HOSTNAME.bash file should be the same as the env.bash file but hard-coding the appropriate hostname and home directory instead of using $(hostname) and $HOME. Here’s an example:

#!/bin/bash
set -euo pipefail

RESTIC_HOST=xenocrat
RESTIC_PATH=/home/shan
RESTIC_REPOSITORY="$(pass "$RESTIC_HOST"/RESTIC_REPOSITORY)"
B2_ACCOUNT_ID="$(pass "$RESTIC_HOST"/B2_ACCOUNT_ID)"
B2_ACCOUNT_KEY="$(pass "$RESTIC_HOST"/B2_ACCOUNT_KEY)"
RESTIC_PASSWORD_COMMAND=pass\ "$RESTIC_HOST"/RESTIC_PASSWORD

export RESTIC_HOST RESTIC_PATH RESTIC_REPOSITORY B2_ACCOUNT_ID B2_ACCOUNT_KEY RESTIC_PASSWORD_COMMAND

For the env.HOSTNAME.bash scripts to work you’ll need to add a RESTIC_REPOSITORY, B2_ACCOUNT_ID, B2_ACCOUNT_KEY and RESTIC_PASSWORD for each host for the password store:

~> pass ls
Password Store
├── beatsworking
│   ├── B2_ACCOUNT_ID
│   ├── B2_ACCOUNT_KEY
│   ├── RESTIC_PASSWORD
│   └── RESTIC_REPOSITORY
├── chamlis
│   ├── B2_ACCOUNT_ID
│   ├── B2_ACCOUNT_KEY
│   ├── RESTIC_PASSWORD
│   └── RESTIC_REPOSITORY
└── xenocrat
    ├── B2_ACCOUNT_ID
    ├── B2_ACCOUNT_KEY
    ├── RESTIC_PASSWORD
    └── RESTIC_REPOSITORY

When I do this I create a new application key in Backblaze and add a second RESTIC_PASSWORD to the repo using the restic key command (remembering to back up the RESTIC_PASSWORD in Bitwarden) rather than copying the same key and password to two machines. This just means that each machine’s key and password can be revoked separately.

Now you can easily move between backups by just sourcing the env files and then running restic commands:

$ source ~/.restic/env.bash
$ restic check
...
$ source ~/.restic/env.beatsworking.bash
$ restic check
...
$ source ~/.restic/env.xenocrat.bash
$ restic check
...

I extended my annual maintenance script to source each env file in turn and run my maintenance commands on each repo:

#!/bin/bash
set -euo pipefail

function maint {
    # Print this so you can tell which backup restic's output belongs to.
    echo "maint $1"

    source "$1"

    restic forget \
        --host "$RESTIC_HOST" \
        --path "$RESTIC_PATH" \
        --tag '' \
        --keep-within-daily 7d \
        --keep-within-weekly 1m \
        --keep-within-monthly 1y \
        --keep-within-yearly 100y

    restic prune

    restic check --read-data-subset=1G
}

for env_file in ~/.restic/env*.bash; do
    maint "$env_file"
done

restic’s built-in help

restic has built-in reference documentation for all its subcommands and options. For a list of restic’s subcommands and global flags run restic help or restic --help:

$ restic help

For documentation of a subcommand and its options run restic help <SUBCOMMAND> or restic <SUBCOMMAND> --help, for example:

$ restic help backup

Troubleshooting

“Please specify repository location” error

If you get this error:

Fatal: Please specify repository location (-r or --repository-file)

It probably means you forgot to source your env file. Run:

$ source ~/.restic/env.bash

And then re-run your original command.

“unable to create lock in backend” error

If you get this error:

unable to create lock in backend: repository is already locked exclusively by PID ...
...
lock was created at ...
storage ID ...
the `unlock` command can be used to remove stale locks

Just run the restic unlock command, as the error message says:

$ restic unlock

And then re-run your original command.

Stale locks can happen if a restic command crashes or if a computer that was running a restic command crashes or loses power etc.

TODO

restic find

There’s a restic find command for finding files that match a pattern, searching across all snapshots at once which could be useful. It’s poorly documented and didn’t seem to behave as I expected it to. Just grepping the output of restic ls can probably get you pretty far.

restic mount

There’s a restic mount command for mounting your snapshots as a browseable FUSE filesystem. You can restore files by just copying them normally.

$ mkdir /tmp/restic
$ restic mount /tmp/restic
repository d59c3d15 opened successfully, password is correct
Now serving the repository at /tmp/restic/
Use another terminal or tool to browse the contents of this folder.
When finished, quit with Ctrl-c here or umount the mountpoint.

I found restic mount to be buggy when I tried it. It worked once, but after that the mounted filesystem kept appearing to be empty.

What if restic check finds a problem?

What should you do if restic check reports a problem with your backup? I’ve never had this happen so I don’t know how to handle it. Will restic offer to fix the problem or suggest a solution? Should you create a new repo and start again from scratch?