Periodic backup of Rpi3 Stretch SD card

Question

Starting with a pristine install of Raspbian Stretch on a 16GB SD card, I plan to set up various sets of software packages. I would like to start with a baseline install and create a restore-point (baseline backup) so I can frequently revert back to that state, and iterate on to other sets of software packages.

On Windows, I rely on Win32DiskImager to backup the SD card and restore on to it. I was looking for a way to do the same using dd on Linux. But I see multiple /dev/loop* in the output of a sudo fdisk -l command and I am not sure what represents the full SD card in order to backup and restore.

If I run the sudo fdisk -l command on the running RPi, I see multiple /dev/ram* disks and a /dev/mmcblk0 disk. Again, I am a bit lost on what I should specify for the if parameter of the dd command.

Updated Oct 20 2019: I appreciate the answers and comments. It appears that the one tool I have been able to use quite reliably as well as trust for backing up an SD card with a Raspbian install is still Win32DiskImager. Backing up a Live OS is not a strong preference for my requirement, and I am mainly looking for a holistic procedure that works well on a non-Windows platform, specifically Linux. Insert and back up the SD card as a whole, for a future restore is my main goal.

Answer 1

There are many questions on this site asking how to backup a Raspberry Pi.

The most important thing is to actually perform backups, and the faster and easier the process the better. I have used the following 4 methods (although there are others)

1. Create a disk image

Advantages:

• This can be done (off-line) on any computer system (Windows, macOS or Linux) although the methods differ.
• Can backup any type of SD Card even NOOBS or other multi OS systems.

Disadvantages:

• The image files created are LARGE (as large as the SD Card) and contain useless data, although they generally compress well.
• The image can only be restored to a SD Card of the same size or larger. (Nominally identical cards often differ in size.)

How do I backup my Raspberry Pi?

2. rsync file copy

These techniques (there are many variants) performs a file by file copy retaining Linux attributes (links, permissions etc.). Variants exist to copy a live system or an off-line SD Card.

Advantages:

• This only copies essential files (if properly configured) so makes smaller "images".
• The copy can be faster, especially if the previous backup is incrementally updated.

Disadvantages:

• The process CAN be slow (especially for large systems).
• The backup medium NEEDS to understand Linux file systems.
• The process does not retain partition information, so needs to be restored to a correctly formatted image.

https://raspberrypi.stackexchange.com/a/5492/8697 discusses this technique, and is recommended reading.
https://raspberrypi.stackexchange.com/a/28087/8697

3. SD Card Copier

This is a tool included in newer Raspbian images which copies a live OS to a SD Card. This is a hybrid technique which creates an image then uses file copy to populate the image.

Advantages:

• Can copy to any size SD Card (larger or smaller - assuming the Card is large enough)

Disadvantages:

• Can be very slow as it performs a fresh copy each time, and file copy is inherently slower than block copy.
• Mainly applicable to conventional Raspbian images, although it can copy multi OS systems only the last partition size is adjusted.
• SD Card Copier seems to be intolerant of SD Card defects and works better with fresh cards.

4. Create an image of a running system

This is a hybrid technique which creates an image then uses rsync to populate the image. It is useful for creating smaller images of a system or backup images which can easily be customised.

Advantages:

• Can create an image on any storage medium which allows large files and is supported by Pi e.g. ext4, exFAT or a network drive.
• Can create full size images, minimum size images or images of arbitrary size.
• Incremental backups are fast and simple.

Disadvantages:

• Only applicable to conventional 2 partition images e.g. Raspbian.

Backup image of SD Card

General Observations

There is a potential risk performing a backup of an active system, although the risk is small. Obviously you would not perform a backup during an upgrade or other process which is likely to require extensive filesystem changes. Most of these techniques can be adapted for off-line use.

Modern Linux systems utilise UUID (PARTUUID for Raspberry Pi systems) to identify partitions. It is a good idea to ensure each image has a unique PARTUUID and there are tools to perform this. (SD Card Copier has an option to perform on copy.)

There are a number of tools which can shrink an OS Image.

Answer 2

There is a solution that seems to fit ideal to your needs. You can use the Logical Volume Manager lvm2. I use it since about two years for my test setups on Raspberry Pi. With it you can take snapshots of the running system and take backups from the snapshot without worry about changes during backup. You can also revert to the snapshot so you can always start over again from the snapshot. You only have to take some effort one time to prepare and install lvm2. But you should know that there is a little disadvantage.

lvm isn't supported out of the box by the Raspbian kernel. The lvm-driver isn't compiled into the kernel. You have to load it as module. To access the root partition on boot up you have to load the driver at a very early boot up stage using an init ramdisk. This makes kernel and firmware updates a bit complicated. You have to monitor such updates manual and update the init ramdisk before booting with the new kernel.

With lvm you can just take a snapshot, mount and backup it to e.g. an usb-drive /dev/sda1:

rpi ~$ sudo lvcreate --snapshot --name rpi_base.lv --size 3g rpi.vg03/root.lv
  Logical volume "rpi_base.lv" created.

rpi ~$ sudo lvs
  LV          VG       Attr       LSize Pool Origin  Data%  Meta%  Move Log Cpy%Sync Convert
  root.lv     rpi.vg03 owi-aos--- 3.00g
  rpi_base.lv rpi.vg03 swi-a-s--- 3.00g      root.lv 0.01

rpi ~$ sudo mkdir /mnt/backup
rpi ~$ sudo mount /dev/rpi.vg03/rpi_base.lv /mnt/backup
rpi ~$ sudo mount /dev/mmcblk0p1 /mnt/backup/boot   # mount boot partition

rpi ~$ ls /mnt/backup
bin/  boot/  dev/  etc/  home/  lib/  lost+found/  media/  mnt/  opt/  proc/  root/  run/  sbin/  srv/  sys/  tmp/  usr/  var/

# create the backup
rpi ~$ sudo tar -czf /dev/sda1/rpi-backup.tar.gz -V "RPi backup from a snapshot" -C /mnt/backup ./

rpi ~$ sudo umount /mnt/backup/boot
rpi ~$ sudo umount /mnt/backup
rpi ~$ sudo lvremove rpi.vg03/rpi_base.lv

Or if you want to revert to the snapshot you can just do:

rpi ~$ sudo lvconvert --merge rpi.vg03/rpi_base.lv

and reboot. The snapshot has been removed so you should recreate it immediately after login.

You can create as much snapshots as you like but you have to know that the disk driver has to write additional data for each snapshot. With one snapshot it has to write data two times to the storage. So too much snapshots will slow down write access, but two, maybe three snapshots should do on a Raspberry Pi. It's your try.

How to setup lvm2 you can look at Easy backups and snapshots of a running system with LVM.