Retrieve data from several Raspberry Pi disk images

Question

I have about 40 Raspberry Pi SD-Card backups, dating back several years, with different projects, some of which I barely remember. As they are clogging up my harddrive (400+ GB), I would like to reduce them to a minimum.

The files were mostly created with Win32 Disk Imager, a few maybe with dd.

I would prefer no to write them back to SD card, boot them and check what is on them.

My idea was to mount them in Ubuntu, and retrieve the most common files and directories.

I am mostly interested in:

the home directory (/home/pi , /home/*)
including the bash history (~/.bash_history)

I wrote this untested workflow, but I am not experienced with fdisk, grep and bash variables:

#!/bin/bash
for filename in /RPi_images/*.img; do
    # creating a filedir for the backed up data
    mkdir /RPi_data/$filename

    # retrieving boot start off the second partition
    bootstrt = $(fdisk -l $filename 2>&1 | grep -o 'img2\s*[0-9]*' | awk '{print $2}')

    # multiplying boot start with sector size
    # assuming every image has sectors of 512 bytes
    offst = $bootstrt * 512 

    # mount the partition
    sudo mount -v -o offset=$offst -t ext4 $filename ~/rpi_mnt

    # copy all files from home dir to back up folder 
    cp -a ~/rpi_mnt/home/. /RPi_data/$filename/home
done

Output of fdisk:

Disk /RPi_images/150101_project1.img: 7.4 GiB, 7948206080 bytes, 15523840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xbbda3dc5

Device                                                   Boot Start      End  Sectors  Size Id Type
/RPi_images/150101_project1.img1       8192    93236    85045 41.5M  c W95 FAT32 (LBA)
/RPi_images/150101_project1.img2      94208 15523839 15429632  7.4G 83 Linux

How do I manage to get the variables $bootstrt $offstcorrect?
What else would you recommend an average pi user to back up?

This is a general Linux question. The normal approach would be to loop mount the image. I could post some code I use later. — Milliways, Jan 12 '20 at 22:33
@jsotola Yes, 7-zip is definitely the easier way to go here. I did not know it works! Thanks! — Markus, Jan 14 '20 at 15:41
i have a windows computer ... 7-zip lets you browse the contents of a lot of file formats ... you can even use it to extract images and icons that are embedded in exe files — jsotola, Jan 14 '20 at 17:23

Ingo · Answer 1 · 2020-01-14T21:43:47.927

How do I manage to get the variables $bootstrt $offstcorrect?

You do not need them. You should losetup give a try. It is for beginners and will mount your images automagically. You do not have to fiddle with error prone offsets and it will simplify your script very much. For example I have used the Raspbian Buster Lite image:

pc ~$ sudo losetup --find --partscan --show 2019-09-26-raspbian-buster-lite.img
/dev/loop0

pc ~$ ls /dev/loop0*
/dev/loop0  /dev/loop0p1  /dev/loop0p2

pc ~$ sudo mkdir /mnt/p2
pc ~$ sudo mount /dev/loop0p2 /mnt/p2
pc ~$ sudo mount /dev/loop0p1 /mnt/p2/boot

You can mount how you like but I have mounted the partitions as they occur on the running system. If doing that you must have attention to the order of the mounts because the mountpoint for the boot partition (in this example) is only available after the first mount.

If you like you can attach another image:

pc ~$ sudo losetup --find --partscan --show another.img
/dev/loop1
pc ~$ ls /dev/loop1*
/dev/loop1  /dev/loop1p1  /dev/loop1p2

What else would you recommend an average pi user to back up?

I would suggest just to create a compressed archive. So you do not have to worry about to something forget. It will reduce the backup a lot because you only save the files and not the hole space of the image.

pc ~$ sudo tar -czf 2019-09-26-raspbian-buster-lite.tar.gz -V "Backup of Raspbian Buster Lite from mounted image" -C /mnt/p2 .   # dot as last character

Check with:

pc ~$ sudo tar -tvf 2019-09-26-raspbian-buster-lite.tar.gz

With unmount have attention to the reverse order:

pc ~$ sudo umount /mnt/p2/boot
pc ~$ sudo umount /mnt/p2

Don't forget to detach all images:

pc ~$ sudo losetup --detach-all

Milliways · Answer 2 · 2020-01-19T00:07:18.047

I use the following to periodically Backup SD Card This uses losetup to mount a remote image and rsync to update the backup.

My script is based on backup tool, although this has since changed.

It is relatively simple to loop mount a partition in an image. The package includes an image-mount utility which can be used to mount images on the Pi (or any Linux machine).

Based on Ingo's Answer I have modified one of my scripts to mount images without calculating offset.

#!/bin/bash
# 2020-01-19
# Mount partitions on an OS Image
# Assumes only 2 partitions boot & root exist in Image

BOOT_MOUNT='/mnt/SDA1'
ROOT_MOUNT='/mnt/SDA2'
IMG="$1"

errexit()
{
  echo ""
  echo "$1"
  echo ""
  exit 1
}

if [[ "${IMG}" = "" ]]; then
    errexit "Usage: $0 imagefile"
fi
if [ $(id -u) -ne 0 ]; then
    errexit "$0 must be run as root user"
fi

# Check/create Mount Points
if [ ! -e $BOOT_MOUNT ]; then
    mkdir $BOOT_MOUNT
fi
if [ ! -e $ROOT_MOUNT ]; then
    mkdir $ROOT_MOUNT
fi

# Check Mount Points
if $(mountpoint -q $BOOT_MOUNT); then
    errexit "Mountpoint $BOOT_MOUNT already in use"
fi
if $(mountpoint -q $ROOT_MOUNT); then
    errexit "Mountpoint $ROOT_MOUNT already in use"
fi


# Loop mount all partitions on Image
LOOP="$(losetup --find --partscan --show ${IMG})"
if [ $? -ne 0 ]; then
    errexit "Unable to create loop device"
fi

SD1=$LOOP
SD2=$LOOP
SD1+="p1"   # partition containing boot files
SD2+="p2"   # partition containing OS files

# Mount Partitions 1 & 2
if ! $(mountpoint -q $BOOT_MOUNT); then
    mount $SD1 $BOOT_MOUNT  # mount partition containing boot files
fi
if ! $(mountpoint -q $ROOT_MOUNT); then
    mount $SD2 $ROOT_MOUNT  # mount root partition containing OS files
fi

echo "When finished, run:"
echo "sudo umount ${BOOT_MOUNT} ${ROOT_MOUNT}"
echo "sudo losetup -d ${LOOP}"
echo ""

Thank you. This looks like a very sophisticated solution, for more advanced users. — Markus, Jan 14 '20 at 15:43

score 1 · Accepted Answer · answered Jan 13 '20 at 14:29

How do I manage to get the variables $bootstrt $offstcorrect?

IMHO, Your initial approach is good. I would just avoid using an hardcoded value for the sector size, just in case of ... You maybe could use sfdisk instead of fdisk: it has been developped to ease scripting. Oh, and:

To do arithmetic with bash, use the syntax offst=$((bootstrt*512)) (aka double-parenthesis)
You do not umount the filesystem at then end of your for loop... You'll probably encounter some problems ;-)
Instead of using the same mountpoint (~/rpi_mnt), which may cause problems if the previous run didn't umount correctly, i would instead use one temporary folder per iteration (mktemp -d ~/rpi_mountpoint.XXXX) and remove it at the end.

What else would you recommend an average pi user to back up?

If your project invovled GPIO and electronic stuff, i would backup /boot/config.txt too (for the dt-overlays stuff, etc...). Maybe a backup of /etc/ may be a good idea too.

Thank you! I marked this as the answer, as it is spot-on and answering all the initial questions. The solution by @Milliways seems to be a more advanced approach. — Markus, Jan 14 '20 at 15:44

Retrieve data from several Raspberry Pi disk images

3 Answers3