Is there a way to compress a backup of the RPi because the backup has empty space?

Question

I did a backup for my RPi (64GB microsd card) and it saved the entire 64GB worth .img file... while theres only ~14GB worth actual data, the image file accounts also for that empty space.

Anyone know if theres a way to remove that empty space from the .img file?

Many thanks in advance :-) EDIT: I am using Windows10

Milliways · Answer 1 · 2019-01-21T03:31:57.860

5

https://www.raspberrypi.org/documentation/linux/filesystem/backup.md explains backup and restoration.

You don't indicate what OS you are using but it is simple on any 'NIX system. I use the following on macOS

#!/bin/bash
# script to backup Pi SD card
# 2017-06-05
# 2018-11-29    optional name
# DSK='disk4'   # manual set disk
OUTDIR=~/temp/Pi

# Find disk with Linux partition (works for Raspbian)
# Modified for PINN/NOOBS
export DSK=`diskutil list | grep "Linux" | sed 's/.*\(disk[0-9]\).*/\1/' | uniq`
if [ $DSK ]; then
    echo $DSK
    echo $OUTDIR
else
    echo "Disk not found"
    exit
fi

if [ $# -eq 0 ] ; then
    BACKUPNAME='Pi'
else
    BACKUPNAME=$1
fi
BACKUPNAME+="back"
echo $BACKUPNAME

diskutil unmountDisk /dev/$DSK
echo please wait - This takes some time
echo Ctl+T to show progress!
time sudo dd if=/dev/r$DSK bs=4m | gzip -9 > $OUTDIR/Piback.img.gz

#rename to current date
echo compressing completed - now renaming
mv -n $OUTDIR/Piback.img.gz $OUTDIR/$BACKUPNAME`date +%Y%m%d`.img.gz

edited Jan 21 '19 at 03:31

answered Sep 03 '17 at 09:35

Milliways

54,718
26
92
182

I am very sorry for the late reply I am on Windows10 – Mohd Salman Sep 03 '17 at 10:07
doesn't work. bs=4m makes dd choke and changing it to 4M also doesn't help – Wolfgang Fahl Oct 19 '19 at 08:59
@WolfgangFahl "changing it to 4M also doesn't help" hardly surprising because dd on BSD Unix doesn't support this. – Milliways Oct 19 '19 at 10:26
Since my computer is a MacOS computer i had assumed things would work. – Wolfgang Fahl Oct 19 '19 at 10:58
@WolfgangFahl: Thanks for your comment as I was considering using this, but it seems to have issues? Did you ever get it to work? If so, would you post your approach as another answer? – Seamus Jan 27 '20 at 18:08
@Milliways: You never responded to Mr. Fahl's comment, or revised your script. Does this work on macos or not? – Seamus Jan 27 '20 at 18:13
@Seamus I did respond but what else can you say to "i had assumed things would work" - I have been using this on several Macs with many releases of macOS for years. I still occasionally use it but now incrementally backup live images with https://raspberrypi.stackexchange.com/a/103991/8697 (which is faster and more efficient). – Milliways Jan 27 '20 at 23:32
@Milliways: That's a fair point :) And I only asked because he said he was using macos also... your reference to BSD confused me. But no worries, and thanks for the update. – Seamus Jan 27 '20 at 23:52
@Seamus `macOS` is an accredited implementation of BSD Unix (admittedly with Apple layers on top). – Milliways Jan 27 '20 at 23:57
@Milliways: "accredited implementation"... did not know such accreditation existed!? Without trying to get into 'OS politics', that does seem odd given that Apple is still shipping 15 year-old software (due to licensing issues) with its BSD accreditation. Maybe that's the GNU stuff, but BSD did "derive" some of their code from GNU at some point in time. And I guess that Apple followed in BSD's footsteps recently in that they ditched `bash` as their default shell. Interesting stuff, but much of that licensing brouhaha strikes me as unproductive wrangling (no politics - just my opinion). – Seamus Jan 28 '20 at 03:31

score 5 · Answer 2 · edited Jun 29 '19 at 23:27

Already answered here dd-on-entire-disk-but-do-not-want-empty-portion

Assuming you want to save /dev/sdXN to /tgtfs/image.raw and you are root:

mkdir /srcfs && mount /dev/sdXN /srcfs

Use zerofill or just:

dd if=/dev/zero of=/srcfs/tmpzero.txt

To fill unused blocks with zero (wait for it to fill the file system completely then

rm /srcfs/tmpzero.txt

Take the image with dd and use conv=sparse to punch zeros on-the-fly:

dd conv=sparse if=/dev/sdxn of=/tgtfs/image.raw

If you want to use compression you don't need to punch the zeros with dd as zero blocks are highly compressible:

dd if=/dev/sdxn | gz -c | dd of=/tgtfs/image.raw

PS: You should note that this is not a good idea on a flash memory based storage media (i.e. your source file system be SSD).

flakeshake · Answer 3 · 2017-09-07T08:19:28.397

Shrink your second partition with gparted to eliminate free space and use dd to only backup the parts you need.

I suggest using a gparted Live CD (based on Linux) to modify RPi partitions. You can boot from such a disk without touching your Windows install at all. This is much safer.

You can now use dd to only backup the parts of the SD card that matter.

Any sensible compression program will automatically detect free space and collapse it efficiently.

score 0 · Answer 4 · answered Jul 10 '18 at 20:56

0

Either use raspiBackup to create the dd image and read FAQ 16 or use pishrink.

answered Jul 10 '18 at 20:56

framp

872
7
15

Is there a way to compress a backup of the RPi because the backup has empty space?

4 Answers4

Linked