I am trying to back up a 16GB SD card as it is proposed here: How do I backup my Raspberry Pi?
I have started with 2017-01-11-raspbian-jessie-lite.img and then did some work with FFmpeg (cloned & compiled the source code). The compilation takes quite a long time so I need to create a backup image to go back if needed.
Both dd and win32 Disk Imager generate a ~16GB .img file.
Question: Is there a way to generate a smaller file and not something that reaches the size of the SD card?
Surprise! creating an image of a 16GiB SD Card creates a 16GiB image.
There are 2 options;
only backup files can-a-raspberry-pi-be-used-to-create-a-backup-of-itself
or compress the image https://raspberrypi.stackexchange.com/a/29085/8697 - this is a macOS script, but you can develop similar for other OS.
It's worse than you think == chances are your '16Gb' .img WONT WRITE BACK to another '16Gb' SD card !
1) The stupid Pi system autoexpands to every last block of the SD card on FIRST BOOT !! This is stupid because :-
2) No two SD cards have the same 'block count'. All cards have 'bad blocks' that are 'mapped out' during manufacturing. As compitition drives prices down, SD cards with more and more bad blocks are being 'pssssed' - so 'todays' '16Gb' card will be 15.9 or 15.8 or 15.7 or even less 'tomorrow' ...
3) Win32 disk imager WONT WRITE a 15.90 Gb .img to a 15.8999999 Gb card, full stop. Even if it did, the partition would 'run off the end of the card' and the Pi will abort with 'Corrupted file system' errors
a) You have to reduce the partition to 15.5 or so to 'guarantee' it will write back (BEST way is to MODIFY the .img on your PC BEFORE burning to SD so it doesn't 'autoexpand' on first boot-up (remove ' init= ..' from end of cmdline.txt on the FAT32 partition). Then, manually change /usr/lib/raspi-config/init_resize.sh so it ewxpands to 95% of the card, not 100%
[find 'TARGET_END=$((ROOT_DEV_SIZE - 1))', change to
'TARGET_END=$((ROOT_DEV_SIZE / 100 \* 95 ))' ]
then add ' init=/usr/lib/raspi-config/init_resize.sh' back to cmdline.txt and reboot .
b) Finally, DONT USE win32 imager to make your back-up .img (it will grab every block, you only want to the end partition, not the whole SD card).
Instead (on the PC) use (free) DiskInternals Linux Reader .. (after which Win32 Imager will 'burn' the .img just fine to any card witb at least 95% of the space of the original)
Please note == This is what I ACTUALLY DO (with my PoE headless Pi Zero projects), and I can assure I have written my .img's to many different SD cards and it ACTUALLY WORKS (it seems that most 'guides' on 'How to back-up the Pi' are written by people who have never actually tried to 'restore' their .img to a new SD card :-) )
PS - or, if you are using a Pi B3, by all means start by 'shrinking' the end partition and then add a USB SD card reader/writer and play with 'dd' ... I don't do this because all my projects are based on the Pi Zero (which I usually run 'headless' with a USB/Ethernet 'dongle' on the end of a PoE cable)
Both DD and the disk imager will 'naively' just create an exact replica of the SD card, empty space and all hence why they are 16GB. These backups you can simply image directly onto SD card and its back where you left off.
If you want to snapshot your compilation process outcomes you might want to look at alternatives to entire backups. Using Docker containers would let you incrementally build on what you have done by committing the containers at each step and only pay for (in storage terms) the differences between each version. There is a recent Raspbian base image on the docker hub for the Pi which should give you an accurate enough base system, you can even copy your existing work directly into it to avoid starting from scratch. If you want to backup your work now, you only have to export your docker containers which will be Base Image size + changes, probably under 1gb. This way you can test anyway and if it goes south just bin the image and return to a known good state.
Since nobody mentioned it yet, here is the tool I use to shrink images obtained by dd: https://github.com/Drewsif/PiShrink
The shrunk images are autoexpanded at first boot to cover the SD card they are dd'ed onto by default, which can be disabled.
I recommend using Partimage to create a backup of the partitions on your your SD card. From the documentation:
Partimage will only copy data from the used portions of the partition.
Partimage can also compress its output on-the-fly, to make the resulting output even smaller.
If you are making the copy on an x86 system, you can use Clonezilla instead. It will use a variety of tools to make an image of your entire SD card.
An easy approach if you have less than about 7GB in use on your 16GB card is to use the "Copy SD Card" function to copy to an 8 GB SD card. That will give you an 8 GB image instead of 16. Other answers will give you a smaller image file, but this is the easiest.
ApplePi-Baker (Mac only) now supports shrinking .img files during SD card backup (and expansion during restore): ApplePi-Baker v2 – Backup & Restore SD cards, USB drives, etc. - Shrink IMG Process
Unfortunately the .img is expanded prior to writing back to the memory card, so it actually takes longer overall - but at least the .img backup file is smaller for storage and transmission.
