shrink-backup
I made this script because I wanted a universal method of backing up my SBC:s into small img files as fast as possible (with rsync), indepentent of what os is in use.
shrink-backup is a very fast utility for backing up your SBC:s into minimal bootable img files for easy restore with autoexpansion at boot.
Can backup any device with or without a boot partition as long as the filesystem is ext4, f2fs or btrfs (with subvolumes).
Supports backing up root & boot (if existing) partitions. Data from other partitions will be written to root if not excluded.
For btrfs, all existing top level 5 subvolumes in /etc/fstab will be created with new backups, nested subvolumes will be created and can also be removed/added in an update of the backup img.
Please read Info section for more information.
Autoexpansion tested & supported on following operating systems:
- Raspberry Pi OS (trixie and older)
- Armbian
- Manjaro-arm
- DietPi
- ArchLinuxArm
- Kali-arm
- Ubuntu-server-arm (Ubuntu autoexpands by default, but that can be disabled with
-e option)
Autoexpansion does not work on f2fs due to filesystem limitations.
Other operating systems will most likely work too, but autoexpansion will not.
The script will report the operating system as “unknown” but that does not mean the script will fail.
Feel free to make a feature request if you use an operating system not on this list.
Full functionality for usage inside webmin (including “custom command” button). Thank you to iliajie for helping out. ❤️
Latest release: shrink-backup v1.4
Testing branch: If you want to use the absolute latest version. There might be bugs.
Very fast restore thanks to minimal size of img file.
Default device that will be backed up is determined by scanning what disk-device root resides on.
This means that if boot is a partition, that partition must be on the same device and before the root partition.
The script considers everything on the device before root as the bootsector.
Backing up/restoring, to/from: usb-stick /dev/sdX with Raspberry pi os has been tested and works. Ie, writing an sd-card img to a usb-stick and vice versa works.
Ultra-fast incremental backups to existing img files.
See wiki for information about installation methods, usage and examples.
Ideas and feedback is always appreciated, whether it’s positive or negative. Please just keep it civil. :)
If you find a bug or think something is missing in the script, please file a bug report or Feature request
To restore a backup, simply “burn” the img file to a device using your favorite method.
When booting a restored image with autoresize active, on some operating systems a reboot will occur after resizing is made (you will be informed at the end of the script if your operating system is affected by this), please wait until the the reboot sequence has occurred. The login prompt may very well become visible before the autoresize function has rebooted.
Usage
shrink-backup -h
Script for creating an .img file and subsequently keeing it updated (-U), autoexpansion is enabled by default
Directory where .img file is created is automatically excluded in backup
########################################################################
Usage: sudo shrink-backup [options] imagefile.img [extra space (MiB)]
-U Update existing img file (rsync to existing img)
Optional [extra space] extends img root partition
-a Autocalculate root size partition, [extra space] is ignored.
When used in combination with -U:
Expand if partition is >=256MiB smaller than autocalculated recommended minimum
Shrink if partition is >=512MiB bigger than autocalculated recommended minimum
-t Use exclude.txt in same folder as script to set excluded directories
One directory per line: "/dir" or "/dir/*" to only exclude contents
-y Disable prompts in script (please use this option with care!)
-e Disable autoexpansion on root filesystem when image is booted
-l Write debug messages to logfile shrink-backup.log located in same directory as script
-z Make script zoom at light-speed, only question prompts might slow it down
Can be combined with -y for UNSAFE ultra-mega-superduper-speed
-q --quiet Do not print rsync copy process
--no-color Run script without color formatted text
--fix Try to fix the img file if -a fails with a "broken pipe" error
Will activate rsync options --delete-before & --fsync
--rsync Define custom rsync line manually. Will print rsync line for user to edit
--loop [img] Loop img file and exit, works in combination with -l & -z
If optional [extra space] is defined, the img file will be extended with the amount before looping
NOTE that only the file gets truncated, no partitions
Useful if you for example want to manually manage the partitions
--chroot [img] Use systemd-nspawn. Loop img file, mount to temp directory, enter chroot environment and drop to shell
This will let you make changes in a chroot environment directly on the img file
For example update with package manager or rebuild initramfs
The script will keep running in the background
Type 'exit' when done. Script will unmount, remove temp directory/loop and exit
--f2fs Convert root filesystem on img from ext4 to f2fs
Only works on new img file, not in combination with -U
Will make backups of fstab & cmdline.txt to: fstab.shrink-backup.bak & cmdline.txt.shrink-backup.bak
Then change ext4 to f2fs in both files and add discard to options on root partition in fstab
--version Print version and exit
-h --help Show this help snippet
########################################################################
Examples:
sudo shrink-backup -a /path/to/backup.img (create img, resize2fs calcualtes size)
sudo shrink-backup -e -y /path/to/backup.img 1024 (create img, ignore prompts, do not autoexpand, add 1024MiB extra space)
sudo shrink-backup -Utl /path/to/backup.img (update img backup, use exclude.txt and write log to shrink-backup.log)
sudo shrink-backup -U /path/to/backup.img 1024 (update img backup, expand img size/root partition with 1024MiB)
sudo shrink-backup -Ua /path/to/backup.img (update img backup, resize2fs calculates and resizes img file if needed)
sudo shrink-backup -Ua --fix /path/to/backup.img 1024 (update img backup, automatically resizes img file if needed, fix img free space)
sudo shrink-backup -l --loop /path/to/backup.img 1024 (write to log file, expand IMG FILE (not partition) by 1024MiB and loop)
-t (exclude.txt)
The folder where the img file is created will ALWAYS be excluded in the backup.
If -t option is selected, exclude.txt MUST exist (but can be empty) within the directory where the script is located or the script will exit with an error.
If installed using curl, location and name of file is different. See install with curl for information.
Use one directory per line in exclude.txt.
/directory/* = create directory but exclude content.
/directory = exclude the directory completely.
/directory* = exclude all directories starting with name /directory.
btrfs uses another file to exclude subvolumes, see btrfs for information.
The paths must be absolute, ie ~/directory will not work.
If -t is NOT selected the following will be excluded:
/lost+found
/proc/*
/sys/*
/dev/*
/tmp/*
/run/*
/mnt/*
/media/*
/var/swap
/snap/*
Please read info section for more information.
-l (Log file)
Use -l to write debug info into shrink-backup.log file located in the same directory as the script.
Please provide this file if filing a bug report
[!NOTE]
If installed using curl, location and name of file is different. See install with curl for information.
-z (Zoom speed)
The -z “zoom” option simply removes the one second sleep at each info prompt to give the user time to read.
By using the option, you save 15-25s when running the script.
[!CAUTION]
When used in combination with -y warnings will also be bypassed! PLEASE use with care!
The script will for example overwrite files without asking for confirmation!
--fix (Broken pipe)
Add --fix to your options if a backup fails during rsync with a “broken pipe” error. This will make rsync use options --delete-before & --fsync.
You can also manually add [extra space] instead of using -Ua --fix to solve this.
Example: sudo shrink-backup -Ua --fix /path/to/backup.img
The reason it happens is because rsync normally deletes files during the backup, not creating a file-list > removing files from img before starting to copy.
So if you have removed and added new data on the system you backup from, there is a risk rsync tries to copy the new data before deleting data from the img, hence completely filling the img.
Using --fix configures rsync create a file-list and delete data before starting to transfer new data. This also means the backup takes a little longer.
Having a “broken pipe” error during backup has in my experience never broken an img backup after either using --fix or adding [extra space] while updating the backup with -U.
--loop (Loop img file)
Use --loop to loop an img file to your /dev.
This functionality works on any linux system, just use the script on any img type file (not limited to .img extension files) anywhere available to the computer.
Example: sudo shrink-backup --loop /path/to/backup.img
If used in combination with [extra space] the amount in MiB will be added to the IMG FILE, NOT any partition.
Example: sudo shrink-backup --loop /path/to/backup.img 1024
With this you can for example run: sudo gparted /dev/loop0 (use correct loop it got assigned to) if you have a graphical interface to manually manage the img partitions in a graphical interface with gparted.
You can ofc use any partition manager for this.
If you added [extra space] this will then show up as unpartitioned space at the end of the device where you can create partition(s) and manually copy data to by mounting the new loop partition(s) that will become visible in lsblk.
If you do this, don’t forget to create or update the img with -e (disable autoexpansion) first. Autoexpansion will not work since the space will be occupied by your manually managed partition(s).
You can still update the img file with -U as long as the img root partition is big enough to hold the data from the device you backup from, but make sure to exclude your manually managed partition(s) or they will be copied to the img root partition.
To remove the loop: sudo losetup -d /dev/loop0, (use correct loop it got assigned to)
To remind yourself: lsblk /dev/loop* (if you forgot what loop it got assigned to)
–chroot
Use systemd-nspawn to chroot into an img created from the system you are currently running.
Works on all supported filesystems but you have to run the command on the system you created the img from, the script looks at the system you are running to create mountpoint(s) and chroot into.
A dependency check will be made and if you lack any of the packages needed the script will ask if you want to install them.
Then a temp directory will be created inside /tmp to use as mountpoint, loop the img and mount partitions from the loop appropriately.
Example: sudo shrink-backup --chroot /path/to/backup.img
With this you can for example experiment with things you don’t want to do on your running system, do changes to kernel, rebuild initramfs, update/install packages etc.
You will run as root user inside the img so unless you su to another user there is no reason to use sudo.
When done, type exit. The script will unmount img, remove loop, delete temp directory and exit shrink-backup.
You can then write the img file and test if things work without risking corruption of the system you run.
Systemd logs will not register changes made on the img though because the log-id:s on the mounted img will be the same as on the system you run (but can probably be made to work, if you feel like helping out, creating pr:s to the testing branch of shrink-backup would be highly appreciated)
--f2fs (Convert ext4 into f2fs on img file)
[!IMPORTANT]
ONLY use this for CONVERTING filesystem into img file, if you already have f2fs on the system you backup from, do not use this option.
The script will detect what filesystem is used on root and act accordingly.
Only supported for Raspberry Pi OS with new backups, not when using -U.
Autoexpansion at boot is not supported for f2fs (there is no way of resizing a mounted f2fs filesystem, unlike with ext4) so resizing root partition have to be made manually after writing img to sd-card.
Resize operations (when updating backup with -U) is not available for f2fs as of now.
The script will make backups of fstab & cmdline.txt into fstab.shrink-backup.bak & cmdline.txt.shrink-backup.bak on the img.
It will then change from ext4 to f2fs in fstab & cmdline.txt and add discard to the options on the root partition in fstab.
Please read information about f2fs further down.
Info
The script works on any device as long as root filesystem is ext4, f2fs or btrfs with subvolumes.
Since the script uses lsblk to crosscheck with /etc/fstab to figure out where root resides it does not matter what device it is located on.
If boot is a partition, that partition must be on the same device and before the root partition.
The script considers everything on the device before root as the bootsector.
Even if you forget to disable autoexpansion on a non supported OS, the backup will not fail, it will just skip creating the autoresize script.
Rsync WILL cross filesystem boundries, so make sure you exclude external mounts and other partitions unless you want them included in the root partition of the img backup. (separate /home for example)
- The script will ONLY create
boot (if exits) and root partitions on the img file.
- The script will ONLY look at your
root partition when calculating sizes.
Not excluding other mounts will copy that data to the img root partition, not create more partitions, so make sure to manually add [extra space] if you do this.
btrfs is an exception, all subvolumes (unless excluded) will be created.
See –loop for how to manually include more partitions on the img.
Applications used in the script:
- fdisk
- sfdisk
- dd
- parted
- e2fsck
- truncate
- mkfs.ext4/f2fs/btrfs (depends on fileystem used)
- rsync
- gdisk (sgdisk is only required if the partition table is GPT, the script will inform you)
Image creation
The easiest way to create a backup is to let the script configure the size.
To create a backup img using recommended size, use the -a option and the path to the img file.
Example: sudo shrink-backup -a /path/to/backup.img
The script will set the img size by requesting recommended minimum size from e2fsck or du (e2fsck does not work on f2fs f.ex).
This is not the absolute smallest size you can achieve but is the “safest” way to create a “smallest possible” img file.
If the size of the filesystem you are backing up from does not increase too much, you can most likely keep it updated with the update function (-U) of the script.
To manually configure size use [extra space]
Space is added on top of df reported “used space”, not the size of the partition.
[extra space] is in MiB, so if you want to add 1G, add 1024.
Example: sudo shrink-backup /path/to/backup.img 1024
Smallest image possible
To get the absolute smallest img file possible, do NOT use -a option, instead set [extra space] to 0
Example: sudo shrink-backup /path/to/backup.img 0
This will instruct the script to get the used space from df and add 128MiB “wiggle room”.
If you are like me, doing a lot of testing, rewriting the sd-card multiple times when experimenting, the extra time it takes each “burn” will add up pretty fast.
Example:
-rw-r--r-- 1 root root 3.7G Jul 22 21:27 test.img # file created with -a
-rw-r--r-- 1 root root 3.3G Jul 22 22:37 test0.img # file created with 0
[!IMPORTANT]
Because of how filesystems work, df is never a true representation of what will actually fit in a created img file.
Each file, no matter the size, will take up one block of the filesystem, so if you have a LOT of very small files (running docker f.ex) the “0 added space method” might fail during rsync. Increase the 0 a little bit and retry.
Using this method also means you have VERY little free space on the img file after creation.
If the filesystem you back up from increases in size, an update (-U) of the img file might fail with lack of space.
Order of operations - Image creation
- Uses
lsblk & /etc/fstab to figure out the correct disk device to back up.
- Reads the block sizes of the system’s
root (and boot if it exists) partition.
- Uses
dd to create the boot part of the system + a few megabytes to include the filesystem on root. (this can be a partition)
- Uses
df and/or resize2fs (depends on filesystem) to calculate sizes by analyzing the system’s root partition. (For btrfs: btrfs filesystem du + 192MiB is used instead of resize2fs)
- Uses
truncate to resize img file.
- Loops the img file.
- Removes and recreates the
root partition on the loop of the img file.
- Creates the
root filesystem on loop of the img file with the same UUID and LABEL as the system you are backing up from.
- Creates a temp directory and mounts img file
root partition from loop.
- Checks if
boot partition exists, if true, checks fstab and creates directory on root and mounts accordingly from loop.
- Uses
rsync to sync filesystems.
- Tries to create autoresize scripts if supported on OS and not disabled with
-e.
- Unmounts and removes temp directory and file (file created for
rsync log output).
Image update
To update an existing img file simply use the -U option and the path to the img file.
Example: sudo shrink-backup -U /path/to/backup.img
Autoresizing img when updating
If -a is used in combination with -U, the script will compare the root partition on the img file to the size resize2fs recommends as minimum (or du calculations depending on filesystem).
Example: sudo shrink-backup -Ua /path/to/backup.img
- The img file
root partition needs to be >=256MB smaller than resize2fs recommended minimum (or du calculations) to be expanded.
- The img file
root partition needs to be >=512MB bigger than resize2fs recommended minimum (or du calculations) to be shrunk.
This is to protect from unnecessary resizing operations most likely not needed.
Using combination -Ua on an img that has become overfilled works, if not use --fix or manually add [extra space] and retry.
Manually resizing img when updating
Only expansion is possible with this method.
If [extra space] is used in combination with -U, the root partition of the img file will be expanded by that amount.
[extra space] is in MiB, so if you want to add 1G, add 1024.
Example: sudo shrink-backup /path/to/backup.img 1024
No checks are being performed to make sure the data you want to back up will actually fit.
Resizing operations are not supported with f2fs.
Order of operations - Image update
- Loops the img file.
- Probes the loop of the img file for information about partitions.
- If
-a is selected, calculates sizes by comparing root used space on system and img file by using fdisk & resize2fs (or du depending on filesystem).
- Expands filesystem on img file if requested (
-a) and conditions were met in point 3, or if manually added [extra space] is used.
- Creates temp directory and mounts
root partition from loop.
- Checks if
boot partition exists, if true, checks fstab and mounts accordingly from loop.
- Uses
rsync to sync filesystems.
- Shrinks filesystem on img file if requested (
-a) and conditions were met in point 3.
- Tries to create autoresize scripts if supported on OS and not disabled with
-e.
- Unmounts and removes temp directory and file (file created for
rsync log output).
f2fs
The script will detect f2fs on root automatically and act accordingly.
Do NOT USE --f2fs unless you are converting from a ext4 filesystem (on your system) into f2fs on the img file.
Autoexpansion at boot is not possible with f2fs. User will have to manually expand img to cover entire storage media (f.ex sd-card) before booting a restored img.
Resizing img root partition while updating (-U) is not possible with f2fs as of now. User will have to create a new backup if img runs out of space.
This is something planned to be implemented further down the line.
btrfs
ALL testing has been done on Manjaro-arm
[!NOTE]
THIS IS NOT A CLONE, IT IS A BACKUP OF REQUIRED FILES FOR A BOOTABLE BTRFS SYSTEM!
Deduplication will not follow from the system you backup from to the img.
The script does NOT utilize btrfs send|recieve
The script utilizes rsync witch is significantly faster, but that means deduplication does not work.
All options in script will work just as on ext4. The script will detect btrfs and act accordingly.
Top level 5 subvolumes are checked for in /etc/fstab and are created (only at img creation) and mounted accordingly.
Mount options will be preserved (if you for example changed compression).
Top level 5 subvolumes NOT in /etc/fstab will not be created.
This means that for example top level 5 snapshots (unless mounted in fstab) will not be included.
If you however have the snapshots mounted in /etc/fstab they will be created as subvolumes and files copied as “normal” files and take up space, NOT deduplicated!
Simply mounting the snapshot somewhere will not create the snapshot, only copy the files like from a normal directory.
The initial report window will show all top and nested subvolumes that will be included, make sure these are correct before pressing y.
[!IMPORTANT]
You should not use subvolid in mount options in /etc/fstab. This is because the subvolumes created on the img might get different subvolid and therefore fail to mount.
You can also not use UUID, PARTUUID is ok. This is because the linux kernel will not let you have two btrfs filesystems with the same UUID mounted at the same time.
The preferred option is subvol=@ (or subvol=/@ for example for root, using leading slash or not does not matter)
This means that if you restore an img, you can not update that same img with -U from that restored system, the img mounting will fail during backup due to both having the same UUID.
If you have restored a backup, create a new backup from that system that can then be updated.
All nested subvolumes will be created unless excluded in a file called exclude_btrfs.txt, please see below.
Nested subvolumes can also be both created and removed with an update of the backup.
Autoresize function has only been tested and works on Manjaro-arm.
When creating an img, the initial report window will tell you what volumes will be created. Make sure these are correct before pressing Y.
exclude_btrfs.txt
Because of how snapshots are named, a special file for excluding is necessary. rsync uses PATTERNS witch means undesired results can occur in certain situations.
If for example a nested subvolume exists at /temp, the name of that subvolume will be temp (without leading /), and by excluding that in exclude.txt, rsync will interpret that as a PATTERN, therefore exclude and delete ALL directories called temp on the entire img.
If you want to exclude subvolumes, both top level 5 and nested subvolumes, create the file exclude_btrfs.txt in the same directory as shrink-backup and add subvolumes, one per line.
Wildcards (*) will not work in this scenario, but regex will, see tip below.
If installed using curl, the location to be used is different. See install with curl for information.
For example:
@subvolume1
@home/user/nested_volume
There is no need to use -t option in this scenario (unless you also want to exclude other files/directories), the script will detect the existence of the file and add the paths for the subvolumes to be excluded and deleted on the img with rsync.
[!TIP]
For detecting subvolumes in exclude_btrfs.txt, regex is used (grep -xE)
So if you for example have a collection of nested subvolumes or snapshots somewhere, all starting with the same directory name, let’s say a date, you can choose witch to exclude by date and exclude them all:
path/to/snapshots/2025(.+)
The path to be used is what is shown by using sudo btrfs subvolume list /, ie the subvolume without a leading /, so if you have nested subvolumes in for example @home, the path will be:
@home/path/to/subvolume
And if they are nested subvolumes on root (@), the path will be:
path/to/subvolume
To exclude an entire top level 5 subvolume, the path will be:
@subvolume
The initial report window will show all top and nested subvolumes by path and how they should be excluded if so is desired.
Fun fact about shrink-backup & btrfs
I used the script to create a backup of my Arch linux (BTW) desktop installation using btrfs with grub as bootloader (separate `/home` subvolume included in the image)
Wrote that image to a usb stick, and it booted and autoexpanded without problems.
I do NOT recommend using shrink-backup as your main backup software for a desktop computer!
Use proper backup software for that.
Thank you for using shrink-backup ❤️❤️
“A backup is not really a backup until it has been restored”
评论