chore: improved rsync

This commit is contained in:
Michele Cereda
2023-08-06 15:26:32 +02:00
parent d31fd0f69d
commit 78fa817146
3 changed files with 111 additions and 71 deletions

View File

@@ -1,32 +1,41 @@
#!/usr/bin/env bash
rsync /data/ nas.lan:/data/ \
--archive --copy-links --protect-args --delete \
--acls --xattrs --fake-super \
# Sync directories from a Linux source to a Linux destination.
# Expand symlink at the source to their referred files.
# Assumes the same owner and group at both hosts.
rsync 'data/' 'nas.lan:data/' \
--secluded-args --no-inc-recursive \
--archive --copy-links --acls --xattrs --times --atimes --crtimes \
--partial --append-verify --sparse \
--human-readable --info='progress2' \
--delete --backup --backup-dir "changes_$(date +'%F_%H-%M-%S')" --exclude "changes_*"
rsync 'data/' 'nas.lan:data/' \
-abhstALNSUX --no-i-r \
--partial --append-verify \
--compress --sparse --no-motd \
--human-readable --no-inc-recursive --info="progress2" -vv \
--exclude ".terraform*" --exclude "obsidian" \
--backup --backup-dir "changes_$(date +'%F_%H-%m-%S')" --exclude "changes_*" \
| grep -Ev -e uptodate -e "/$"
--info='progress2' \
--delete --backup-dir "changes_$(date +'%F_%H-%M-%S')" --exclude "changes_*"
# cat '.rsync-filter'
# - .DS_Store
# - .localized
# - .obsidian
# - .terraform*
# + **
/opt/homebrew/bin/rsync 'Data' 'nas.lan:Data' \
-abchszAFLSUX \
--partial --append-verify --fake-super --no-motd \
--delete --backup-dir "changes_$(date +'%F_%H-%m-%S')" \
--no-inc-recursive --info="progress2"
# Sync directories from a Linux source to a Synology NAS.
# The above one, just modified to be accepted from those systems.
rsync 'data/' 'synology.lan:/volume1/data/' \
--secluded-args --no-inc-recursive \
--archive --copy-links --acls --xattrs \
--partial --append-verify --sparse \
--human-readable --info='progress2' \
--delete --backup --backup-dir "changes_$(date +'%F_%H-%M-%S')" --exclude "changes_*" \
--no-motd --fake-super --super --chown='user:users' \
--exclude={'@eaDir','#recycle'}
rsync 'data/' 'synology.lan:/volume1/data/' \
-abhsALSX --no-i-r \
--partial --append-verify \
--info='progress2' \
--delete --backup-dir "changes_$(date +'%F_%H-%M-%S')" --exclude "changes_*" \
--no-motd --fake-super --super --chown='user:users' \
--exclude={'@eaDir','#recycle'}
# .rsync-filter hides files on source, but does nothing for the ones on the remote
/opt/homebrew/bin/rsync 'Data' 'synology.lan:Data' \
-abchszAFLSX \
--partial --append-verify --fake-super --no-motd \
--delete --backup-dir "changes_$(date +'%F_%H-%m-%S')" \
--no-inc-recursive --info="progress2" \
--exclude={'@eaDir','#recycle','changes_*'}
# Use the '.rsync-filter' file.
# The filter file excludes files from the source, but does nothing for the ones
# on the remote side. To exclude them too, explicitly use the `--exclude` option.
$ rsync … -F

View File

@@ -4,6 +4,8 @@
1. [TL;DR](#tldr)
1. [Explored options](#explored-options)
1. [Filters](#filters)
1. [Filter file](#filter-file)
1. [Sources](#sources)
## TL;DR
@@ -86,7 +88,8 @@ parallel -q \
## Explored options
| Long format | Short format | Description |
| ----------------------- | ------------ | -------------------------------------------------------------------------------------------- |
| ----------------------- | ------------ | --------------------------------------------------------------------------------------------------- |
| | `-F` | same as `--filter='dir-merge /.rsync-filter'`<br/>if repeated, same as `--filter='- .rsync-filter'` |
| | `-P` | same as `--partial --progress` |
| `--acls` | `-A` | preserve ACLs; implies `--perms` |
| `--append-verify` | | like `--append`, but use the data already there to check the items |
@@ -127,6 +130,27 @@ parallel -q \
| `--verbose` | `-v` | increase verbosity once for each copy of this switch |
| `--xattrs` | `-X` | preserve extended attributes |
## Filters
### Filter file
Set up a `.rsync-filter` file in any directory. If `rsync` is called with the `-F` option, the filtering rules in that file will be applied from that directory to all its subfolders.
```sh
$ cat '.rsync-filter'
- .DS_Store
- .localized
- .obsidian
- .terraform*
+ **
$ rsync … -F
```
The filter file excludes files from the source, but does nothing for the ones on the remote.<br/>
If one wants to exclude files from the remote, they must be set explicitly using the `--exclude` option.
## Sources
- [cheat.sh]

View File

@@ -9,8 +9,15 @@ The `zstd`, `zstdmt`, `unzstd`, `zstdcat` utilities compress or decompress `.zst
## TL;DR
```sh
zstd -15 -r --compress --rsyncable "folder" -o "folder.zst"
zstd --test
zstd --list
zstd --decompress
# Compress folders into an archive.
zstd --compress -15 --rsyncable -r 'folder' -o 'archive.zst'
# Test archives.
zstd --test 'archive.zst'
# Print information about files in archives.
zstd --list 'archive.zst'
# Decompress archives.
zstd --decompress 'archive.zst'
```