brandon/oam
1
0
Fork 0
mirror of https://gitea.com/mcereda/oam.git synced 2026-02-09 05:44:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added system shared folders and file deduplication section

Browse Source
This commit is contained in:
Michele Cereda
2022-07-24 20:13:11 +02:00
parent a9e468dce1
commit 221054c481
1 changed files with 47 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
knowledge base/synology dsm.md
View File

@@ -1,5 +1,21 @@
# Synology DiskStation Manager
## System's shared folders
Automatically created by services or packages.
Cannot be changed/removed manually if their creator is still active or installed.
```text
/volumeX
├── docker # data container for the Docker service, created by it upon installation
├── homes # all users' home directories, created by the SSH service upon activation
├── music # created by the Media Server package upon installation
├── NetBackup # created by the rsync service upon activation
├── photo # created by the Media Server package upon installation
└── video # created by the Media Server package upon installation
```
## Rsync
Requirements:
@@ -14,17 +30,19 @@ Examples:
```sh
# From a shared folder on a NAS to a local one.
# Use the SSH port defined in the NAS settings.
rsync \
--archive --hard-links \
--append-verify --partial \
--progress --verbose \
--compress --no-motd \
--exclude "@*" --exclude "#*" \
"user@nas:/volume1/shared_folder/" \
"path/to/local/folder/" \
--archive --copy-links --protect-args \
--acls --xattrs --fake-super \
--partial --append-verify --sparse \
--progress -vv --no-inc-recursive \
--compress --no-motd --rsh='ssh -p12345' \
--exclude "@eaDir" --exclude "#recycle" \
--delete --dry-run
# Sync all snapshotted data to a folder
# Sync all snapshotted data to a folder.
find /volume1/@sharesnap/shared_folder \
-maxdepth 1 -mindepth 1 \
-type d
@@ -52,20 +70,36 @@ Gotchas:
> /dev/mapper/cachedev_0 on /volume1/Data/#snapshot type btrfs (ro,nodev,relatime,ssd,synoacl,space_cache=v2,auto_reclaim_space,metadata_ratio=50,block_group_cache_tree,subvolid=266,subvol=/@syno/@sharesnap/Data)
> ```
## Deduplicate blocks in a volume
## Data deduplication
Requirements:
1. `docker` needs to be installed from the package manager, as it is simpler to run a container than installing `duperemove` and all its dependencies on the machine
1. the container needs to be run in privileged mode (`--privileged`) due to its actions on the disk
1. `docker` needs to be installed from the package manager, as it is simpler (and safer?) to run a container than installing `duperemove` or `jdupes` and all their dependencies on the machine
### Remove duplicated files with jdupes
Examples:
```sh
# `sudo` is only needed if the user has no privileges to run `docker` commands.
sudo docker run \
-it --init \
-v "/volume/shared_folder1:/data1" \
-v "/volume/shared_folder2:/data2" \
ghcr.io/jbruchon/jdupes:latest \
-drOZ \
-X 'nostr:@eaDir' -X 'nostr:#recycle' \
"/data1" "/data2"
```
### Deduplicate blocks in a volume with duperemove
Gotchas:
1. `duperemove`'s container needs to be run in privileged mode (`--privileged`) due to it taking actions on the disk
1. the container might fail on very large datasets, usually due to Out Of Memory (OOM) issues; to avoid this:
1. offload the hashes from RAM using a hash file (`--hashfile "/volume1/NetBackup/duperemove.tmp"`)
1. use smaller datasets where possible, like a shared folder and just one of its snapshots instead of all of them
- offload the hashes from RAM using a hash file (`--hashfile "/volume1/NetBackup/duperemove.tmp"`)
- use smaller datasets where possible, like a shared folder and just one of its snapshots instead of all of them
1. `duperemove` can dedupe blocks only if acting on folders in a _rw_ mount; when deduplicating snapshots, use their _rw_ mount path `/@syno/@sharesnap/shared_folder` instead of their _ro_ version `/volumeN/shared_folder/#snapshot`
Examples: