# Synology DiskStation Manager ## Table of contents 1. [System's shared folders](#systems-shared-folders) 1. [Rsync](#rsync) 1. [Snapshots](#snapshots) 1. [Encrypt data on a USB disk](#encrypt-data-on-a-usb-disk) 1. [Data deduplication](#data-deduplication) 1. [Remove duplicated files with jdupes](#remove-duplicated-files-with-jdupes) 1. [Deduplicate blocks in a volume with duperemove](#deduplicate-blocks-in-a-volume-with-duperemove) 1. [Use keybase](#use-keybase) 1. [Manage git repositories with a containerized keybase instance](#manage-git-repositories-with-a-containerized-keybase-instance) 1. [Ask for a feature to be implemented](#ask-for-a-feature-to-be-implemented) 1. [Further readings](#further-readings) 1. [Sources](#sources) ## System's shared folders Automatically created by services or packages. Cannot be changed/removed manually if the package creating them is still active or installed. ```txt /volume1 ├── docker # data container for the Docker service, created by it upon installation ├── homes # all users' home directories, created by the SSH service upon activation ├── music # created by the Media Server package upon installation ├── NetBackup # created by the rsync service upon activation ├── photo # created by the Media Server package upon installation └── video # created by the Media Server package upon installation ``` USB disks are recognized as shared folders automatically and mounted under `/volumeUSBX`: ```txt /volumeUSB1 └── whatever ``` ## Rsync Requirements: 1. the rsync service is enabled under _Control Panel > File Services > rsync_ 1. the user has the right permissions for the shared folder under either - _Control Panel_ > _Shared Folders_ > _Shared Folder_ edit window > _Permissions_, or - _Control Panel_ > _User & Group_ > _User_ or _Group_ edit window > _Permissions_ Examples: ```sh # From a shared folder on a NAS to a local one. # Use the SSH port defined in the NAS settings. rsync \ "user@nas:/volume1/shared_folder/" \ "path/to/local/folder/" \ --archive --copy-links --protect-args \ --acls --xattrs --fake-super \ --partial --append-verify --sparse \ --progress -vv --no-inc-recursive \ --compress --no-motd --rsh='ssh -p12345' \ --exclude "@eaDir" --exclude "#recycle" \ --delete --dry-run # Sync all snapshotted data to a folder. find /volume1/@sharesnap/shared_folder \ -maxdepth 1 -mindepth 1 \ -type d | xargs -I {} -n 1 -t \ rsync \ -AXahvz --chown=user --info=progress2 \ --append-verify --partial \ --no-inc-recursive --no-motd \ {}/ \ /volume2/destination/folder/ ``` ## Snapshots Use the **Snapshot Replication** package available in the Package Center for better control and automation. Gotchas: 1. when the _Make snapshot visible_ option in a shared folder's settings in _Snapshot Replication_ is ticked: - the `#snapshot` folder is created in the shared folder's root directory - the default snapshots directory for that shared folder is mounted on it in **read only** mode: ```txt /dev/mapper/cachedev_0 on /volume1/Data/#snapshot type btrfs (ro,nodev,relatime,ssd,synoacl,space_cache=v2,auto_reclaim_space,metadata_ratio=50,block_group_cache_tree,subvolid=266,subvol=/@syno/@sharesnap/Data) ``` ## Encrypt data on a USB disk Synology DNS does not equip utilities like `cryptsetup` or TrueCrypt or such. Also, creating a docker container for it is at this time a little bit too much for me. But, it does include `ecryptfs`. I found [this solution on Reddit][encrypting an attached external usb drive?] to use the included `ecryptfs`. It has downsides (`ecryptfs`' vulnerabilities, the fact that terminal commands are logged in `/var/log/bash_history.log` and a password would be visible, etc), but hey, that is what is used internally, so... > Implementation: > > 1. Create a shared folder called "crypt" [on your normal Synology Diskstation volume] > 1. Plug in the USB drive if you haven't already > 1. Log into DSM manager > 1. Go to network services, and select terminal > 1. Enable Telnet service. (If you have been manually changing the firewall, make sure you've unblocked port 23) > 1. Telnet into the Synology box - logging in as root > 1. Type this command to create the directory on your USB drive: "mkdir /volumeUSB1/usbshare/@crypt@" > 1. Update the _blahblahblah_ password below and type into your telnet session (note - it should all be on one line): "mount.ecryptfs /volumeUSB1/usbshare/@crypt@ /volume1/crypt -o \key=passphrase:passphrase_passwd=blahblahblah,ecryptfs_cipher=aes,ecryptfs_key_bytes=32,\ecryptfs_passthrough=n,no_sig_cache,ecryptfs_enable_filename_crypto=y" > 1. Any data you copy into "crypt" above, will now be encrypted and saved in "usbshare1/@crypt@". To check - create a new folder in the folder "crypt" and have a look at how it appears encrypted when you look into "usbshare1/@crypt@" from DSM manager. > 1. From here - set up any backup jobs you wish to copy into the "crypt" shared folder you created. > 1. When you are ready to eject the drive make sure you unmount it first by typing into your telnet session "umount /volumeUSB1/usbshare/@crypt@" and then eject it in the normal way from DSM. > 1. Disable the telnet service if you are no longer using it ## Data deduplication Requirements: 1. `docker` needs to be installed from the package manager, as it is simpler (and safer?) to run a container than installing `duperemove` or `jdupes` and all their dependencies on the machine ### Remove duplicated files with jdupes Examples: ```sh # `sudo` is only needed if the user has no privileges to run `docker` commands. sudo docker run \ -it --init --rm --name jdupes \ -v "/volume/shared_folder1:/data1" \ -v "/volume/shared_folder2:/data2" \ ghcr.io/jbruchon/jdupes:latest \ -drOZ \ -X 'nostr:@eaDir' -X 'nostr:#recycle' \ "/data1" "/data2" ``` ### Deduplicate blocks in a volume with duperemove Gotchas: 1. `duperemove`'s container needs to be run in privileged mode (`--privileged`) due to it taking actions on the disk 1. the container might fail on very large datasets, usually due to Out Of Memory (OOM) issues; to avoid this: - offload the hashes from RAM using a hash file (`--hashfile "/volume1/NetBackup/duperemove.tmp"`) - use smaller datasets where possible, like a shared folder and just one of its snapshots instead of all of them 1. `duperemove` can dedupe blocks only if acting on folders in a _rw_ mount; when deduplicating snapshots, use their _rw_ mount path `/@syno/@sharesnap/shared_folder` instead of their _ro_ version `/volumeN/shared_folder/#snapshot` Examples: ```sh # small/medium dataset # 2 folders in a shared folder sudo docker run --privileged \ --rm --name duperemove \ --mount "type=bind,source=/volume1/Data,target=/sharedfolder" \ michelecereda/duperemove:0.11.2 \ -Adhr \ "/sharedfolder/folder1" "/sharedfolder/folder2" # large dataset # 1 shared folder and all its snapshots sudo docker run --privileged \ --rm --name duperemove \ --mount "type=bind,source=/volume1,target=/volume1" \ michelecereda/duperemove:0.11.2 \ -Adhr \ --hashfile "/volume1/NetBackup/duperemove.tmp" \ "/volume1/Data" "/volume1/@sharesnap/Data" ``` ## Use keybase Just use a containerized service and execute commands with it: ```sh # Run the service. docker run -d --name 'keybase' \ -e KEYBASE_SERVICE='1' \ -e KEYBASE_USERNAME='user' \ -e KEYBASE_PAPERKEY='paper key' \ 'keybaseio/client:stable' # Execute commands using the containerized service. docker exec \ --user 'keybase' \ keybase \ keybase whoami ``` ### Manage git repositories with a containerized keybase instance See the [readme for michelecereda/keybaseio-client][michelecereda/keybaseio-client]. ## Ask for a feature to be implemented Use the [online feature request form]. Posting a request on the community site will not work. ## Further readings - [CLI Administrator Guide for Synology NAS] - [Making disk hibernation work on Synology DSM 7] ## Sources All the references in the [further readings] section, plus the following: - [Configuring deduplication block on the Synology] - [Encrypting an attached external USB drive?] [cli administrator guide for synology nas]: https://global.download.synology.com/download/Document/Software/DeveloperGuide/Firmware/DSM/All/enu/Synology_DiskStation_Administration_CLI_Guide.pdf [online feature request form]: https://www.synology.com/en-global/form/inquiry/feature [further readings]: #further-readings [michelecereda/keybaseio-client]: ../docker/keybaseio-git-client/README.md [configuring deduplication block on the synology]: https://onedrive.live.com/?authkey=%21ACYMJq62iJaU7HY&cid=1E8D74207941B8DD&id=1E8D74207941B8DD%21243&parId=1E8D74207941B8DD%21121&o=OneUp [encrypting an attached external usb drive?]: https://www.reddit.com/r/synology/comments/jq4aw6/encrypting_an_attached_external_usb_drive/ [making disk hibernation work on synology dsm 7]: https://www.reddit.com/r/synology/comments/10cpbqd/making_disk_hibernation_work_on_synology_dsm_7/