If you are using Google Drive (or similar cloud storage) for your decoy activities when trying to appear as a normal person, then this tutorial may be useful for you.
Google Drive doesn't provide native bi-directional file syncing application on Linux and almost all solutions either suck or are paid. One free option is Rclone - Rclone is a great project, but the configuration interface is utterly horrible (if there's interest I can write down all the things that are wrong with it), so in this tutorial I'll try to provide a specific pre-configured setup to make this hopefully less painful. As you can see the following steps are mostly manual and runnable by a copypaste woodpecker, so this will hopefully automated away in the future.
NOTE: Check out if there's an updated version of this tutorial at https://github.com/notnout/tutorials
High level summary
- Install rclone
- Create Google Drive "remote" connection (called "drive")
- Create filter file to ignore some files from syncing
- Create a command using new feature called
rclone bisyncand run this command for the first time - [optional] Create systemd service to run the same command for you
- [optional] Create systemd timer to run the service regularly
- [optional] Create systemd path watcher to run the service automatically when you change the files
Files & Folders
$HOME/.config/rclone/rclone.conf- rclone config for your connection$HOME/.config/systemd/user/- folder to put your systemd services and watchers (and rclone filters file for our case)$HOME/sync/- folder where we put the actual files.
Common commands & Required knowledge
- Create and edit a file
nano ~/.config/SOMEFILE(save withCtrl+Oand close withCtrl+X)
Rclone InstallationRclone Installation
To use the new rclone bisync feature you need to get a new version v1.66+ of rclone. This version is not currently available in your usual repositories/packaging systems, so you need to install it manually.
If you are coming from the future, then try running e.g. sudo apt install rclone.
Go to https://rclone.org/downloads/ and download the appropriate Release binary (most likely "Intel/AMD - 64 Bit" and ".deb") to any folder on your disk. Try double clicking on the downloaded .deb file and that may open installer depending on your Linux distribution. If not, then open terminal in the same folder and run:
sudo dpkg -i rclone-*.deb(this may again vary based on your Linux distribution and packaging system)
Create Google Drive ConnectionCreate Google Drive Connection
Unfortunatelly for this you will have to follow another tutorial.
NAME YOUR CONNECTION "drive" or update all further commands accordingly
These are decent:
- https://ostechnix.com/mount-google-drive-using-rclone-in-linux/
- https://youtu.be/YDF1nBaAptw?t=291
- or the rclone documentation: https://rclone.org/drive/
Summary
- run
rclone configand follow the steps to add new remote for Google Drive - Name your remote "drive"
- there are many footguns in that process. I'm sorry.
Create the filters fileCreate the filters file
When syncing files it's useful to filter out some files that you don't want to be synced. For example hidden files that start with ".". Run the following command to create this file. The documentation is quite bad, but you can learn how to set up filtering here: https://rclone.org/filtering/.
Create the config folder for your systemd programs:
mkdir -p ~/.config/systemd/user/Create ~/.config/systemd/user/rclone.filter.txt and paste:
# Filter file for use with bisync
# See https://rclone.org/filtering/ for filtering rules
# NOTICE: If you make changes to this file you MUST do a --resync run.
# Run with --dry-run to see what changes will be made.
# Exclude all hidden files and folders
- .*
- .*/**
# Exclude temp files
- ~*.tmpFirst time running rclone bisync commandFirst time running rclone bisync command
Now we can finally run the sync.
We need to do this in 3 phases:
- First run with
--resync --dry-runparams to see outputs, but not change anything on your system so you can verify it works correctly. - Second run with
--resyncwhich triggers the initial redownload of the data from Google Drive - Third run without these params to check that the "normal operation" works.
Steps
- Create the folder on your Linux where you want to sync files.
mkdir -p ~/sync- (also make sure you have the systemd config folder
mkdir -p ~/.config/systemd/user/)
- First time run the command with --resync
/usr/bin/rclone bisync drive: ~/sync
-MvP \
--create-empty-src-dirs \
--compare size,modtime,checksum \
--filters-file ~/.config/systemd/user/rclone.filter.txt \
--conflict-resolve newer \
--conflict-loser delete \
--conflict-suffix sync-conflict-{DateOnly}- \
--suffix-keep-extension \
--resilient \
--recover \
--no-slow-hash \
--drive-skip-gdocs \
--fix-case \
--resync \
--dry-run- Now run the same command, but remove the last
--dry-runparam (and remove the backslash on the previous line).- Check that your files have been correctly downloaded to your selected folder.
- Now run the same command, but remove both
--dry-runand--resync(and the two backslashes).- Check that everything looks as intended, both on your local file system and in your Google Drive.
Create the serviceCreate the service
Create ~/.config/systemd/user/rclone.service and paste:
[Unit]
Description=Syncing local files with Google Drive using rclone bisync
Documentation=man:rclone(1)
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=60
StartLimitBurst=1
[Service]
Type=oneshot
ExecStart= \
/usr/bin/rclone bisync drive: %h/sync
-MvP \
--create-empty-src-dirs \
--compare size,modtime,checksum \
--filters-file %h/.config/systemd/user/rclone.filter.txt \
--conflict-resolve newer \
--conflict-loser delete \
--conflict-suffix sync-conflict-{DateOnly}- \
--suffix-keep-extension \
--resilient \
--recover \
--no-slow-hash \
--drive-skip-gdocs \
--fix-case
[Install]
WantedBy=default.targetNote: In the service we ensure that it runs when we are online and that it only starts once in 60 seconds (this is useful when used with file watcher later).
ENABLE AND RUN THE SERVICEENABLE AND RUN THE SERVICE
# Read and reload all unit configs. Required so it gets new config file state.
systemctl --user daemon-reload
# Enable to run after reboot and also start immediately with "--now"
systemctl --user enable --now rclone.serviceHELPFUL COMMANDSHELPFUL COMMANDS
# See status
systemctl --user status rclone
# See the service logs
journalctl --user -xeu rclone
# Stop the service (but won't disable after next reboot)
systemctl --user stop rclone
# Disable the service
systemctl --user disable rcloneCreate the timerCreate the timer
Following tutorial
Create ~/.config/systemd/user/rclone.timer with:
[Unit]
Description=Rclone timer (triggers every 10 minutes)
[Timer]
OnCalendar=*:0/10
Persistent=true
Unit=rclone.service
[Install]
WantedBy=default.targetENABLE AND RUN THE TIMERENABLE AND RUN THE TIMER
systemctl --user daemon-reload
systemctl --user enable --now rclone.timerHELPFUL COMMANDSHELPFUL COMMANDS
# See timer logs
journalctl --user -xeu rclone.timer
# List all timers
systemctl --user list-timers --all
# See the log for the timer
journalctl --user -xeu rclone.timer
# Stop the timer
systemctl --user stop rclone.timer
# Disable the timer
systemctl --user disable rclone.timerCreate the file watcherCreate the file watcher
Following stackoverflow. This is useful to automatically trigger the sync as soon as the files on your disk change.
Create ~/.config/systemd/user/rclone.path with:
[Path]
Unit=rclone.service
PathChanged=%h/sync
# trigger on changes to a file not just create/delete
[Install]
WantedBy=default.targetENABLE AND RUN THE FILE WATCHERENABLE AND RUN THE FILE WATCHER
systemctl --user daemon-reload
systemctl --user enable --now rclone.{timer,path,service}SEE THE STATUS OF THE COMPLETE SETUPSEE THE STATUS OF THE COMPLETE SETUP
systemctl --user status rclone.service rclone.timer rclone.path
# Combined logs.
journalctl --user -xeu rclone.service rclone.timer rclone.path
Just to offer an alternative to the solution proposed in the tutorial: buy a InSync license (lifetime, no subscription!), set, and forget.
Yeah, there are multiple paid solutions. Even some that are based on rclone, like https://docs.s3drive.app/ . But I think it's cool to do some things yourself...
Sorry, but using a public cloud with rclone without their crypt feature should be a no-go!
See https://rclone.org/crypt/ and use full folder & file name encryption to prevent leaking metadata.
Even with crypt better don't sync confidential things like private keys, passwords or userdata to any public cloud.
Agreed, never put private info like keys into a public cloud!
For the crypt feature it depends on the specific usecase, but I intentionally wrote the tutorial to make it easy to customize it to whatever you need... (Also it was cool to learn to set up the systemd file watchers and timers...)
I could hardly think of any use case for storing unencrypted data on a public cloud, only if you want to share links publicly and are aware, that the data is indeed PUBLIC the second you upload it.
It would be cool to see the tutorial updated and strongly advise users what they are doing when using a public cloud with unencrypted data.
@ek @k00b, what is the thinking on allowing editing of posts later (maybe configurable by territory)?
My usecase - I see some typos and I'd like to fix them.
Another brainstorm - what about syntax highlighting in markdown? Using following syntax to pick the language (e.g. here "sh" or "bash"):
```sh
echo "hello from bash"
```
Ohh, good request! Everytime I create a post with code, I think about this but then I forget. Will create a ticket this time.
We want to allow it but also expose edit history (maybe like Github does).
Here is the relevant Github ticket.
Thanks, it works pretty well with the new iCloud remote that was added to rclone too!
The only issue I see is that the file watcher is only checking the top-level folder but does not check for changes recursively. Would there be any way to improve that?
I think I have a typo both in the script and serve. There's supposed to be "" at the end of the line of
/usr/bin/rclone bisync drive: ~/syncOne highlight is that the steps for setting up systemd file watchers and timers can be used for anything you want and you most likely already haver systemd installed on your Linux.
This tutorial works exactly the same for following providers/methods:
What a tutor... I'm really exited to see something like this over here
Today when people search something they add "reddit" to the prompt. In the near future they'll start to add "sn" or "stackerNews" and get excellent answers
This is a nice in depth tutorial. A little above my technical capabilities :)
Where is the point to use Google Drive? It's a big tech company and Bitcoiners should have the mindset to be souvereign and free. So please avoid Google. And if you want to use any cloud service then there is a bunch of open source alternatives. If you want to sync files between devices: https://syncthing.net/
I agree with you. But pls check the first sentence in the tutorial... ;)
Let's not "normalise" the use of spyware
A guide for nextcloud would be awesome
The exact same tutorial works for Nextcloud too (I tried hinting at that in the first sentence)
Do you think you're suspicious if you don't use Google Drive? This is a very paranoid view in my opinion. Sounds to me like a reason to just keep using it.