Differences

This shows you the differences between two versions of the page.

--- encrypted_backups_to_the_cloud [13.08.2017 14:50] – [setup the mount points] Pascal Suter
+++ encrypted_backups_to_the_cloud [15.08.2017 08:44] (current) – [the script] Pascal Suter
@@ Line 76: / Line 76: @@
 <code bash offsiteBackup.sh>
+#!/bin/bash
+# (c) 2017 Pascal Suter, Version 0.10 Beta
+# this script creates an enecrypted offsite backup of a locally kept backup.
+# ideally suited to work with rubi (http://www.0x1b.ch/misc/hacks/rubi)
+# for a full description and setup instructions read
+# http://wiki.psuter.ch/doku.php?id=encrypted_backups_to_the_cloud
+# uses gocryptfs (https://nuetzlich.net/gocryptfs/) with -reverse option.
+# you may use, modify and re-distribute this script AT YOUR OWN RISK free of charge.
+CRYPTED="/offsiteBackup/crypted" # folder where the encrypted files are at
+TARGET="/mnt/offsiteBackup" # remote location: use server:/target/directory for ssh or rsync targets
+LATEST=$(cat /backupHome/lastdst) # full path to the last successful backup to sync to offsite target
+PLAINDIR="/offsiteBackup/plain" # directory where the plaintext version of the cryptfsmount is
+PLAINMOUNT="$PLAINDIR/backup" # mount point where the lastdest should be mounted to. this must be inside PLAINDIR!
+RECIPIENTS="root" # email address of mail recipients, separate multiple addresses with space
+LOCKFILE="/var/lock/offsiteBackup" # to make sure this script is not run twice at the same time
+RSYNCOPTS="" # additional options to rsync.
+#RSYNCOPTS='-e "ssh -p 2882"' # use a non-standard port for an ssh connection to the remote target
+function fail {
+    echo "$1" | mail -s "offsiteBackup failed" "$RECIPIENTS"
+    exit 1
+}
+function success {
+    ( echo "the offsite backup was successfully updated to backup version $LATEST"
+    echo "here are the last lines of the rsync process:"
+    tail -n 3 /tmp/offsiteBackup.log ) | mail -s "offsiteBackup successfully updated" "$RECIPIENTS"
+    umount $PLAINMOUNT 2>/dev/null
+    exit 0
+}
+me=`basename "$0"`
+# get a lock and run me embedded
+if [ "$1" != "--embedded" ]; then
+    echo "staring $0"
+    flock -E 66 -n ${LOCKFILE} $0 --embedded | tee /tmp/offsiteBackup.log 2>&1
+    state=$?
+    if [ $state -eq 66 ]; then
+        fail "there was another offsiteBackup process still running, so we skipped this round"
+    fi
+    exit $state
+fi
+# make sure our crypted directory is mounted
+grep "$CRYPTED" /proc/mounts > /dev/null
+if [ $? -gt 0 ]; then
+    fail "$CRYPTED was not mounted, please login to your server and run # gocryptfs --reverse $PLAINDIR $CRYPTED"
+fi
+# unmount any previous bind mounts to $PLAINMOUNT and check it is no longer mounted
+umount $PLAINMOUNT 2>/dev/null
+grep "$PLAINMOUNT" /proc/mounts > /dev/null
+if [ $? -eq 0 ]; then
+    fail "There seems to be a stale mount on $PLAINMOUNT, please login to your server and unmount this directory manually"
+fi
+# mount the latest backup:
+mount -B "$LATEST" "$PLAINMOUNT"
+if [ $? -gt 0 ]; then
+    fail "there was a problem mounting the latest backup from $LATEST to $PLAIMOUNT"
+fi
+# rsync to offsite location
+rsync -AaHvXx --delete $RSYNCOPTS "$CRYPTED/" "$TARGET" 2>&1
+res=$?
+if [ $res -gt 0 ]; then
+    if [ $res -eq 24 ]; then
+        #some files vanished during the backup, that's not a failure of the backup, so send the success message
+        success
+    else
+        fail "there was a problem with the offsite backup, check /tmp/offsiteBackup.log on the server"
+    fi
+else
+    success
+fi
 </code>
+=== Known Issues ===
+For some reason gocryptfs seems to generate some files (two in my case) like '' gocryptfs.longname.*.name'' that are there when rsync scans the directory but vanish before rsync can sync them. this leads to rsync complaining about vanished files. when i mount the backup with gocryptfs (forward) and then compare the unencrypted backup with the original through rsync --dry-run i find a single file to be missing in my encrypted backup. It has a very long filename: ''xxxx/xxxx/vlc-shares/public/images/fsthumbs/thumbs/MjovVFYvQWxhcm0gZnVyIENvYnJhIDExIC0gRGllIEF1dG9iYWhucG9saXplaSAtIFNlYXNvbiAxLTEyL0NvYnJhIDExIFN0YWZmZWwgMDEvQ29icmEgMTEgUzAxIEUwMDYgKERlciBBbHRlIHVuZCBkZXIgSnVuZ2UpLmF2aQ==.jpg''. the script accepts exit code 24 (vanished files) as successful ending of rsync for now until I have time to further investigate and either file a bug at gocryptfs or work around it somehow.
+===== Restoring Files =====
+to restore files you could use ''sshfs'' for example to mount the remote directory via ssh on your local server...
+  sshfs user@remote.server:/offsiteDirectory /mnt/offsiteBackup
+and now use gocryptfs to uncrypt the contents and restore some files:
+  gocryptfs /mnt/offsiteBackup /mnt/uncrypted
+now you should see all your files in /mnt/uncrypted
+unmount both mounts once you are done.