my_new_backup_solution_with_burp

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
my_new_backup_solution_with_burp [20.09.2021 21:47] – [first start] Pascal Sutermy_new_backup_solution_with_burp [20.09.2021 22:01] (current) – [setup a backup report script] Pascal Suter
Line 81: Line 81:
   0,20,40 * * * * /usr/sbin/burp -a t -q 1200 >>/var/log/burp-client 2>&1   0,20,40 * * * * /usr/sbin/burp -a t -q 1200 >>/var/log/burp-client 2>&1
  
-===== setup a backup report script =====+====== setup a backup report script ======
 i wrote a [[burp backup report generator script|script to send reports on a daily basis]] listing all backups and their age. since burp does not run a backup of all machines at once, but rather lets the client machines start a new backup whenever it is due, we have to create a backup report independent of any running jobs at a fixed time of the day (or week). I personally like systems that also send an email when things are good. this assures that you will get the alert if one is needed.  i wrote a [[burp backup report generator script|script to send reports on a daily basis]] listing all backups and their age. since burp does not run a backup of all machines at once, but rather lets the client machines start a new backup whenever it is due, we have to create a backup report independent of any running jobs at a fixed time of the day (or week). I personally like systems that also send an email when things are good. this assures that you will get the alert if one is needed. 
  
 +====== offsite backup solution ======
 +theoretically one could probably use burp to create an offsite backup of another burp server. However, i wanted something different, so in case there is a security issue with burp, the offsite backup would not automatically be compromised as well. 
  
 +I chose to use a combination of rclone on the offsite machine and sftpgo on the burp server. Basically sftpgo is a dedicated sftp server daemon. It does not have the full ssh functionality such as port forwarding, shell etc, it only allows file access. It also supports a few other protocols, but we will stick to sftp for now. Another advantage over just configuring a very limited user for our normal OpenSSH server is, that it sftpgo can also create read-only file shares, which is exactly what we want. 
  
 +So set up sftpgo and then configure a new user (which should not be a sytem user, it's an internal user of sftpgo). Then share the backup directorie(s) with read-only access to this user. 
  
 +now depending on your offsite backup machine, if it has a fixed ip address, you can create a more specific rule for ufw, that allows only access from this specific IP. This is not possible if your remote server has a dynamic ip of course. 
 +  ufw allow proto tcp from 172.16.28.1 to any port 2022
 +my offsite backup wrapper script for rclone will be able to work on multiple offsite backups at the same time, so make sure you allow enough concurrent connections on the sftpgo site. The default of 20 was not enough for 4 concurrent rclone downloads, i had to set it to something above 30. went to 100 and haven't had an issue since: 
 +  sed -i 's/\("max_per_host_connections":\) .*/\1 100/' /etc/sftpgo/sftpgo.json
 +  systemctl restart sftpgo.service
  
 +====== Offsite-Backup server ======
 +on the offsite backup server, isntall the latest version of rclone
 +
 +then run 
 +  rclone config
 +to set up a new client. 
 +
 +add the backup  host and make sure that you pass the path to the unencrypted ssh key where it asks for the ''ssh_key'' and not the ''ssh_pem'' .. it is NOT the ''Raw pem-encoded'' key, it is just the ''pem-encoded'' key which was confusing to me :) 
 +
 +  rclone lsd planb:/
 +  
 +should now show our backups
 +
 +Now download my [[rclone wrapper to create offsite backups of a burp server]] and set up a cron job for it. (currently this script is a WIP, so come back later when it's done)
  • my_new_backup_solution_with_burp.1632167239.txt.gz
  • Last modified: 20.09.2021 21:47
  • by Pascal Suter