Seconded. rsync along with its powerful and little known options
I did that for a while, when I was using disks. I rather liked the 3 way Rsync where you have the source, old destination and new destination and it will hardlink to the old destination if there was no change, so you get efficient snapshots.
Also this (well my main backup hard drive is in a server, is encrypted and normally unmounted), only thing that doesn't back up this way is my gaming PC, where I create a disk image to an external hard drive using the Win7 backup tool. My phone also backs up via rsync through a chroot, although it's kind of a messy system and restoring wouldn't be so straightforward.
I wish there was a way to full backup my wife's iPhone automatically, that wasn't iCloud. Unfortunately as far as I can tell it's impossible because iOS kills background apps so aggressively. The best suggestion seems to be to keep the screen on for an initial full backup, and then hope the app can upload your new data within the seconds allowed by the OS now and then.
I wish there was a way to full backup my wife's iPhone automatically, that wasn't iCloud. Unfortunately as far as I can tell it's impossible because iOS kills background apps so aggressively. The best suggestion seems to be to keep the screen on for an initial full backup, and then hope the app can upload your new data within the seconds allowed by the OS now and then.
Well, iTunes can - if you create an encrypted backup. I assume its replacement has the ability to do so too.
We were in a similar area, woods and mountains, high fire risk, and actually did get hit with a forest fire that took out most of our neighboorhood.. Luckily I did have good backups (and time to get important things out), so all the data was safe, offsite is a definite must here! We also didn't have blazing fast bandwidth, so cloud backups were painful at best.
I was using amanda (which is just a front end for tar/gzip) for server backups, the backup server powered on (with an apc masterswitch) once a wee
There was a fire just a few miles away from us (Left Hand Canyon Fire in Colorado). No real chance of it getting to us but we had the cat carriers out and on the table, the two lock boxes with papers, and the drive sitting up next to the door just in case.
And yea, we have high speed WiFi for our network access so over the air backups would take quite a long time. I do burn an occasional DVD with photos and such and send them out to my daughter down on the flatlands and my brothers on the coasts but that's n
Same here.. we had a few hours of warning to collect some critical physical things, as well as all the pets... and most of the non-media stuff once compressed is around 300gb.. the rest is multiple tb of media that I do want to keep, just needed a better way. I didn't intend to test it this way, but I'm glad it worked!
It would be nice to have a slightly more flexible and integrated system set up though, as well as multiple version support. I have found that anything which isn't entirely automatic tends to get neglected.
By the way, there is a tool called rclone that does something similar to rsync but for cloud services. Handy for syncing off-site backup sets, although you should of course encrypt first. That brings up another issue with many solutions - the encryption prevents effective differential backups.
By the way, there is a tool called rclone that does something similar to rsync but for cloud services. Handy for syncing off-site backup sets, although you should of course encrypt first.
I believe rclone offers encryption so you can make your unencrypted local backups encrypted on the cloud. Also various cloud services have their own encryption systems.
That brings up another issue with many solutions - the encryption prevents effective differential backups.
Thanks, I hadn't looked at encryption in rsync. I only used it to periodically backup my cloud services. I also use Thunderbird for that, having it automatically start in the middle of the night, sync IMAP mailbox with Gmail and then close.
Borg seems quite good, similar to Duplicati but no Windows support. How robust is it? As I say occasionally Duplicati uploads fail (due to internet connection problems) and you have to run a manual command to recover, but it's never lost any data or completely trashed a b
[borg] How robust is it? As I say occasionally Duplicati uploads fail (due to internet connection problems) and you have to run a manual command to recover, but it's never lost any data or completely trashed a backup set.
Seems OK. So, the only nonlocal mode it has is SSH, and I use rclone for uploading. I've had many backups abort half way through, and it's something that's explicitly supported. Essentially it always creates BACKUPNAME.[0,1,2,etc], and on completion, renames it to BACKUPNAME. If it aborts,
Duplicati just creates more encrypted zip files every time, and occasionally removes old ones that no longer contain any required data. Once a file has been made it never gets renamed or added to, it's always a new file.
It seems to handle large amounts of data and large numbers of files without any problem, which is the downfall of many apps.
Duplicati just creates more encrypted zip files every time, and occasionally removes old ones that no longer contain any required data. Once a file has been made it never gets renamed or added to, it's always a new file.
Borg uses some sort of transactional database (made out of files), so it's just a symbolic name given to a snapshot. Quantity wise I've got one repository doing all my machines and it's fine, though it turns out that's not necessarily the best way of doing things. I'm probably happily into t
Borg produces an opaque* directory structure representing some sort of database and rclone just syncs that to a remote directory, so you can handle moving the database around, but renaming individual files in the database would break the structure. Actual file names are stored inside the database itself.
* opaque for normal use. It's FOSS and documented, but the internal structure is opaque to anyone not digging around in the guts of borg.
Maybe I misunderstood about the renames. How does it handle versioning, or doesn't it?
What I mean is you say it creates a file BACKUPNAME, so is there an older one like BACKUPNAME.20210307 or something? Like the current one renamed or some sort of shuffle?
Oh right, the backname is for the snapshot, so each backup is a complete (or incomplete) snapshot of your entire tree. You get file versions by going to older snapshots. So you can list all the snapshots you have, then dive into one to get the file. The files within a snapshot keep their names.
One other thing about Duplicati, it only keeps one file locally while the upload is in progress so you don't need much disk space if you are only using remote targets. Each file has a max size, I think they default is 50MB.
That's the downside of my method, you need a disk either on the local machine, or something you can SSH to, and I don't know how well it works over high latency connections. I use my desktop PC to hold that disk.
2. rsync to a remote HDD (remember that offsite backups are crucial).
Depends upon your level of risk. Do I have irreplaceable data here? If so, then sure, do the offsite. Would you be okay if the building were to burn to the ground? If so, maybe it's not necessary.
nextcloud to my own internal only server with a nightly rsync from that server to main backup. Monthly rsync from main backup to hdds stored in drawer.
I've got a similar setup, partially mimicking features our UNIX/ Linux work setup had: hourly, daily, weekly and monthly rsync copies between two systems that are used in my home. The one is a desktop that is used most productively, if it's down, no sync is done but it wasn't used anyway.
Then a nightly rsync to an off-site system, an encrypted drive on an open wrt router. I have to log in remotely and connect the encrypted drive if it's been down, but it's very stable, so that only happens when the power g
I have yet to see any problem, however complicated, which, when
you looked at it in the right way, did not become still more complicated.
-- Poul Anderson
Rsync (Score:5, Insightful)
1. rsync to a HDD which is stored in the desk drawer.
2. rsync to a remote HDD (remember that offsite backups are crucial).
Re: (Score:2)
Re: (Score:2)
Seconded. rsync along with its powerful and little known options
I did that for a while, when I was using disks. I rather liked the 3 way Rsync where you have the source, old destination and new destination and it will hardlink to the old destination if there was no change, so you get efficient snapshots.
Re: (Score:3)
Also this (well my main backup hard drive is in a server, is encrypted and normally unmounted), only thing that doesn't back up this way is my gaming PC, where I create a disk image to an external hard drive using the Win7 backup tool. My phone also backs up via rsync through a chroot, although it's kind of a messy system and restoring wouldn't be so straightforward.
Re: (Score:2)
I wish there was a way to full backup my wife's iPhone automatically, that wasn't iCloud. Unfortunately as far as I can tell it's impossible because iOS kills background apps so aggressively. The best suggestion seems to be to keep the screen on for an initial full backup, and then hope the app can upload your new data within the seconds allowed by the OS now and then.
Re: (Score:2)
Well, iTunes can - if you create an encrypted backup. I assume its replacement has the ability to do so too.
iCloud backups aren't full backup
Re: (Score:2)
I only have Windows and need something automatic, but thanks anyway.
Re:Rsync (Score:4, Interesting)
Pretty much this. We live in a forested area so there’s fire danger.
I have an offsite server, a local homelab with 90 tb of disk, and an external 14 tb drive.
Desktop is rsync’d to a VM on the homelab regularly and to the 14 tb drive about once a week.
The offsite server gets my websites and photo library so it’s not a full offsite backup of my desktop.
The 14 tb drive also backs up the vms on the homelab. This includes my terraform and ansible scripts used to build servers.
Basically in the event of fire, I top off the 14 tb drive, drop it into my go bag, grab the important papers, wife, and pets and bug out.
[John]
Re: Rsync (Score:5, Funny)
1 - drive 2 - papers 3 - wife, nice to see you have your priorities straight ;)
Priorities (Score:5, Funny)
Note that the unimportant papers, wives, and pets don't even get a look-in.
Re: (Score:3)
3 things in the world a man never lends out. a) His tools b) his car c) his wife. In that order.
Re: (Score:2)
My wife is more likely to follow me out the door than the papers are.
Re: (Score:1)
I was using amanda (which is just a front end for tar/gzip) for server backups, the backup server powered on (with an apc masterswitch) once a wee
Re: (Score:2)
There was a fire just a few miles away from us (Left Hand Canyon Fire in Colorado). No real chance of it getting to us but we had the cat carriers out and on the table, the two lock boxes with papers, and the drive sitting up next to the door just in case.
And yea, we have high speed WiFi for our network access so over the air backups would take quite a long time. I do burn an occasional DVD with photos and such and send them out to my daughter down on the flatlands and my brothers on the coasts but that's n
Re: (Score:1)
Re: (Score:3)
It would be nice to have a slightly more flexible and integrated system set up though, as well as multiple version support. I have found that anything which isn't entirely automatic tends to get neglected.
By the way, there is a tool called rclone that does something similar to rsync but for cloud services. Handy for syncing off-site backup sets, although you should of course encrypt first. That brings up another issue with many solutions - the encryption prevents effective differential backups.
Re: (Score:3)
By the way, there is a tool called rclone that does something similar to rsync but for cloud services. Handy for syncing off-site backup sets, although you should of course encrypt first.
I believe rclone offers encryption so you can make your unencrypted local backups encrypted on the cloud. Also various cloud services have their own encryption systems.
That brings up another issue with many solutions - the encryption prevents effective differential backups.
Borg can encrypt its database. You provide a key
Re: (Score:2)
Thanks, I hadn't looked at encryption in rsync. I only used it to periodically backup my cloud services. I also use Thunderbird for that, having it automatically start in the middle of the night, sync IMAP mailbox with Gmail and then close.
Borg seems quite good, similar to Duplicati but no Windows support. How robust is it? As I say occasionally Duplicati uploads fail (due to internet connection problems) and you have to run a manual command to recover, but it's never lost any data or completely trashed a b
Re: (Score:2)
[borg] How robust is it? As I say occasionally Duplicati uploads fail (due to internet connection problems) and you have to run a manual command to recover, but it's never lost any data or completely trashed a backup set.
Seems OK. So, the only nonlocal mode it has is SSH, and I use rclone for uploading. I've had many backups abort half way through, and it's something that's explicitly supported. Essentially it always creates BACKUPNAME.[0,1,2,etc], and on completion, renames it to BACKUPNAME. If it aborts,
Re: (Score:2)
Duplicati just creates more encrypted zip files every time, and occasionally removes old ones that no longer contain any required data. Once a file has been made it never gets renamed or added to, it's always a new file.
It seems to handle large amounts of data and large numbers of files without any problem, which is the downfall of many apps.
Re: (Score:2)
Duplicati just creates more encrypted zip files every time, and occasionally removes old ones that no longer contain any required data. Once a file has been made it never gets renamed or added to, it's always a new file.
Borg uses some sort of transactional database (made out of files), so it's just a symbolic name given to a snapshot. Quantity wise I've got one repository doing all my machines and it's fine, though it turns out that's not necessarily the best way of doing things. I'm probably happily into t
Re: (Score:2)
So is rclone smart enough to handle renames on the server and not re-upload? Sounds like a decent solution if you don't need the web UI.
Re: (Score:2)
Not sure what you mean by renames on the server?
Borg produces an opaque* directory structure representing some sort of database and rclone just syncs that to a remote directory, so you can handle moving the database around, but renaming individual files in the database would break the structure. Actual file names are stored inside the database itself.
* opaque for normal use. It's FOSS and documented, but the internal structure is opaque to anyone not digging around in the guts of borg.
Re: (Score:2)
Maybe I misunderstood about the renames. How does it handle versioning, or doesn't it?
What I mean is you say it creates a file BACKUPNAME, so is there an older one like BACKUPNAME.20210307 or something? Like the current one renamed or some sort of shuffle?
Re: (Score:2)
Oh right, the backname is for the snapshot, so each backup is a complete (or incomplete) snapshot of your entire tree. You get file versions by going to older snapshots. So you can list all the snapshots you have, then dive into one to get the file. The files within a snapshot keep their names.
Re: (Score:2)
One other thing about Duplicati, it only keeps one file locally while the upload is in progress so you don't need much disk space if you are only using remote targets. Each file has a max size, I think they default is 50MB.
Re: (Score:2)
That's the downside of my method, you need a disk either on the local machine, or something you can SSH to, and I don't know how well it works over high latency connections. I use my desktop PC to hold that disk.
Re: (Score:2)
2. rsync to a remote HDD (remember that offsite backups are crucial).
Depends upon your level of risk. Do I have irreplaceable data here? If so, then sure, do the offsite. Would you be okay if the building were to burn to the ground? If so, maybe it's not necessary.
Re: (Score:2)
nextcloud to my own internal only server with a nightly rsync from that server to main backup.
Monthly rsync from main backup to hdds stored in drawer.
Re: (Score:1)
If you are using rsync, you might want to checkout Dirvish. https://dirvish.org/ [dirvish.org]
Re: Rsync (Score:2)