Backup Solutions for Mac OS X?

SpartanVII asks: "I purchased a Mac roughly two years ago and have made the switch with a fair amount of ease. However, one thing that has troubled me is how best to backup my important data to an external hard drive. Right now, I have rigged up an Automator workflow that runs every night, but I have also seen software options like SuperDuper and Knox. Since the Automator workflow lacks much of the flexibility and features available with these apps, I am ready to try something else. What app have you come across that provides the best backup solution?"

rsync, bash script, calendar event

[dbc (135354)]

it's unix. except that cron isn't useful on a system that sleeps, and launchd is badly broken in several painful ways. anacron is supposed to be good, but i haven't looked into it.

Re:

[AccUser (191555)]

launchd is badly broken in several painful ways

How so?

Re:

[kidordinn (1034942)]

I've used SuperDuper! for the past two years and I'm very happy with it. SuperDuper! is truly a no-brainer: it checks and fixes file permissions and makes the backup disk bootable. That last function saved me when my iMac internal drive drive died. I lost only a few hours of work and I was able to run my iMac on the backup drive until Apple shipped me a replacement unit. Then it was even very easy to copy all my OS and data back to the new internal drive. SuperDuper! is the best way to protect your work an

Re:

[Burz (138833)]

If his system is 2 yrs old, it could be Panther. That means using rsyncX instead of rsync; it's what I use. If you want snapshot-like backups from rsync, then use rsnapshot (it uses rsync, so on a pre-10.4 system you'll need to replace rsync with the rsyncX version).

If Leopard is on the horizon, then just use the Timewarp(?) snapshot tool built into the OS.

If you want a full image backup done efficiently, then CCC (Carbon Copy Cloner) is the free GUI tool of choice.

Other great options: DAR, rdiff-backup, Un

script for snapshots

[goombah99 (560566)]

find -d /Users/me | cpio -dpl mybackup

This makes a zero space snapshop of me. preserves everything except permissions. This is not a backup it's a snapshot. do the backup with rsync. then take a snapshot.

I sometimes use rdiffbackup too

Re:

[rthille (8526)]

Learn something new every day... I've been working with Unix for about 20 years and haven't really used cpio much. That cpio command will work nicely for me for mirroring my image files into the chroot my webserver runs in...

Re:rsync, bash script, calendar event

[xil (151104)]

On OS X, rsync -E will copy resource forks and extended attributes. Works fine for backup.

Re:rsync, bash script, calendar event

[iangoldby (552781)]

Here's my script. This is an amalgamation of several ideas that I easily found by searching the web. It keeps the last three copies, using links to avoid copying or storing twice any file that hasn't changed.

#!/bin/sh
 
# To use Apple's rsync switch commented lines below
# To use rsyncx:
# RSYNC=/usr/local/bin/rsync --eahfs --showtogo
# To use built-in rsync (OS X 10.4 and later):
RSYNC="/usr/bin/rsync -E -v"
 
DEST=/Volumes/Backups
 
# Function for toggling Spotlight indexing
spotlight_switch()
{
/usr/bin/mdutil -i $1 /
# /usr/bin/mdutil -i $1 /Volumes/Backups
}
 
# sudo runs the backup as root
# --eahfs enables HFS+ mode
# -a turns on archive mode (recursive copy + retain attributes)
# -x don't cross device boundaries (ignore mounted volumes)
# -S handle sparse files efficiently
# --showtogo shows the number of files left to process
# --delete deletes any files that have been deleted locally
# $* expands to any extra command line options you may give
 
# make sure we're running as root
# id options are effective (u)ser ID
if (( `id -u` != 0 )); then
{ echo "Sorry, must be root. Exiting..."; exit; }
fi;
 
! test -d $DEST && echo "Please mount the backup drive!" && exit
spotlight_switch off
 
rm -rf $DEST/backup.2
mv $DEST/backup.1 $DEST/backup.2
mv $DEST/backup $DEST/backup.1
 
$RSYNC -a -x -S --delete --link-dest=../backup.1 \
    --exclude-from backup_excludes.txt $* / /Volumes/Backups/backup
 
# make the backup bootable - comment this out if needed
 
bless -folder $DEST/backup/System/Library/CoreServices
 
spotli ght_switch on

My excludes file:

/tmp/*
/Network/*
/cores/*
*/.Trash
/afs/*
/a utomount/*
/private/tmp/*
/private/var/run/*
/p rivate/var/spool/postfix/*
/private/var/vm/*
/Pr evious Systems.localized
.Spotlight-*/
/Users/*/Library /Caches

The only 'issue' is that I don't seem to be able to boot from the backup, but this may be no bad thing, given that a backup is not supposed to be a mirror, nor a mirror a backup.

Any suggestions (or flames as to why my backup strategy will fail catastrophically) welcomed!

[[Your comment has too few characters per line (currently 25.1). Aenean orci mi, lacinia varius, varius in, suscipit ut, purus. Donec pharetra lorem nec odio. Mauris accumsan sem non pede. Etiam pulvinar eros at massa. Curabitur consectetuer. Pellentesque imperdiet cursus diam. Sed tincidunt nunc. Donec fermentum, nisl at hendrerit mollis, turpis leo consequat elit, volutpat condimentum velit augue facilisis nisl. Vestibulum dapibus ligula non turpis. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Nulla risus lorem, aliquet imperdiet, accumsan id, aliquet vel, tellus. Quisque mi dui, pulvinar ut, iaculis pharetra, rutrum eget, nisl. Aliquam et erat in sem dapibus vehicula. Curabitur rhoncus ipsum id dui. Nullam venenatis. Phasellus vitae sapien quis pede ultrices mollis.]]

Re:rsync, bash script, calendar event

[sribe (304414)]

On OS X, rsync -E will copy resource forks and extended attributes. Works fine for backup.

If you don't mind resetting creation and modification times of every file (not just changed ones) on the backup every time you backup.

rdiff-backup creates and maintains a copy of not only the current data but also keeps reverse diffs so you can recover old versions too.

It's extremely fragile. Any interruption in any backup and it will leave things in a state where manual cleanup and starting the backup over from scratch is required.

Retrospect will compress the data to save drive space, and it allows you to restore via a date of your choice.

It works great when it works. But it also has a nasty tendency to corrupt its catalog files, forcing you to run a "repair" operation on you backups. For disk-based backups this is not too bad since it just takes time; for tape you get to feed in all the tapes in the set so it can read them. This bug has persisted across at least 3 paid upgrades now. Not everybody experiences it, and I don't know what conditions trigger it, but I've seen it at multiple sites with different setups.

As for SuperDuper, I've heard only good things about it. Seems to be a very solid little product for individual backup. I haven't tried it because I need network backup for multiple machines. (I'm so frustrated I'm about 90% of the way to deciding to write my own!)

Re:

[Em Adespoton (792954)]

I suggest you try BackupPC [sourceforge.net] which is extremely configurable, runs over samba, ssh, and other protocols, makes full and incremental backups, and can back up Windows, Linux and OS X machines. I've used it for years in network settings, and it hasn't failed me yet. --The best part is the restore process; a user can log in to the web interface and select individual files from any of their backup sets and those files will automatically be restored to their computer via the same network interface used for backup

backups

[nelomolen (128271)]

I use RsyncX (http://archive.macosxlabs.org/rsyncx/rsyncx.html) on the OSX server (10.3) in a lab I do some work in. It works well, and you can just set up cron jobs. Last I checked, the Rsync that comes with OSX wouldn't handle resource forks, which is why a third-party app is necessary. This may be fixed in newer versions of OSX, but since the lab isn't upgrading until 10.5 is released I have no experience with 10.4.

Re:

[mattjb0010 (724744)]

10.4 rsync has
-E --extended-attributes copy extended attributes, resource forks

rdiff-backup

[swillden (191260)]

rdiff-backup creates and maintains a copy of not only the current data but also keeps reverse diffs so you can recover old versions too. You can backup to another hard driver or directory, or over a network. For remote backups, it uses the rsync protocol so it only transmits changes.

It's a command-line tool, so it's not very OSX'y, but it works very, very well. I use it to back up all of my machines, including some remote servers. I do it all with cron jobs, and all over network links, because that way I can just ignore it, but you can also run it manually if you prefer.

SuperDuper!

[coffee_bouzu (1037062)]

I'm a big fan of SuperDuper! since it's trivial to use, does incremental backups and you don't have to worry about missing files or applications if you mirror your entire hard drive.

If you have a firewire external hard drive, you can have SuperDuper! backup your computer's drive to it and if you should ever want to step back to your last backup or lose your laptop's hard drive, all you have to do is plug in the external drive, press option while you are starting up your mac, boot from the external drive, ru

Re:

[piojo (995934)]

SuperDuper! is for people that don't want to take the time to mess with shellscripts and cronjobs and such. Back up everything with almost no options and nothing to screw up. If you wanted to muck around with shellscript and rsync backup solutions, you probably wouldn't be asking here. I installed SuperDuper for my boss and did a couple backups for her (never had to restore), and it seems like a wonderful piece of software.

Re:

[pboulang (16954)]

SuperDuper! is for people that realize that reinventing the wheel is bad use of their time. Though with time machine Jaguar(OS X 10.5), it will become obsolete. Rsync is still quite viable in remote mirroring, though.

Re:

[eschasi (252157)]

Not. If your primary fs is totally borked, time machine is no help. I'm not a SuperDuper user myself, but from the descriptions given here it's clear that it makes a backup onto a second disk that is a bootable copy of your primary as of the last sync.

Re:

[pboulang (16954)]

I'm not sure that time machine wouldn't do the same thing. It is a pretty motivating feature to include. I guess we'll find out on release. Until then, I have zero qualms about paying for superduper! as a stopgap. Cheap, effective, straightforward.

Re:

[pboulang (16954)]

duh, yes of course.. oops.

Cross-Platform Solution

[RAMMS+EIN (578166)]

I'd rather not use a backup solution "for Mac OS X". Instead, I'd use a solution that works on multiple platforms. The main argument for this is that I can still use the solution and access my old backups when I decide to switch platforms (and, consequently, that the backup solution isn't an impediment to switching). I also have the feeling that if software is used and maintained on multiple platforms, there is a lesser chance of it just being end-of-lifed one day than if the software is just for one platfo

Re:

[billsoxs (637329)]

So what DO you use? I agree with what you are saying but.....

Re:

[RAMMS+EIN (578166)]

``So what DO you use? I agree with what you are saying but.....''

Well, gnutella is portable...

Re:

[RAMMS+EIN (578166)]

``So what DO you use? I agree with what you are saying but.....''

For now, rsync and a couple of scripts. I regularly sync the important parts of my home directory between 3 different machines. Soon, I will add a fourth, off-site one. I don't bother backing up anything outside my home directory, as I can get all that with apt-get. Perhaps one day I will make a couple of files containing a list of packages I have installed.

I've looked at a couple of dedicated backup solutions, but, so far, the conclusion has

Re:

[Burz (138833)]

If you don't opt for Subversion, consider the rsync-like tool Unison. It can synchronize both-ways at once and their site has tips for syncing 3+ machines.

Will unison behave correctly if used transitively? That is, if I synchronize both between host1:dir and host2:dir and between host2:dir and host3:dir at different times? Are there any problems if the "connectivity graph" has loops?

This mode of usage will work fine. As far as each "host pair" is concerned, filesystem updates made by Unison when synchroniz

Re:

[knappe duivel (914316)]

Most backup programs just copy the files, so you are in no way tight to, or dependant on such program. I do avoid programs like Retrospect, which compress the backups, forcing you to also use the program for restoring of browsing your backup data.

Re:

[Osiris Ani (230116)]

Instead, I'd use a solution that works on multiple platforms.

That's one of the reasons why I prefer [Symantec] VERITAS NetBackup Enterprise Server — running on a Sun E450 with Solaris 9 — as my personal backup solution. Its native format is tar, so I should never have any problems with data retrieval, regardless of whether or not I have access to that particular app.

Well, there's that, and my great love of overkill. It's akin to using an interocitor [wikipedia.org] to make popcorn.

Re:

[Burz (138833)]

The good cross-platform tools IMO are rsync (rsyncX for pre-10.4 systems), rsnapshot, DAR, rdiff-backup, Unison.

Here is my script for rsyncX that handles resource forks- Note the list of excludes you can edit starting with /afs:

#!/bin/bash

sudo time rsync -vaxHS --delete --eahfs --showtogo \
--exclude-from=- /. /Volumes/destinationdrive <<- _END_
/afs/*
/private/tmp/*
/private/var/vm/swa pfile*
/private/var/run/*
/Network/*
/Library/C aches/*
/.Trashes
_END_

Retrospect

[MrGHemp (189288)]

http://www.emcinsignia.com/ [emcinsignia.com]

We use it to back up our web and database servers. The high end products might be over kill but the Express version might do you right. Retrospect will compress the data to save drive space, and it allows you to restore via a date of your choice. Lots of scheduling and etc options. Works like a champ.

Synchronize Pro X

[MacMerc.com (470860)]

Although it is on the expensive end of the backup software scale, Synchronize Pro X [qdea.com] is extremely versatile and has saved my bacon through a series of drive failures (that resulted in Apple replacing my PowerBook). I currently have it running 4 different scheduled backups on my system and have another backup set up that activates when I attach a certain thumb drive: it syncs a selected group of folders to the thumb drive and then unmounts the drive automatically. Plug, sync, unplug--very cool.

rsnapshot

[perlionex (703104)]

I use rsnapshot [rsnapshot.org]. It's written in Perl, and uses rsync, so it should work on Mac OS X as well as it does on my Linux box. It's pretty configurable, and rotates backups hourly, daily, weekly, monthly, etc. It uses filesystem hardlinks to do incremental backups.

Today shell scripts, tomorrow Time Machine

[Fulkkari (603331)]

I don't know about right now, but once Leopard comes out, I guess it would be Time Machine [apple.com]. Just wait until it starts shipping in the beginning of the next year.

If you don't want to wait or upgrade, write a shell script doing the job for you. I don't know what kind of experiences others have had with backup tools on the Mac, but Retrospect kept crashing on me when trying to run it. I wouldn't trust that kind of software to keep track of my backups. So I guess it's pretty much shell scripts or nothing right now.

Re:

[dangitman (862676)]

I don't know about right now, but once Leopard comes out, I guess it would be Time Machine. Just wait until it starts shipping in the beginning of the next year.

I'd advise against doing this. Backup solutions should really be time-tested and proven. I would not adopt a newly-released backup solution. Especially one from Apple. As much as I love Apple software, their existing backup software, "Backup" is an absolute disaster. Given this track record, I see no reason to trust Time Machine.

I currently use Retrospect, but recent versions haven't been too impressive either. I'd take the recommendations of checking out Super Duper and perhaps Carbon Copy Cloner. I've u

Re:

[Yvan256 (722131)]

Except that for most people, an unknown backup method is better than no backup at all, which is the point of Time Machine.

Me? I'm sticking with monthly backups to CD-Rs. Time Machine will simply add a daily fallback.

Re:

[Coppit (2441)]

What I'm thinking about is using Amazon's S3 service [slashdot.org] along with JungleDisk [jungledisk.com] to get a cheap online, reliable, unlimited virtual drive for Time Machine to store its backups on. I just hope that Time Machine is smart enough to queue up its transactions when the network storage is not available. I also wonder what the performance will be like.

Things are moving fast in this space. I'd love to see a general online storage solution with WebDAV support, something like Gallery2 or Flickr built-in, permissions manag

Required reading

[pesc (147035)]

Backup on Mac is not as easy as one would think...

http://blog.plasticsfuture.org/2006/03/05/the-stat e-of-backup-and-cloning-tools-under-mac-os-x/ [plasticsfuture.org]
http://blog.plasticsfuture.org/2006/04/23/mac-back up-software-harmful/ [plasticsfuture.org]

Maybe TimeMachine will offer an interesting solution...

http://www.apple.com/se/macosx/leopard/timemachine .html [apple.com]

Don't use Backup from dotMac!

[wembley fraggle (78346)]

I used the Backup application from dotMac faithfully for over a year. Ran well every night, backing up my system. Then, my computers were stolen. No problem, they were insured and I had a backup. These things happen. I got my new Macintosh and went to Backup.app to restore. Selected everything and hit restore.

Backup crashed.

Tried again. Crashed again.

Backup won't restore more than one or two files at a time without crashing. It seems to be a memory leak, as it dies during a memory allocation routine. Granted, I had a lot of files and a lot of incrementals. But this is the JOB OF BACKUP! To be able to RESTORE my FILES! The files are there, I can see them (each backup file has a disk image inside it which you can mount manually). I just can't get at them systematically.

So, I contacted Tech Support. Got something like "wow, that's strange", sent my logs and such. It's been two weeks and I've heard nothing. My followup emails go into the bit-bucket.

By now, it would have been easier for me to have spent the last four nights manually mounting disk images and copying files over by hand.

Needless to say, I'm going with Retrospect as soon as I have something to backup again. Cancelling my dotMac account too.

Re:

[Therin (22398)]

That's a major bummer. My experience is quite different, over the last couple of years I've used dotMac Backup.app three or four times to restore things, and it's worked just fine.

ChronoSync

[Red_Winestain (243346)]

I use ChronoSync [econtechnologies.com].

I'm happy to fiddle and tweak and produce home-brew solutions to many things, but not as the sole backup: The point of a backup program is to ensure that you have backed up exactly what you think you have backed up. ChonoSync provides a reliable and flexible back-up system. It is commercial ($30) -- which you may not like -- but they offer free updates to a reasonably priced product, and have been around for a while. Their customer service is also excellent: they provided a less restrict

Retrospect

[Rick Zeman (15628)]

I back up my desktop, my PB, the wife's PB, my Dell, and my Linux server to an extra hard drive in my Desktop. Always been able to restore files that I've needed, and I've had to do one bare metal restore of my PB. Did a barebones install of 10.4.x on it, added the Retro client, clicked the mouse a few times...and came back to MY perfectly functioning PowerBook. A lot of people here will sneer at a commercial solution, but that restore paid for the software in my time and aggravation not spent.

Oh no, not again...

[gidds (56397)]

This seems to have been discussed in many places over the last couple of months.

I'm no expert, but I can point you to a couple of interesting web pages by people who do seem to know a lot of the details:

• Mac Backup Software Harmful [plasticsfuture.org] and the earlier The State of Backup and Cloning Tools under Mac OS X [plasticsfuture.org] at plasticsfuture
• MacOS X Backups [dataexpedition.com] at Seth's Unix Tips

In short, there are lots of different backup and cloning tools, from the Unix cp, ditto, and rsync commands up to the free Carbon Copy Cloner [bombich.com], cheap SuperDuper! [shirt-pocket.com], and expensive Retrospect [emcinsignia.com]. And very few of them preserve everything. HFS+ carries a lot of baggage from the old Mac OS, and adds a lot more stuff from Unix: there are resource forks, HFS+ extended attributes, BSD flags such as creation date and owner/group permissions, ACLs, symbolic links, aliases, and lots more -- and almost none of the options can preserve all of those.

You also need to think about what your backups are for and how much time and money you're prepared to expend: for some, burning a few personal files to CDR every few months will suffice, whereas for others an external HD holding a complete clone is the thing, and power users may need daily or weekly incremental backups with the ability to retrieve any file going back years.

Personally speaking, I'm in the middle category, with a large external Firewire HD holding a clone of each of my drives, which I redo every month or so. (Having it bootable is also a good idea, and has saved my bacon at least once!) I've mostly been using Carbon Copy Cloner, which has given good results, but I've recently switched to SuperDuper! which is cheap and seems to preserve absolutely everything. But don't take my word for it: read the linked pages, work out your needs, and make up your own mind.

But DO think about it! Disaster WILL strike in some form or other; disks DO fail (as I know to my cost), and you need to plan for it. It's not a question of how much time or money you can afford to spend; it's a question of how much data you can afford to lose!

Deja Vu, it comes with toast

[k3v0 (592611)]

you can set it to back up over the network or to another drive, you can specify manual or automatic, and you can schedule different backups at different times. it's easy and quick.

harddisk no problem

[Tom (822)]

For external drives, there is plenty of software around. iBackup is what I have installed and it does what you want.

What I'm looking for and haven't found yet is something that'll do backups over the network, and is not .Mac

Shell scripts

[lar3ry (10905)]

As mentioned in another article, shell scripts are usually the best. I'll look at Time Machine when it comes out, but the very fact that it's OS X-specific would relegate it to a curiosity for me.

OS X provides "rsync," which is one of the best tools for the job, and it works on most (all?) Unix-based platforms as well as Windows (using Cygwin). With rsync, you should definitely look into the following options:

--exclude (exclude file name patterns from being backed up. You don't really nead your web cache

Deja vu of course

[Pliep (880962)]

I always use Deja Vu and always keep returning to it: http://propagandaprod.com/ [propagandaprod.com]

It uses psync (like rsync but with resource forks etc.) and is generally brilliant. I simply create an incremental duplicate of my entire hard drive to an equally sized other hard drive every day at 6 PM.

ChronoSync

[varontron (460254)]

from EconTechnologies [econtechnologies.com] is my choice. It's easy to use, supports archiving, and unattended operation. That's pretty much all I need. I back up my home folder with all my shtuff, and /usr/local where I have data and config files. Everything else in my world is downloadable, configurable, or forgotten. If I lose my hard drive once a year, I'll spend less time rebuilding then I would searching for and configuring a more advanced backup package.

Try Backuplist+

[cellocgw (617879)]

Backuplist+, available at the usual search engines in your neighborhood :-), is a pretty nice gui frontend to several backup options. You can select Finder-like copy/backup, rsync, ditto, and cpio IIRC. It's free/donationware, so I'd recommend at least reading thru the documentation to see if it'll do all you need.

Re:

[billsoxs (637329)]

I have done something like this - I run dual raid (mirrored) drives on my main computer. It would be nice to have a similar method for the laptop....

Re:

[billsoxs (637329)]

It is if all that you are worried about is a disk crash..

Re:

[ICantFindADecentNick (768907)]

Come on - that's just mean. I might've modded funny but somebody might actually try it.

Re:

[DannyO152 (544940)]

Connecting the laptop via FireWire cable after starting up in remote drive mode (hold down t after powering on) and copying the entire home directory from one to the other will do it. The working account should not have administrative powers. I'm guessing that it will go more smoothly if the working account on both machines were set up identically with the same name, password, userid, and groupid. This should happen if they were established at the same sequence point (next account after the initial one, per