Forgot your password?
typodupeerror
Data Storage OS X Linux

Best Filesystem For External Back-Up Drives? 484

Posted by timothy
from the just-mirror-to-the-internet dept.
rufey writes "I've recently embarked on a project to rip my DVD and CD collection to a pair of external USB drives. One drive will be used on a daily basis to access the rips of music and DVDs, as well as store backups of all of my other data. The second drive will be a copy of the first drive, to be synced up on a monthly basis and kept at a different location. The USB drives that I purchased for this are 1 TB in size and came pre-formatted with FAT32. While I can access this filesystem from all of my Windows and Linux machines, there are some limitations." Read on for the rest, and offer your advice on the best filesystem for this application.
"Namely, the file size on a FAT32 filesystem is limited to 4GB (4GB less 1 byte to be technical). I have some files that are well over that size that I want to store, mostly raw DVD video. I'll primarily be using these drives on a Linux-based system, and initially, with a Western Digital Live TV media player. I can access a EXT3 filesystem from both of these, and I'm thinking about reformatting to EXT3. But on Windows, it requires a 3rd party driver to access the EXT3 filesystem. NTFS is an option, but the Linux kernel NTFS drivers (according to the kernel build documentation) only have limited NTFS write support, only being safe to overwrite existing files without changing the file size). The Linux-NTFS project may be able to mitigate my NTFS concerns for Linux, but I haven't had enough experience with it to feel comfortable. At some point I'd like whatever filesystem I use to be accessible to Apple's OS X. With those constraints in mind, which filesystem would be the best to use? I realize that there will always be some compatibility problems with whatever I end up with. But I'd like to minimize these issues by using a filesystem that has the best multi-OS support for both reading and writing, while at the same time supporting large files."
This discussion has been archived. No new comments can be posted.

Best Filesystem For External Back-Up Drives?

Comments Filter:
  • Don't bother (Score:2, Informative)

    by Anonymous Coward on Wednesday December 23, 2009 @07:01PM (#30539822)

    If you're like me, you won't be happy with the compromises you have to make when picking a multi-platform filesystem. I'd outline them, but you've done a great job of doing so above. So, what to do?

    Get thee a cheap, cheap Linux box, format your drives EXT3, and all other machines access over the network. It's the only way you'll get the interoperability you want, without making compromises on max file size, cluster sizes, etc.

  • Re:The solution.. (Score:5, Informative)

    by Dynedain (141758) <slashdot2NO@SPAManthonymclin.com> on Wednesday December 23, 2009 @07:05PM (#30539852) Homepage

    Thats exactly what I did. Threw a couple of external drives on a Mac Mini. Formatted as HFS+ and did software array. Then using afp and smb provided the contents as shares to the Windows media center and various client machines on the network.

    Sure, software raid over USB is slow, but the bottleneck is the network so it doesn't really matter.

  • Re:Ext3 (Score:5, Informative)

    by moderatorrater (1095745) on Wednesday December 23, 2009 @07:05PM (#30539856)
    I personally go the other way. Sure, there's an Ext3 driver for windows, but from what I've seen it's not that good. On the other hand, I've used the NTFS driver on Linux quite a bit and it's worked pretty well. More importantly, I have confidence that the NTFS driver will continue to get better.
  • Fat32 and VLC (Score:3, Informative)

    by caubert (1301759) on Wednesday December 23, 2009 @07:07PM (#30539882) Homepage
    VLC can play rar-compressed-splitted files beautifully, So 4GB is not a very big problem
  • by BitZtream (692029) on Wednesday December 23, 2009 @07:10PM (#30539916)

    Via FUSE you'll get consistent features and useability across all 3 OSes. Of course moving zfs drives between those OSes isn't something I've tried, but in theory it should work fine.

    Not what your asking for, but Id put a FBSD samba server up with ZFS drives. You can still mount them on other OSes later if need be via FUSE.

  • Re:The solution.. (Score:4, Informative)

    by Keruo (771880) on Wednesday December 23, 2009 @07:11PM (#30539922)
    One shoud never consider raid vs synced copies, use both simultaneously. They protect against different data-loss threats which aren't mutually exclusive.
  • Re:NTFS (Score:3, Informative)

    by elh_inny (557966) on Wednesday December 23, 2009 @07:11PM (#30539930) Homepage Journal

    I second this choice.
    For some zealots it's hard to admit but the performance is really good, you have commercial backing of the biggest software company on the planet.

    Recently a commercial company (Tuxera) was formed to provide commercial support for NTFS-3G and provide paid-for version of the driver for MacOS and Linux in addition to the free NTFS-3G.

    So the future and cross platform access is looking really good.

    On the other hand, if I were just a little bit more adventurous, I would much rather use Sun ZFS for storage to have even better reliability, flexibility than with equivalent hardware RAID. But this pretty much requires a separate NAS box running OpenSolaris just for that.

    For me and many other garage hackers that just doesn't cut it, all I have is one laptop running a really fancy looking BSD and two external drives (NTFS) and some backup scattered over the Internet...

    All I'm hoping for is that one day Apple will reintegrate ZFS support, it's already been promised, implemented and now ditched...

  • by FreelanceWizard (889712) on Wednesday December 23, 2009 @07:13PM (#30539956) Homepage

    Honestly, if FAT32 won't do what you need, NTFS is pretty much where you'll need to go. NTFS-3g [tuxera.com] gives you stable read/write capability on Linux and OS X as a FUSE driver; in fact, many distributions have NTFS-3g in their repositories. There's also native NTFS write support in Snow Leopard if you want to risk turning it on. I personally haven't had any issues with it, but some people have encountered file corruption when using it, so you might want to be wary. It is worth noting, however, that many embedded devices won't read anything other than FAT. If you plan on hooking this drive up to, say, a DVD player to show pictures, NTFS won't work for you.

    Like it or not, Microsoft file systems are the lingua franca of file transfer on portable drives these days, merely due to the installed base of Windows computers.

  • Words of caution (Score:5, Informative)

    by RenHoek (101570) on Wednesday December 23, 2009 @07:14PM (#30539970) Homepage

    I have ~6TB on external USB drives and I've been doing this for a few years now. I have a few words of caution about NTFS. If you get an USB drive that for example spins down or if you turn your USB drive off without properly dismounting it (or if Windows crashes), you might see this line:

    Delayed write failed!

    And on two occasions that meant that Windows fucked up the file allocation table or whatever it's called under NTFS and I lost the _entire_ disk.

    Windows loves getting its fingers into that table whenever you mount a USB filesystem. It's not like it tries to keep its write cache empty. Nooo.. every file access needs to be continuously recorded in that thing.

    Anyway, be careful when you use NTFS on a USB drive. Alternatively use EXT3, which you can still mount under Windows using:

    http://www.ext2fsd.com/ [ext2fsd.com]

    (Note that these experiences are under Windows XP, I have no clue if Vista or 7 does any better, I assume not.)

  • Re:NTFS (Score:2, Informative)

    by rhavenn (97211) on Wednesday December 23, 2009 @07:22PM (#30540018)

    I second this choice.
    For some zealots it's hard to admit but the performance is really good, you have commercial backing of the biggest software company on the planet.

    You only have commercial backing if the OS you're running it on starts with Windows and doesn't include Linux, Solaris, BSD, AIX, Haiku, Amiga OS, etc... in the name.

  • Re:NTFS (Score:3, Informative)

    by rpresser (610529) <rpresser@[ ]il.com ['gma' in gap]> on Wednesday December 23, 2009 @07:22PM (#30540028) Homepage

    UDF fits that bill, doesn't it?

  • Re:Ext3 (Score:3, Informative)

    by xSauronx (608805) <xsauronxdamnit.gmail@com> on Wednesday December 23, 2009 @07:23PM (#30540034)

    yeah, i use linux and have my exteral drive formatted as NTFS

    honestly, i cant recall having a single write-related problem and i do raw DVD video often enough. linx NTFS support has been solid for me for well over a year now.

  • Re:I wouldn't.... (Score:4, Informative)

    by Barny (103770) <bakadamage-slashdot@yahoo.com> on Wednesday December 23, 2009 @07:23PM (#30540036) Homepage Journal

    I used to use freeNAS, but after a while I just wanted more than what it was offering.

    I switched it for a windows home server (server 2003 SBE based), mainly for the backup features, and what with the freeNAS machine being the only non-windows machine left in my house it didn't matter that it lacks full compatibility with unix.

    But yes, freeNAS is damn good at what it does, have set up some nice diskless server systems with freeNAS running from a USB stick and having all the client machines on the network sharing their drives with iSCSI, freeNAS would collect them all, turn them into a big redundant storage array, and share them back to the network, works well :)

  • Re:The solution.. (Score:5, Informative)

    by multimediavt (965608) on Wednesday December 23, 2009 @07:25PM (#30540054)

    WRONG!

    Irregardless [reference.com] is not a proper English word. Its usage has *ALWAYS* irked me from when I was a small boy to now. To use common vernacular, it's a mashup of 'irrespective' (one negative; prefix) and regardless (also, one negative; suffix). 'Irregardless' is a double negative and is thusly illogical by construction and would only be understandable to people born in the U.S. since 1970, and those less literate in the U.S. prior to that.

    On my words that aren't words list it's right up there with 'impactful'.

  • by Anonymous Coward on Wednesday December 23, 2009 @07:26PM (#30540062)

    Sharing a zpool between different OS installations doesn't work: First of all, you have to be careful when chosing the zpool version you'd like to use (they're not downwards compatible!). While this can still be solved, the zpool must know of which device(s) it consists, and they will vary from machine to machine (particulary when using an external drive, but it doesn't work with internal disks and two different operating systems booting on the same box either).

  • Re:The solution.. (Score:2, Informative)

    by bertoelcon (1557907) <.berto.el.con. .at. .gmail.com.> on Wednesday December 23, 2009 @07:33PM (#30540110)

    Yes it is. [reference.com]

  • by Mr. Protocol (73424) on Wednesday December 23, 2009 @07:40PM (#30540158)

    Sadly, if you create a ZFS tank on a Solaris box and then move the tank physically to a FreeBSD 8.0-RELEASE machine, it won't even see that there's a tank out there. Apparently GPT table layout is different on FreeBSD or something.

    Won't stop you from serving ZFS over NFS/Samba/whatever, but you can't move the tank itself around. I know, I tried. Booted FreeBSD on a machine with a Solaris-issue ZFS tank, and it was like it wasn't there. It saw the drives fine, just not the tank.

  • Re:The solution.. (Score:3, Informative)

    by jeffehobbs (419930) on Wednesday December 23, 2009 @07:45PM (#30540198) Homepage

    I second this (DNS-323 myself). Runs like a champ, very low-power, files
    available to every machine (and a WD TV Live) from any room in the house.

  • by Mr. Protocol (73424) on Wednesday December 23, 2009 @07:54PM (#30540282)

    I should mention that the tank on disk was ZFS v4, so it was not a case of the Solaris tank being of a rev level higher than what FreeBSD could handle.

  • Check Tuxera NTFS (Score:2, Informative)

    by replicant_deckard (447694) on Wednesday December 23, 2009 @07:55PM (#30540296) Homepage

    If you need Linux/Mac/Windows interoperability then we recommend NTFS for both Linux and Mac users. Instead of the old NTFS kernel driver you may want to check our open source NTFS-3G. It has read/write, and tons of options:
    http://www.tuxera.com/community/ntfs-3g-advanced/ [tuxera.com]

    If you need just high-performance NTFS read/write, this is our offering for Mac users:
    http://www.tuxera.com/products/tuxera-ntfs-for-mac/ [tuxera.com]

    If you need high-performance for a commercial Linux application or device, you may want to check this:
    http://www.tuxera.com/products/tuxera-ntfs-commercial/performance/ [tuxera.com]

    Regards,

    Mikko Välimäki
    CEO, Tuxera Ltd

  • Re:Words of caution (Score:4, Informative)

    by compro01 (777531) on Wednesday December 23, 2009 @07:56PM (#30540302)

    Turn off write caching for the drive and this problem goes away. It's supposed to be off by default (at least on removable drives, but some IDE/SATA-to-USB bridges show up as normal fixed drives rather than removable for whatever reason), but I've found it seems to turn itself on for whatever stupid reason.

  • Openfiler anyone? (Score:2, Informative)

    by lacourem (966180) on Wednesday December 23, 2009 @08:04PM (#30540370)
    Personally I like Openfiler. It can be picky about the hardware though. With that said, the speed is great, and I can mount iscsi on linux and windows. Has been stable as hell for me to boot.
  • ext3 (Score:2, Informative)

    by solid_liq (720160) on Wednesday December 23, 2009 @08:16PM (#30540466) Homepage Journal
    I go with ext3 for this personally. NTFS doesn't store *nix style filesystem permissions, and causes various other issues with you Linux systems. With ext3, you can store all your files with all of your permissions intact, the filesystem is mature and trustworthy, and you can still access all of the files from any operating system by simply connecting the drives to a dedicated fileserver machine (an older computer or small device works perfectly for this). Simply share your files via NFS, Samba and ftp (if you need ftp access for something like xmbc). Having a dedicated machine for this means you can also script your replication to the secondary drive, so that you only have to attach the drive for the mirroring process to take place.

    This is the solution I've been using for about four years now, and it works great for me.
  • Re:Ext3 (Score:2, Informative)

    by IdleTime (561841) on Wednesday December 23, 2009 @08:26PM (#30540546) Journal
    I have a Seagate 1TB USB drive and I used it both under Windows and Linux, so the FS is NTFS:
    Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
    255 heads, 63 sectors/track, 121601 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0xa4b57300

    Device Boot Start End Blocks Id System
    /dev/sdb1 1 121601 976760001 7 HPFS/NTFS

    I currently run Linux exclusively and I'm going to buy the new 2TB drives and I will format it with btrfs since I will only be using it under Linux, but I have had no problems reading and writing to the drive under Ubuntu 9.10 nor under Windows XP.
  • by Kappy (23246) on Wednesday December 23, 2009 @08:35PM (#30540614)

    My backup has been worked good for the last few years. I had a large windows tower filled with hard drives...
    * One of my HDD was encrypted using Truecrypt.
    * A scheduled Acronis task would image my OS drive to the encrypted drive.
    * I have another encrypted HDD in one of these KINGWIN KF-91-BK trays (http://www.newegg.com/Product/Product.aspx?Item=N82E16817121172)
    * After Acronis was done, I would kick off a little batch file to mount the tray drive (which was encrypted) and run SyncBack Pro to mirror the encrypted source. Then dismount.

    I just bought a few extra trays. Each month I sync all the other HDD in the system (including the encrypted drive) for my off-site backup. One flaw in my system was if one of the other "data drives" crashed, I would loose at most a month of changes. They were just music/videos and didn't change very much. I accepted this fault.

    I out grew the tower and recently built a cheap unRaid box. Most if it should work the same, except now I have redundancy for all my drives in the case of a crash.

  • Re:Don't bother (Score:4, Informative)

    by tknd (979052) on Wednesday December 23, 2009 @08:36PM (#30540626)
    If you're going to build a box, why not use FreeNAS with ZFS? It installed to a USB stick and everything is configurable from a web interface.
  • by joseph.spiros (640998) on Wednesday December 23, 2009 @08:38PM (#30540642) Homepage
    zfs-fuse does not currently compile for platforms other than Linux, despite FUSE's general compatibility with other systems. The Mac OS X ZFS port worked for me under Leopard, and I think it works under Snow Leopard now, though it is no longer being developed by Apple. Last I checked, the ZFS pool version supported by these two implementations differs, so you'll need to create your pool with the latest commonly supported pool version to ensure compatibility. In short, I do support the use of ZFS, and I personally use zfs-fuse, but it's of far more use when shared over the network due to the problems with ZFS implementations on various platforms.
  • Re:Ext3 (Score:5, Informative)

    by Sophira (1364317) on Wednesday December 23, 2009 @08:42PM (#30540666)

    Sure, there's an Ext3 driver for windows, but from what I've seen it's not that good.

    Which particular driver are you referring to? There are a few.

    Personally, I use Ext2 IFS [fs-driver.org] in Windows (it works for Ext3 too) and it is, hands-down, the stablest and best Ext2/3 Windows driver I've used. Every other one I've tried would have stability issues; with IFS I don't have to worry. (There's been precisely *one* time in pretty much years that the driver crashed on me, and that's when I was doing something weird and stupid; I don't remember what. But more importantly, it didn't do anything bad to the filesystem in that crash.)

  • PSA (Score:2, Informative)

    by dotgain (630123) on Wednesday December 23, 2009 @09:08PM (#30540848) Homepage Journal

    Ignore this thread. Once you get past all the first post trolls, the rest of it mainly contains bickering over whether "irregardless" is a word or not. Slashdot at its finest, I know. So I'm jumping in at the top of the thread to warn you and save you the time.

    As for which filesystem to use: It's a shitfight at the end of the day. OSS-NTFS is reverse engineered so you can never trust that it really is complete, nor that Microsoft will never bend/break it in the future. FAT32 sucks but is unquestionably the lowest common denominator. I like to use Ext2 for getting big files from Linux to Windows (I transfer a lot of DV files), but you have to install something on the Windows/Mac/etc. to use that. HFSplus support on Linux is fine, you just need to disable the journal on the Mac first.

    Like I said, it's a shitfight. Learn the pros and cons of the main filesystems: NTFS, Ext-2/3 family, HFS, FAT32. Break them out as each situation merits. But with regard to NTFS and the specs changing: Microsoft have done it before and they'll do it again, just ask Norton Utilities. While my Windows PC is a very important part of my operation (I edit all my video on it), I don't use Windows for enough other things to justify using NTFS unless I'm forced to.

  • Re:The solution.. (Score:3, Informative)

    by Sancho (17056) on Wednesday December 23, 2009 @09:55PM (#30541078) Homepage

    100mbit link provides approximately 10MB/s. It's the difference between megabits and megabytes. I get about twice that on my USB drive. With gigabit, I can get around 50MB/s, and then the USB drive becomes the bottleneck.

  • Please read this (Score:5, Informative)

    by Anonymous Coward on Wednesday December 23, 2009 @10:00PM (#30541114)

    Dear AP,

    I was pondering with a very similiar problem two years ago, when I bought my first terabyte-class external USB hard drive to my home server (old 800 MHz Duron). I was thinking exactly like you are thinking now, and decided FAT-32 would be the way to go. MISTAKE. Four important things stand out, that I want you to read:

    1. You never really transfer that drive. Much less than you think. Chances are more than 99% of their lifetime they will be sitting hooked to one computer, never really being moved; portability is just another nice extra feature that you hardly ever use. During the two years I have switched my drives a couple of times between the desktop machine and the server, both of which run Linux as main OS. I have never ever taken any of these drives outside of my home.
    2. 4 GB limitation is really bad! Most DVD ISOs are bigger that that. An hour of HD video in state-of-the-art H.264 is more than 4 GB. And rest assured, when you have the space and facilities to accquire gigabyte-class multimedia, the temptation will be there. As BluRay becomes the new DVD, maybe you want to RIP your fav. movies to your hard drive for quick access? NO WAY with FAT-32!
    3. Lack of UNIX parameter support. Okay, so you just want to store back-ups? Okay.. Remember than FAT-32 doesn't support symbolic links, file ownership, user/group/others access permissions, file name character case (in Microsoft Windows, "Soviet Union" equals to "soviet union"; WILL result in a conflict when copying data from UNIX filesystems!). This information is LOST, unless you use some container format like tar (but remember the 4GB limit again). These little things are a) very helpful everyday things, value of which you realize only after loosing them (e.g. any file on extfs can be replaced/virtualized without moving files around; it can even point to a non-existstent file! And all works seamlessly, as long as the program understands symbolic links; now how valuable is that?), and b) what makes your UNIX fs work. The value of your backups is lower if they dont work "out of the box", e.g. data is lost when transfering to FAT-32. I mean, you just have a chance to save so much hassle there.. When needed, you can NFS-mount the filesystem (and its free space and contents) to your local machine from your drive, and everything works transparently to BOTH Windows and Linux (the properties of FAT-32 are a small subset of those of extfs.)
    4. Acccess speed. Ext3 and Ext4 or just about any 21st century UNIX fs are lightyears ahead of the archaic FAT in data structures. E.g. if I "ls" a big directory on my only FAT-32 drive, it is SLOW! You can see the entries being fetched one by one. Whereas, if I do the same in a similiarly-sized directory on the ext3, the files appear immediately! Access is almous immediate even over NFS mount in LAN. This comes handy, rest assured.

    Okay, those are my four vocal points. They could be in any order, because all of them are equally important reasons NOT to choose FAT-32! As it happened, after using the 0.5TB drive for 6 months with FAT-32, I bought more space (a new drive). This time there was no question about the filesystem. I made a small, few-sector long 200 MB FAT-32 partition to the beginning of the drive and downloaded all the latest Win32 EXT2 drivers there from different vendors, just for the really unplausible situation that I would ever want to mount these drives in Windows. Then I just made the rest Ext3. And, I am REALLY satisfied with the decision! Ext3 just work so sparklingly faster and better with Linux than FAT-32 ever does. Since then I have bought one more drive and did the same 200MB + 1TB thing. I will probably never use these drives in Windows, but it gives me a warm feeling to the heart that there's always a way if I should, even if the computer doesn't have an Internet connection.

    Oh, one thing I forgot to mention: get a file server! It makes your life so much easier. Nowadays I am running a desktop computer with a 60 GB SSD drive and no HDD at a

  • Re:The solution.. (Score:3, Informative)

    by Jah-Wren Ryel (80510) on Wednesday December 23, 2009 @10:04PM (#30541148)

    You can't just create words out of the blue. People will never be able to grok your meaning if you do. You know I'm right, so don't be all fugnutish about it.

    Don't be such an asshat. [newsvine.com]

  • Re:Ext3 (Score:4, Informative)

    by creepynut (933825) <teddy(slashdot)&teddybrown,ca> on Wednesday December 23, 2009 @11:30PM (#30541498) Homepage

    Except that it's basically unusable now.
    I don't know the details 100%, but the Ext2 IFS driver doesn't support Inodes that are larger than 128 bytes, according to the site you linked.

    My understanding is that Ext3 defaults to 256 byte Inodes now. The only way to change it is to manually create the file system with some flags, no gui option, certainly not while I'm installing Fedora or Ubuntu.

    I tried getting it to go but when I attempt to access the drives windows tells me it needs to be formatted before use. Not good!

    Perhaps it will eventually be fixed, but I will agree with the GP, using NTFS from Linux has been rock solid for at least a couple years for me.

  • by KonoWatakushi (910213) on Thursday December 24, 2009 @01:54AM (#30542068)

    The fundamental problem lies in USB bridge chips which do not properly implement the cache management commands. Others have replied that you need to disable the write cache, and while that would be a solution, it is often impossible. Even with bridge chips that do support the cache disable command, some hard drives will not honor it anyway.

    Most USB bridges simply lie about when data has been written, which makes it very difficult for a filesystem on top of it to make any guarantees. While it may not happen often, this can have disastrous results, as you have seen.

    The copy on write nature of ZFS left it especially vulnerable to broken USB storage, and could easily leave you with a corrupted pool requiring manual intervention and a bit of luck to recover. Thankfully, the recent bits address this, and ZFS is now the only filesystem that I would trust on top of USB storage. Most other filesystems will survive without incident, but at the cost of some silent data corruption.

  • by MirzaD (1357375) on Thursday December 24, 2009 @05:51AM (#30542756) Journal
    Create different partitions on external drive, one for example EXT3 and other FAT32. EXT3 would be large and FAT32 would be very small couple of megabytes. On smaller partition you would have EXT3 drivers for Windows machines. With this you would be able to access EXT3 from any machine. You could also try different file systems in same way.
  • by Anonymous Coward on Thursday December 24, 2009 @09:38AM (#30543400)

    In my opinion, external backup drives cannot be trusted. You could work around this by mirroring the drives yourself, but I think that I have a better solution.

    Use ZFS.

    Here's how I plan to backup Windows files onto ZFS external drives.

    1. Install Virtualbox to run OpenSolaris which will manage the ZFS drives.
    2. Get a distro of OpenSolaris that supports dedup. The first distro with all of that was 128a so google for this filename "osol-dev-128a-x86.iso" or download a more recent OpenSolaris.
    3. Get some USB hubs, one for each of your USB ports.
    4. Plug in a bunch of USB drives. I am using eight 8Gb thumbdrives but that only gets me 48Gb. Use spinning hard drives to get more storage.
    5. Decide on RAIDZ2 or RAID 1+0 (a stripe of mirrors). If you are using drives bigger than 500Gb then you probably should go for RAID 1+0 aka RAID 10. Be careful, some advice on the net gets this backwards. I am using RAIDZ2 with 2 parity drives which is why my eight drives only give me 6*8=48Gb of storage.
    6. Install OpenSolaris in Virtualbox. If you are an uberhacker like me, do a console install on a 2G CF card. If not, then choose a 6Gb virtual drive and do the standard GUI install.
    7 Set up your ZFS pool on the USB drives, make some filesystems and export them as CIFS shares or make some virtual devices and export them as iSCSI.
    8. On Windows, copy your files to the network drive.
    9. In the virtual OpenSolaris machine run a ZFS scrub.
    10. Shutdown OpenSolaris and pack away the drives until the next backup session.

    You now have a reasonably secure backup which can survive the loss of one, or maybe two drives depending on how you configured it. And if a sector on a drive goes bad, ZFS scrub will recover the data. Normally, an external drive failure means either total loss of the data, or at best, loss of some files. And an external drive may have already lost some of your files but you won't know it until you try to use them years from now. On other filesystems there is no equivalent of the ZFS scrub with sector-based checksums to automatically recover bad blocks.

    Yes, I know this solution is a bit techy, but expect someone to package this up as a neat backup-in-a-box solution. I believe that you can run Virtualbox on Windows in a headless mode so the customer would never know that their Fort-Knox grade backup application is actually OpenSolaris in a VM.

  • Samba and Unison (Score:2, Informative)

    by poolecl (170874) on Thursday December 24, 2009 @11:00AM (#30543926)

    I myself wanted a setup like this several months ago and settled on setting up 2 Ubuntu systems with internal 2TB hard drive. One at home and one at my remote location. I share the drive over my local network with Samba. (The drives themselves are formatted ext3 or ext4, but that hardly matters to the rest of the network once they are shared.) (To later support Mac OS X you could try getting afp working, but it will still happily mount samba also, which is what I do with my mac.) Which is fast enough to serve video, and means I don't have to lug a drive around the house. And I periodically run Unison to keep the systems in sync over SSH. Even pre-mirroring the drives, Unison takes a day or so to run the initial sync. But after that first run, the syncs are pretty quick. At the moment I am manually running the GUI Unison, but I plan on eventually switching to the command line version and automating it in a cron task.

  • Re:Ext3 (Score:2, Informative)

    by ppc_digger (961188) on Thursday December 24, 2009 @05:35PM (#30547096)
    It has at least two other issues: it doesn't support ext3 journaling (so ext3 is treated as ext2), and it can't fsck (and because it doesn't support ext3, it can't do a journal replay either), which means that it can't mount a dirty volume until it is mounted by Linux.
  • NAS is the way to go (Score:3, Informative)

    by seangee (1709612) on Tuesday December 29, 2009 @09:31AM (#30581350)
    + 1 for build a NAS. Your backup doesn't need to be very portable. I started out with FreeNAS, if you go this route ZFS would be the logical choice. I got pretty frustrated with this so rebuilt my NAS using Ubuntu and EXT3 - it made sense to me to use the native FS for the OS. I currently have 4 x 1TB internal drives in raid 5 (one spare). I use a USB drive to back this up. SMB internally and all the files are accessible by every OS I use. If I need to take files out and about there are good old USB sticks or portable drives. Mine are usually NTFS or FAT 32 because anything can read these - and of course there's always good old FTP on the NAS

The biggest mistake you can make is to believe that you are working for someone else.

Working...