Journaling Filesystems and Network Mirroring? 22
CustomDesigned asks: "We are looking at the feasibility of mirroring all changes to selected filesystems to a hot backup over an internet or WAN link.
This would provide a degree of protection for a business in the wake
of a disaster like Sep 11. It seems that journaling filesystems have
much of the work already done. All that is needed is a hook to copy
log data into a message queueing system for delivery to the hot
backup, and then running fsck for the unmounted file system to apply
each log update. (This is more complex for ext3 where the log data
is kept internally within the filesystem.) One problem is that JFS
(and I presume ext3) only journal filesystem metadata. Has anyone
seen a fully journaling filesystem? Is there any other work on remote
hot backups for Linux? The toolset for any such capability should
include a way to measure bandwidth required for a given filesystem
without actually doing it. This would allow intelligent
administrative decisions to balance bandwidth costs against
traditional removable media backups."
I found something about this. (Score:1, Informative)
Re:I found something about this. (Score:2, Funny)
While the current story could very quickly be resolved by a Google search, at least it's an interesting topic. I really got pissed when Cliff posted the one a couple days ago, something like, "Ask Slashdot: Don't you just HATE it when people don't QUOTE properly in E-MAIL?" When I saw that, I thought they should change the tag line to "News for Nerds, Stuff that Matters, and Geeks' Petty Annoyances."'
DRBD (Score:3, Informative)
Take a look at Philipp Reisner's DRBD [tuwien.ac.at]. It does more or less what you're after, although it doesn't do it using journalling filesystems.
You might best describe it as a sort of network RAID 1
Re:DRBD (Score:1)
Um ... ext3? (Score:3, Insightful)
Check out InterMezzo (Score:2)
Consider the political ramifications.. (Score:5, Insightful)
If politics and such are important in your company, Look into "real" technologies such as EMC (Sym or Clariion), Hitachi, or IBM storage, even if you have no real intention of buying. Make a brief outline as to why these systems would not be cost-effective, or fit your needs, just so your investors will know you put some thought into it.
I'm not trying to troll, i'm just offering information you may want to consider. Again, YMMV
Just my unsolicited $0.02
Re:Consider the political ramifications.. (Score:2, Interesting)
BUY EMC, and yes it does work with Linux, and you can install any of the supported Linux File systems on it.
Re:Consider the political ramifications.. (Score:1)
My company uses EMC, Legato Notworker (ugh!), and Veritas Cluster Server/First Watch/File System for our needs. Linux isn't allowed in our organization by our investors, and everyone down the line follows that philosophy.
[9fans] Venti and the new plan 9 file system (Score:1)
From: Sean Quinlan To: 9fans Mailing list
For those of you interested in the direction we are heading
with respect to plan 9's file system, you might want to
checkout our paper on Venti that will appear in the
USENIX fast conference.
http://www.cs.bell-labs.com/~seanq/pub.html#venti
Venti is a block level storage server that replaces the optical
juke box for a plan 9 file system. Some of the benefits include:
coalescing of duplicate blocks
compression
no block fragmentation
Also, we have switched from optical to magnetic disks as the storage
technology. I know many of you already use magnetic disks to
"fake" a worm, but for those of us using a optical juke box,
the performance improvement is rather substantial!!
seanq
Miralink Principia (Score:1)
This is a device which emulates a standard SCSI hard drive. Plug it into your host adapter, and any data written to it is automatically mirrored to a remote location over the Internet.
Here's some useful information... (Score:2, Informative)
Realtime data mirroring under Linux [linuxfocus.org]
And some other resources...
The Advanced Maryland Automatic Network Disk Archiver (AMANDA) [amanda.org]
Creating Filesystem Backups with 'rsync' [oreillynet.com]
Linux Backup [linux-backup.net]
Veritas is your friend (Score:2)
Of course you have to pay for them. But you get what you pay for.
--NBVB
p.s. (VxFS rocks. Saved my bacon more times than I can count!)
Network Raid? (Score:1)
Wrong question (Score:3, Informative)
If you need to maintain a consistent filesystem image across multiple sites, you don't need a journaling filesystem. You need a distributed filesystem.
There's a huge difference. Journaling filesystems write a bit more information to the local disk before they report 'success' to the caller. Distributed filesystems write data to systems at multiple sites before they report 'success.'
This means that one of your key issues is network performance. Locally distribtued filesystems (over a 100 Mbps LAN won't show much of a performance hit, but you're going to notice it if you are writing to a distributed FS with nodes in multiple cities. For a lot of applications the latency is not an issue (e.g., if you're selling commodities and need a consistent time sequence everywhere.), but in other applications the latency will be unacceptable.
Another key issue is how you handle network partitioning. You need to be able to continue functioning when the network is down, or individual nodes are down, but that means you need to handle resyncing the systems.
The good news is that this is possible, but the bad news is that there's not a lot of good free software yet. CODA is probably your best bet, but I've heard some reports that it has some serious shortcomings. I think some of those problems are because the authors misunderstood what CODA was designed to do, but not all.
If you're willing to lose a day's worth of data you would be better off making nightly backups and fedexing them to remote locations. But be sure this is acceptable - there are many applications where any data loss is unacceptable.
Re:Wrong question (Score:1)
err, journaling has nothing to do with it... (Score:2)
- A.P.