Recoverable File Archiving with Free Software?

Viqsi asks: "Back in my Win32 days, I was a very frequent user of RAR archives. I've had them get hit by partial hardware failures and still be recoverable, so I've always liked them, but they're completely non-Free, and the mini-RMS in my brain tells me this could be a problem for long-term archival. The closest free equivalent I can find is .tar.bz2, and while bzip2 has some recovery ability, tar is (as far as I have ever been able to tell) incapable of recovering anything past the damaged point, which is unacceptable for my purposes. I've recently had to pick up a copy of RAR for Linux to dig into one of those old archives, so this question's come back up for me again, and I still haven't found anything. Does anyone know of a file archive type that can recover from this kind of damage?"

Are you sure tar is unacceptable?

[nado]

Are you sure it's unacceptable that tar archives are breakable? The way I see it, you'll tar your files then bzip them and finally put them on a backup server/CD/DVD. The bzip layer will provide the auto-repairing features, I don't see how it could break between having the tar and bzipping it. Is this for a normal environment? If your harddrive breaks during or after creating the tar, then the bzip would fail, no? Please tell us more about your situation if not.

Yeah

[photon317]

The format you're looking for is any format you like stored on reliable storage.

Why bother with all the intricacies of a pseudo-fault-tolerant data structure? Ultimately the best archive format for recovery will be one that just duplicates the whole archive twice over, doubling space requirements and improving immunity to lost sectors on drives. At which point one asks, "Why don't I just stick to simple files and archives, and use reliable storage that handles this crap for me, for all my data, automagically?" Storage of any sort just keeps getting cheaper and bigger. If you have any interest in the longevity of your data these days, there's almost no excuse for not using the data-mirroring built into virtually every OS these days and doubling your storage cost and read performance while preventing yourself from worrying about drive failure.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:Yeah

[Viqsi]

Why bother with all the intricacies of a pseudo-fault-tolerant data structure?

I'm on a laptop. I like my laptop. It's a very nice laptop. However, it doesn't exactly support those kind of hardware upgrades, and I am still ultimately on a bit of a budget.

I kind of put forth the question not only out of the hope that a Magical Solution To All My Archival Problems would Mystically Appear (puff of smoke optional but appreciated) but because I want to find something I also feel like I can unreservedly reccomend to non-ideological friends who are looking for, say, something slightly more reliable than ZIP files. I could've mentioned that in the article post, but it was already getting long. :)

Re:Yeah

[QuantumG]

Ultimately the best archive format for recovery will be one that just duplicates the whole archive twice over, doubling space requirements and improving immunity to lost sectors on drives.

Obviously you know nothing about error correction, so STFU.

Re:RAR isn't completely non-free

[Kris_J]

But if you purchase it, as I have, you get a product you can use from now until forever, so long as your OS supports it, plus you can get the decompression source so that you (or someone else) can always write a decompressor for a future platform. Surely you don't need to worry about replacing it until both the following are true: None of the versions you've purchased run on your current platform AND no version compatible with your current platform is available (at a reasonable price). At that point you stop creating RAR archives and simply keep the decompressor around (porting and recompiling as necessary).

(Personally, I don't care about recovery records, I just keep two copies of everything, and I moved to 7-zip -- which can decompress RAR -- about six months ago.)

tarfix

[morelife]

tarfix

may help some of those archive issues.

But, the archive format is not going to save you. Use multiple media. You need more than one physical archive for better safety, regardless of format. Hell, you'll probably die before some of today's media fails.

Re:RAR Archives

[jonadab]

> were there any wide-spread, legitimate uses of .RAR?

RAR was heavily used in Germany, among the gamer community. A lot of Descent
players for example distributed their custom levels, missions, textures,
hogfile utilities, savegame editors, and whatnot in RAR format. It was
annoying; I had to go hunt down and download a RAR extractor just to install
some of the stuff.

The usual argument was that RAR was "better" than ZIP either because of the
compression rates or because of the partial recoverability or whatever. My
opinion on the matter has always been that for distributing stuff over the
internet, the most ubiquitous format is automatically the best, so ZIP is
better than RAR irrespective of technical issues, due to compatility concerns.
By the same reasoning, gzip is automatically better than bzip2, and no amount
of technical superiority makes a good enough reason to use bzip2 over gzip.
Frankly, for anything that's not inherently *nix-specific, ZIP is better than
gzip for the same reason. Not everyone agrees with me about this, obviously.

Re:Yeah

[sasami]

Par archives is just a scam popularized by cluless usnet abusers. Think about it, if those files really could reconstruct a corrupt rar archive, why not post only the smaller par files ... Get youself double copies and you'll be far better off

Ignore this post. It's either a troll or an idiot.

PAR files substitute for missing pieces. They don't regenerate the whole file by themselves. Go look up how RAID 5 parity [somacon.com] works. They're not called PAR files for nothing [utk.edu].

Just because you don't understand how something works has no bearing on the fact that it does work. Except in certain performance-sensitive cases, doubling up is the least intelligent way of adding redundancy.

---
Dum de dum.