Examples of Obsolete File Formats?

reedk writes "I was having a discussion with my boss about long-term archives, and we got on the topic of older files becoming un-readable by newer versions of software. Not only are those old Ami pro files unreadable by today's common word processors, but I have heard that newer version of Office can't consistently open very old versions of Office documents. With the increasing retention periods being forced by current and coming regulations, this could become a problem of compliance in the future. We want to pursue this topic, but to build support for it internally, I am looking for examples of older file formats that are no longer readable by newer version of the same software or due to the market death of the product. If true, this would lend a lot of force behind moving to products that have an open file format. Can Slashdot readers come up with examples of this, or ways they have had to get around these kinds of problems?"

Open up the standards

[i.r.id10t]

Well, if it is an open format, nothing is stopping someone from writing something to read it and convert it to something "modern". If it is a closed format, and no longer in use, then the owner really should open it up. Would it be possible to setup an escrow of (closed) file formats - automatic open if the company goes defunct or individual dies.

Also, if you know what the end result data is supposed to look like, would it be possible to start "decompiling" it? Works with binary executables (sometimes)...

EBCDIC and dead voters

[markjugg]

I once worked on a research project for a newspaper to investigate voter fraud.

To start, they used open records requests to get the details of people who recently voted, and details of those who recently died.

The goal was to find people who continued to vote after they died, which may sound funny, but is still happening [citypaper.com].

The data the government data gave us was on magnetic reels. The data on the reels was stored in a fixed-width EBCDIC format [dynamoo.com]. Talk about a dead format!

It turned out the local college still had a working magnetic reel reader, and was able to help me get the data out of EBCDIC into ASCII, but the project was cancelled anyway.

how about codecs?

[artifex2004]

While the file format itself may be a standard wrapper, there's many codecs out there that are obsolete and that only ever had proprietary drivers written for early MS Windows versions, for example.

Some thoughts about it

[Alpha27]

It first depends on what you want to achieve with them, do you want them to be read only, or do you wish to edit them as well in the future? They may not be too much of an issue but something to think about.

For images, I would look at the past to see what file formats were around before the internet was mainstream, circa 1995. I remember Paintbrush PCX as a file format, but haven't since a file in that format since then. TGAs and TIFFs were around and still are today, that might be one possibility. You also have SVG formats, and that being an XML file format, allows you to convert it to another format in the future.

As for text documents, one definite possibility is XML. You can convert to many other formats from XML (HTML, PDF, RTF, etc.) Another possibility is RTF and plain text, though you might lose some of the more advance features. You might even have to extend the XML to deal with anything special in your files. Latex or Tex might be another solution since it's still around, though I have no experience with it, beyond being awware of them.

I would also recommend keeping a copy of the original software you used at the time, in case you need to get access to the files with a program that actually created. This way, you still have some sort of access. If that means you need to keep a copy of the original O/S as well, so be it.

I happen to have a computer museum at my disposal

[gdav]

But even so, the other day I got a shock, seeing how quickly the door closes.

A professor at the university where I work turned up with his original doctoral thesis from 1989 on disk. 3" disk, to be exact - the format that famously lost out to the ubiquitous 3.5" disk. He had written it on the Amstrad PCW 8256, a weird British CP/M machine from the mid 80s. No matter, I have several of these rotting in my loft!

But they don't boot. At this point you brace yourself for the long haul. The drive belts used to perish on those models, but look! There are loads of drive belts in the Maplin Electronics catalogue. You just need to order the right size.

No problem! You carefully dismantle the drive and dig out the belt. You broke it? No problem! Just makes it easier to measure. You can only measure the circumference, whereas Maplin only quotes the diameter? No problem! You are about to use Pi for the first and last time in your entire life! Order one that's slightly too big, and one that's slightly too small, just to feel safe.

When the belt arrives, you fit it. You carefully re-assemble the drive. You insert that CP/M boot disk that you carefully prepared in 1987, the one with the custom PROFILE.SUB that copies important utilities to RAMDISK. You power up and it boots! You feel young again.

Now your try your Locoscript boot disk - remember, Locoscript did not run under CP/M - it was an entire little operating system unto itself. It works, and when you swap disks (f7) you can read the Prof's work! It's yesterday once more! Shoo-bee-doo-lang-lang!

At this point I got lucky - I had the LOCOLINK package including the special Amstrad Bus PC parallel port link cable, so I was able to go Locosript PCW -> Locoscript PC -> Wordstar 3.3 -> Wordperfect 5.1 -> Winword. Those nice chaps at Ansible could have shortened that trip by a step or two.

In the absence of the proprietary LOCOLINK cable I could also have gone Locoscript 1 PCW -> Locoscript 2 PCW -> ASCII on PCW -> ASCII on PC via Kermit -> Winword. But I'd have lost all his bolds and underlines.

Now I got a fine bottle of Metaxa Greek Brandy out of this exchange, so I'm not exactly complaining. But I was shocked to realise that his files were younger than my eldest child, and she's got two years of school ahead of her.

In the absence of any credible international initiative to create a reliable permanent archive format, I'd say print it to acid-free paper, multiple copies in separate places, and hope for the best, like Cassiodorus.

reStructured Plaintext

[FFFish]

I contract out as a technical writer. For my primary client, I strongly encouraged and then delivered a plaintext solution that uses plaintext files stored repositoried in CVS, using the reStructured Text markup conventions processed through Docutils; and an XSL:FO template that is used by XEP to render the DocutilsXML to PDF. An autobuild system updates our documentation on a nightly basis.

This system has worked superlatively. In addition to creating a documentation solution that will forevermore be accessible without special software, our authors can focus entirely on content without concern for layout and visual appearance, our customers get a reasonably open file format (PDF) that looks as good on-screen as it does in print. It's win-win all around, by my reckoning.

Realplayer (sort of)

[Curmudgeonlyoldbloke]

No, don't laugh.

Realplayer 10 doesn't support Realplayer 2 "out of the box". It will happily connect to Real to download said codec if you want - although obviously this assumes that Real will always be with us.

Re:Open up the standards

[Anonymous Coward]

The question and responses like to blame Microsoft, but certainly early versions of Word had a well-documented, open file format. I had the SDK complete with sample code. Never-the-less, old Word files are here used as an example of data that can't be retrieved.

Even if the format is open, that's not really an assurance that you can really write a file converter to properly retrieve the data, especially for significantly complex file formats.

An open source project will fare no better after everyone moves on to the next trendy replacement and their home page on sourceforge disappears. The question really has nothing to do with open source or not; just complexity and the accuracy of documentation.

There's "open" and then there's *open*!

[fm6]

When people say "open format", they usually refer to documenting the details of the format. (Or, as with XML, using a format that's self-documenting.) Now, that does save a lot of work, but it doesn't address a much harder problem. Namely: OK, you've got the data, now how do you use it?

Classic example: sharing MS Word files with other word processors. The problem isn't getting at the data in .DOC format (not an easy problem, but one that was solved years ago). The problem is rendering Word formatting using the conventions of other word processors. As anybody who's tried to import complex Word documents into Open Office will testify, that's a problem that's a long way from being solved -- if it ever is.

I've been working on a project for an organization that has a bunch of certificates created in Adobe Illustrator 6. The files are saved in EPS format, which belongs to Adobe, but is very well documented. So accessing the files should be a snap, right? Wrong. I have Adobe Illustrator 11 (better known as Illustrator CS), which uses completely different conventions for creating an EPS file. It can read the old files OK -- but it horribly mungs the formatting. Somebody's going to have to sit down and undo all that munging, which will be a day or two of work. Then we can make the simple change (inserting a new signature), that's the only change we want to make!

So true openness has more to it than knowing what all the bits and bytes do. It's making sure that all the different design teams for different products that use the format (or the same product at different times!) are on the same page when it comes to the fine details.

Re:EBCDIC and dead voters

[saintp]

Do what the GP did: ask your local university. We still have a nine-track drive around, although it hasn't been fired up in a few years. Lots of data from the state government, ACT test reports, etc., came on 9-track tapes until just five or six years ago, so lots of universities still have them around.

Shrink Wrapped Apple IIe

[sysadmn]

I work in an aerospace division of a very large corporation. I was talking to a design engineer about the FAA's data retention requirements - he said in most cases, it's the life of the product, plus a little cushion. For us, that's about 40 years. In addition to preserving data, you have to be able to recreate the analysis - so if you used a visicalc spreadsheet to perform an analysis, you have be able to do it again. (I think this is more an "in case we get sued" requirement than an FAA one). I was joking about 40 years being a long time when the coworker said, "Just be glad you don't work for the medical division. They have to keep their design data for the lifespan of the patient. For a neonatal ultrasound product, that's effectively one hundred years!"