Filesystems For Removable Disks?

An anonymous reader asks: "I have recently (as in today ) acquired a 250GB external HD with both USB2 and Firewire ports, with an eye to using it to carry around all my stuff (my humungous e-mail archive, ISO images of whetever distro I'm running, music and work files - I do a lot of database work, so I often need to move 40GB+ database dumps). The thing is, In order to make proper use of it I have to be able to mount and write to it on all three platforms I use: Windows (easy, it comes formatted as FAT32), Linux (trivial mount syntax) and Mac OS X (it just works as is, since it also supports FAT32). However, I'd like to get rid of FAT32." What filesystems, aside from the FAT varieties, have decent support across the major operating systems?

"The disk comes factory-formatted (Windows doesn't allow you to format a disk this big as a single FAT32 partition), and even though I'm not running against any FAT32 limitations yet, I was wondering if there was a better filesystem to use. NTFS would be perfect (given its rock-solid transaction support - always useful on an external drive), but the Linux versions are far from reliable for large file writes and Mac OS X lacks it. ext3 isn't supported on Windows or the Mac (as far as I know).

In short, my requirements are:

• The filesystem must be read/write for Windows, Linux and Mac
• The disk must have a single partition
• There must be tools available for all three OSs to format the HD with that filesystem in case something goes wrong and I'm away from home base

However, if I'm to be stuck with FAT32, I'd appreciate pointers to utilities for reformatting the HD with a single 250GB partition for Mac OS X and Windows (the built-in Disk Manager only lets me format 40GB partitions in FAT32, to force people to move to NTFS)."

hm

Fat32 works across all, so I cannot use it.

OK.

The question reworded

Hello, I currently have a portable storage system working just fine with FAT32 across three different platforms. However, this is much too easy to me and I'd like to bash Windows at the same time, so I'm asking for advice on how to make my life difficult and go with some obscure filesystem which won't have many third-party tools available to alter it if something goes wrong.

Have you ever heard "If it ain't broke, don't fix it?"

If not, can you at least tell me WHY you want to break a good thing? It works FINE for you in all 3 OSes! Is your question a troll? If so, damn fine work!

Re:The question reworded

Actually, he *can* reformat the drive as a single FAT32 partition, and use it on all OSes, since he's not running into a physical limitation of FAT32 but rather a deliberate design limitation. "The built-in Disk Manager" bit means he's running NT/XP and IIRC Microsoft has deliberately limited Disk Manager's FAT32 partitions in an effort to encourage people to move to the more advanced NTFS system. Despite this NT/XP is perfectly happy to access FAT32 partitions *much* bigger than 40GB, as long as they are created elsewhere.

I have successfully created FAT32 partitions in excess of 100GB and mounted them under XP using Partition Magic, Linux's *fdisk tools, and Windows 9x. We're talking a removable drive here, so it's not going to be too much hassle to partition the drive on another OS (it's the partitioning that's the problem, not formatting).

A simple process of elimination shows that FAT32 is the most portable filesystem that offers a realistic level of confidence that your OS wont trash the data. It may not be the most sophisticated system out there, but unfortunately that's the price you pay for portability at present. Plus it has the added benefit that it's accessible from a single DOS/Linux boot disk in emergencies - something that's save my ass on numerous occassions.

Windows is your limiting factor

With Windows, your choices are FAT16, FAT32 and NTFS. NTFS isn't amazingly portable, so you're pretty much stuck.

Re:Windows is your limiting factor

Linux supports ext*, fat*, iso9660 xfs, jfs, reiserfs, efs, isofs, ufs, udf (experimental), minix, VxFS, HPFS, HFS, HFS+ (limited?), QNX, ntfs (limited), BFS, Amiga FFS, ADFS, BeFS, and finally System V/Xenix/V7/Coherent FS (that is a long one!).
MacOS 10.x supports HFS, HFS+, UDF, ISO9660, AFS, and FAT*
Windows XP supports ISO9660, NTFS, FAT*, and UDF.

I believe that MacOS and Windows both require 3rd party software to use UDF.. but I could be wrong about that.

The solutions would be FAT*, ISO9660, and UDF. ISO9660 is read-only and I've never heard of someone using UDF on a harddrive (it is for those 'direct cds' you might have seen). FAT* sucks, but it works everywhere. It might be worth the effort to see if UDF could be used at all, but a small FAT32 partition would have to be made to accomodate the utilities for using it on the target system.

Before everyone flames the story submiter for being bias against Microsoft, the issue is that FAT really does suck and it would be great if there was something else that everyone supported.

Personally, I'd like to have a 6 gigabyte external (usb/firewire) harddrive that I could boot MacOS9 from AND share it between Linux and Windows computer. I guess I'll keep dreaming for a while :)

NTFS

Supported on variety of operating systems, including Windows 2000 Professional, Windows 2000 Server, Windows 2000 Advanced Server, Windows 2000 Datacenter, Windows 2003 Server, Windows XP Home, Windows XP Professional, Windows 2003 Server Web Editition and many, many others.

Give it a rest!

To all of you saying "FAT works fine", consider that the OP may be asking the question because he wants a filesystem that's faster, more reliable or perhaps more secure (encrypted?) than FAT. Just because FAT works doesn't mean it works the best. He's asking if there are any alternatives, besides what he's already using.

Re:Give it a rest!

Right, exactly. This isn't a 'switch', its a 'compare'.

I myself am very interested in the answer. I wonder if the solution may be to have an ext3 (or xfs, or jffs) shim for Win32, also?

Man, if only there were an *open*, *journaled*, *fast* and *efficient* filesystem which all 3 OS's were allowed to play well with.

Seems to me if the answer to this "Ask Slashdot" ends up being "just use FAT32", then there's an opportunity for a decent OSS project: completely open, cross-platform, fast, journaled filesystem, with code tarballs for all major platforms.

Hmmm...

fat32 is your best bet.

Like it or not, fat32 is the only option that works on all the OSs in question and doesn't cost extra money.

Your other options include Paragon's Mount Everything [mount-everything.com] and their Ext2fs Everywhere [ext2fs-anywhere.com] (which is really just a subset of `Mount Everything'.) These programs let you mount ext2/ext3 under Windows, or let you mount NTFS under Linux (I don't know how good that is -- I know that Linux has some NTFS support itself, but know it's not very mature.)

If that's not clear enough -- if you want to spend some money, spend it with Paragon and you can use ext3 or NTFS. If not, stick with fat32.

Why not XFS?

If what you want to do with the *data* on this external drive is copy to/from the other three platforms, then maybe having a Knoppix CD (or homegrown) that can mount your external drive (XFS), plus can mount any of the other filesystem types needed, and can transfer the data. At this time, AFAIK, only the NTFS writing is not considered safe. But it may be soon.

In other words, look at the problem from a tools perspective, not a filesystem perspective.

ext2 is a valid option

as well as using ext3 under linux.

See here for general info:
http://www.ibiblio.org/mdw/HOWTO/Filesystem s-HOWTO -6.html
And here for windows tools, but read the link... First.
http://www.it.fht-esslingen.de/~zimmerma/s oftware/ ltools.htm
And finally for OSX:
http://www.versiontracker.com/dyn/moreinfo/m acosx/ 18619

Play nice and have fun!

I think you're stuck...

...with FAT32 for now; I know I am.

I too have an external (FW + USB2) drive that I like to keep quick backups on from my WinXP desktop and my MacOS X laptop. Unfortunately, while FAT32 works across platforms, it doesn't support any of the permissions, etc. that are native to each platform. One more thing to consider is that FAT32 doesn't support large files (>4GB per file).

What I've done so far is to use FAT32, but then use "disk images" on the MacOS X side to emulate the HFS+ file system (to keep permissions, etc.)

There is hope, though, as I read somwhere (can't remember where) that MacOS X has NTFS support planned in Jaguar. While, I don't think that will address the permissions issue across platforms, at least I can start making 4+ GB disk image files in MacOS X.

stuck

you are indeed stuck with fat32 as others have stated. It is the only filesystem that works on all 3 platforms. If you want to format the whole drive as one partition here's a hint. FDISK. I'm tired of all these people using partition magic or other pussified partitioning programs. Get a win98 boot disk or a linux disc and use fdisk. *gasp*!!!

Try this

I bet there is a way to trick it into being formatted like a giant CD-RW or DVD-RW or something.

I suggest looking into that, as all oses should be able to read one of the standard CD formats...

Why are you asking this?

However, I'd like to get rid of FAT32.

Why? It's the only real choice you have.

That's easy

...what filesystems, besides FAT...?

UDF and iso9660, duh! That's all that's left.

repartition or you're stuck

windows sucks. FAT32 can't have files larger than 2gb and its allocation unit is *FUCKING HUGE*. You can experiment with ext2 access programs and drivers for windows, etc. but they're not quite up to snuff yet. If you do that; create a small FAT partition containing the drivers for various OSes, and make the remaining partition ext2.

(why ext2 and not ext3? many tools for other OSes can't handle ext3)

Windows support for ext[23]

I've used a tool on windows to read ext[23] partitions before. You can find it by doing a search for win32, linux, etc. on google. This was quite a while ago and it could read ext2 just fine. At the time there was a beta or talk of such that supported writing to ext2 from windows. Maybe by now it might be worth checking out. I think this was the program:

http://uranus.it.swin.edu.au/~jn/linux/explore2f s. htm

Ext2 or fat32

Ext2 may work- see my sig for windows ext2 driver (somewhat mature), and ext2fsx [sf.net] for a OS X driver. Since the OS X ext2 driver seems somewhat unstable, I would guess that fat32 is your best bet. Since I've never had any reason to work with a partition size bigger than 20GB before, I don't know what would allow you to format the disk that way (unless a mkvfatfs or suchlike under linux does the job).

I really wish people would give more support to those who are trying to develop cross-platform filesystems. If you're interested in being able to share data and swap partitions between windows and linux, please consider helping the projects in my sig. (No, I am not affiliated with either of them, I just happen to use them.)

Ext2/3 on Windows

I can't speak for OS X, but there is a piece of software available for Windows that will let you access an ext2 or ext3 drive - Paragon's Ext2FS Anywhere (http://www.ext2fs.com/). I've heard it works quite well, but I've never seen it tried with a removeable drive. Be careful to shut down cleanly, though, or you risk screwing the filesystem up big. It says it has just added support for files of >4GB, although I thought the ext2 limit was 2GB. Something to look into.

Formatting the partition

Try the command line format tool (intuitively called format.exe). I don't believe it has the "migration encouragement" limitations of the GUI tool, but I don't have a spare >40G partition to check for you. Note that if the problem is *creating* a large FAT32 partition (rather than just formatting it), you may need to first *create* it as an NTFS (or unformatted) partition and then re-format it.

Other than that, any of the other platforms available should be able to format the partition if you so desire.

And why do you want a single monolithic partition?

I have an 80GB file server and have 4 partitions on it. Hey partitions get fragmented, become bad, get crashed etc. You dont want all your data to be at risk there. Say somehow a large block gets deleted or bad sectors, the worst that can happen is the partition falls with it. And I'm sure your data is classified like emails, mp3s, games so you could divide them among the partitions easy.

Still need a more elegent solution than FAT32, put zipslack in a smallish FAT32 and boot into Linux there (or knoppix linux) and you have access to the remaining big partition formatted as XFS.

Yes I too wish someone gracious would make win32 drivers for ext2. That will help the Linux community far more than Bill Goates.

How about 2 out of 3? ;)

Okay -- this one is purely for the "FWIW" file...

You could run NT's other original filesystem, HPFS. Linux has decent HPFS support available, the allocation unit is a 512b sector, and it's organized for fast searches and minimal fragmentation. It can also format up to 64GB partitions (although it still has a 2GB filesize limit).

The trick is to get HPFS support for Windows. To do this, you need to get the driver files from Windows NT v3.x (something that, admittedly I'm not sure works with Windows versions > NT 4. I don't do Windows personally, so I haven't tested it -- like I said above, FWIW). That will give you two of the three OSs supported. HPFS has been around for a while, (circa 1988), so you might be able to find something from the Intel FreeBSD world you could port to OS X.

I use HPFS for my 100MB ZIP disks (which I admittedly rarely use anymore for anything than quick archival purposes). It's not journalled, but it uses a bidirectional sector pointer system, so chaining errors are amost always fixable. The big downside is that if the filesystem is dirty, checking it can take a huge amount of time.

It's probably not a practical solution (I didn't and won't claim it is), but it's still a slightly more constructive answer than "Your stuck with FAT32" :).

Yaz.

FAT32 is still your best option at this time

Microsoft supports FAT12, FAT16, FAT32, and NTFS. NTFS is in many
ways superior to FAT, but Linux support for NTFS is still limited
to read-only unless you want to use highly experimental write support
that is officially labelled DANGEROUS (i.e., more bleeding edge than
code that is merely EXPERIMENTAL) and guaranteed to corrupt the
filesystem. Then there's the issue of NTFS support in OS X; I'm
not aware of any options there. Also, if you need to access it
from Win9x/Me, those versions don't support NTFS either.

The ideal of course would be a journaling filesystem like Reiser.
But Windows doesn't support those, last I knew.

Now, if you're willing to forego native support by some OSes, there
are third-party apps to bring support to Windows and/or Mac for
various filesystems that aren't supported natively. Explore2fs and
the like. These are a kludge and a pain to use, and they don't
integrate with the OS much less with apps, but they do (mostly)
work. But when you go down that path you lose most of the nicer
advantages of those filesystems. The real advantage of these apps
is for multiboot systems, so that if you happen to be booted into
the other OS and somebody on usenet asks how you got foo to work,
you can pull your config file off the other filesystem and look.
It's not really practical to use this as a regular filesystem.

FAT32 may suck, but it's supported by everything. Go with it.
If you need to store things FAT32 can't store directly (symlinks,
permissions, other attributes...), wrap them up in a ZIP archive.

What if it wasn't just a disk?

Could you put together a custom hardware solution? A disk plus a small embedded Linux SBC, run whatever filesystem you want, put a fast ethernet connection on it and export the drive as a samba share?

I guess this would be a lot slower than USB or Firewire, even with gigabit ethernet.

ext2

You could use the ext2 filesystem. Mac OS X Ext2 Filesystem [sourceforge.net] is a beta and Explore2efs [swin.edu.au] is available for Windows.

What's the big deal

I have a 80 gig hd in an external firewire encloser. I partitioned it using partition magic (one of the greatest tools created) in FAT32 specifically so win xp and linux could read and write to it easily and so that I could plug it into pretty much any other computer and get it working with little pain. I use it for storing gigs upon gigs on mp3's and divx files and linux rpms.

My question is why is this really so bad? For my purposes and the original poster's purposes what real world benefits would either one of us see if we use a more modern and efficient file system. I'm sure there are lots of reasons dealing with speed and file permissions. I'm the only one using it so I don't care about permissions. But really? Is any other file system that requires tweaking on each OS going to be worth it when playing a movie or song? And how many people have have files over 4 gb on their home machine anyway? Maybe these are considerations for the database, I don't have much experience in that so I may be wrong.

I'm not trying to seem that I know the answer to this and I'm trying to give this guy an answer. I'm really just asking why is FAT32 so bad for these specific purposes?

Two partitions

Well, since its possible to get read-only access to NTFS partitions from Linux, and to get read-only access to ext partitions from Windows, the solution is simple:

Create two equal partitions, one of NTFS, and one of ext*. Depending on the OS being used, one partition will be readonly and one will be read-write. Therefore, you always have the ability to read and write data.

Just because..

Just because it's USB or Firewire, doesn't make it portable.

NTFS

I would recommend NTFS. It actually works better than FAT (A copy of Windows 2000 on my system works faster on NTFS than on FAT) and is interoperable between Windows NT 4, 2000, and XP. NTFS also offers better file management (Less fragmentation) added security (NTFS5 is required to have File Encryption), and Limitless files sizes.

The Linux NTFS Project is at Source-Forge.

http://linux-ntfs.sourceforge.net/

Re:Samba to the rescue!?

"I often need to move 40GB+ database dumps"

Does that answer your question?

Dude, you're stuck in a rut

Yeah, Windows sucks. I think I may have heard this mentioned once or twice before. What I don't get is how that's relevent to the discussion. We're trying to solve a technical problem here, not fantasize about some perfect world where the problem doesn't exist.

I'm unfamiliar with the "redirector"

it is no more difficult to support a non-Microsoft filesystem in Windows

The www.microsoft.com web site is not responding as I type this, but it lists the price of the IFS Kit (the headers and libraries required to compile NT filesystems) as $1000. Are the "redirector services" any different? And even then, is an additional FAT## partition the best way to get the particular redirector driver onto a particular machine?