Filesystems For Removable Disks? 138
An anonymous reader asks: "I have recently (as in today ) acquired a 250GB external HD with both USB2 and Firewire ports, with an eye to using it to carry around all my stuff (my humungous e-mail archive, ISO images of whetever distro I'm running, music and work files - I do a lot of database work, so I often need to move 40GB+ database dumps). The thing is, In order to make proper use of it I have to be able to mount and write to it on all three platforms I use: Windows (easy, it comes formatted as FAT32), Linux (trivial mount syntax) and Mac OS X (it just works as is, since it also supports FAT32). However, I'd like to get rid of FAT32." What filesystems, aside from the FAT varieties, have decent support across the major operating systems?
"The disk comes factory-formatted (Windows doesn't allow you to format a disk this big as a single FAT32 partition), and even though I'm not running against any FAT32 limitations yet, I was wondering if there was a better filesystem to use. NTFS would be perfect (given its rock-solid transaction support - always useful on an external drive), but the Linux versions are far from reliable for large file writes and Mac OS X lacks it. ext3 isn't supported on Windows or the Mac (as far as I know).
In short, my requirements are:
- The filesystem must be read/write for Windows, Linux and Mac
- The disk must have a single partition
- There must be tools available for all three OSs to format the HD with that filesystem in case something goes wrong and I'm away from home base
hm (Score:4, Funny)
OK.
Re:hm (Score:3, Informative)
not to mention other things (although for the purpose carry crap from one place to another it's probably good enough), speed, stability, security, etc. Although NTFS sucks as well, at least it's better than fat32...
Unphat FAT (Score:2)
I'm curious to know why you think NTFS sucks. Because Mister Bill owns it? Not that it really matters -- you can't access it from any non-NT OS.
Re:Unphat FAT (Score:2)
Re:Unphat FAT (Score:2)
Fragmenting (Score:2)
I've experience with two JFSs: NTFS (because I use a lot of Windows boxes, and I avoid FAT where possible) and XFS (because I used to work at SGI). Both were originally released without any defragmentation utility, because they were thought not to fragment. Both Microsoft an
Re:Fragmenting (Score:2)
That's interesting - I wonder why they thought that. I haven't actually implemented a journaling filesystem myself, but, in theory, it's just a two-step write with some flags, so the fragmentation, or layout optimization, would seem to rely more upon the underlying algorithms to place data rather than the journaling. I can't think of a good reason why a journaled fil
Why do JFSs Fragment? (Score:2)
Re:hm (Score:2)
I'd mod you up if you were right. I want to mod you down since you're ill informed but I'd rather make a correction.
Your statement could be taken three ways, first, if you are implying the allocation capabilities of FAT32 is a disk size of 2GB:
FAT 16 has a 4GB limit.
FAT 32 has a 127GB limit.
Personally, I thought FAT16 had a 2GB allocation limit, and dated sources from Microsoft concur
The FAT32 Limits: (Score:1, Informative)
Here's what I think the original poster was trying to get across, based on skimming the MS KB [microsoft.com]:
So, given that (and the obvious lack of a journaled FS across the three OSes), I guess instead of ranting on about Windows being the
The question reworded (Score:5, Funny)
Have you ever heard "If it ain't broke, don't fix it?"
If not, can you at least tell me WHY you want to break a good thing? It works FINE for you in all 3 OSes! Is your question a troll? If so, damn fine work!
Re:The question reworded (Score:2)
He can't reformat the drive as a single partition. Perhaps you missed the parts where he said "the built-in Disk Manager only lets me format 40GB partitions in FAT32" and "I do a lot of database work, so I often need to move 40GB+ database dumps". That's a serious usability problem.
Also, and I know this is hard to believe, there are very good reasons for not wanting to use FAT, especially in a professional situation where something like, say, not
So don't do the formating on Windows. Duh. (Score:2, Insightful)
He can't reformat the drive as a single partition. Perhaps you missed the parts where he said "the built-in Disk Manager only lets me format 40GB partitions in FAT32" and "I do a lot of database work, so I often need to move 40GB+ database dumps". That's a serious usability problem.
He's obviously using two other OSes without that artificial 40GB limit. He could just use one of them when he needs to format.
Re:The question reworded (Score:4, Interesting)
I have successfully created FAT32 partitions in excess of 100GB and mounted them under XP using Partition Magic, Linux's *fdisk tools, and Windows 9x. We're talking a removable drive here, so it's not going to be too much hassle to partition the drive on another OS (it's the partitioning that's the problem, not formatting).
A simple process of elimination shows that FAT32 is the most portable filesystem that offers a realistic level of confidence that your OS wont trash the data. It may not be the most sophisticated system out there, but unfortunately that's the price you pay for portability at present. Plus it has the added benefit that it's accessible from a single DOS/Linux boot disk in emergencies - something that's save my ass on numerous occassions.
Re:The question reworded (Score:2)
I won't argue with the portability of FAT, but in my experience it can be painfully unreliable in certain applications. Having had that experience, I consider looking for alternatives to be a very wise move. I know that I would be very uncomfortable trusting my employers Very Important Data to FAT.
Re:The question reworded (Score:2)
No arguments from me there. FAT32 has serious limitations, especially with the ever increasing security concerns - no FS level encryption option, no journal, file size issues, people are already hitting the partition limits (although in the case an artificial one). I would *much* rather have something like EXT3 or even NTFS which is actually quite a technically advanced file system in design as a standard too.
The problem though is portability
Re:The question reworded (Score:1)
anyway, I'm sure Partition Magic is capable of creating FAT32 drives of at least as large as FAT32's practical limit of 2TB [ (2^32)*512bytes/cluster ], possibly even the technical limit of 140TB [ (2^32)*32768bytes/cluster ] though I'm not sure about that
Re:The question reworded (Score:2)
My nice backup solution (tar --create
(ok so for anyone else in a similar boat, 'man split' should help)
Re:The question reworded (Score:2)
Use a perl script (or a BASH script - whatever) to read the contents of the
Assuming that the first-level backup (over the
Windows is your limiting factor (Score:5, Insightful)
Re:Windows is your limiting factor (Score:1)
Nice, FAT16 with a cluster size of 3,6 MB. ;-)
Yes, I know this is not supported...
Re:Windows is your limiting factor (Score:1)
you could use the Dynamic Volume support of Win2k and later, but... I imagine that even NTFS is more portable than this...
Windows isn't really the limiting factor though. Each operating system has formats they accept.
Linux supports ext* , fat* , and a few other random (but not much used) formats.
Windows supports fat* , ntfs* (there are multiple versions, currently version 4 or 5 (i forget which) and windows dynamic volume, which is a bizarre thing I never quite understood (except that it c
Windows Dynamic Volumes (Score:1)
Re:Windows is your limiting factor (Score:4, Informative)
MacOS 10.x supports HFS, HFS+, UDF, ISO9660, AFS, and FAT*
Windows XP supports ISO9660, NTFS, FAT*, and UDF.
I believe that MacOS and Windows both require 3rd party software to use UDF.. but I could be wrong about that.
The solutions would be FAT*, ISO9660, and UDF. ISO9660 is read-only and I've never heard of someone using UDF on a harddrive (it is for those 'direct cds' you might have seen). FAT* sucks, but it works everywhere. It might be worth the effort to see if UDF could be used at all, but a small FAT32 partition would have to be made to accomodate the utilities for using it on the target system.
Before everyone flames the story submiter for being bias against Microsoft, the issue is that FAT really does suck and it would be great if there was something else that everyone supported.
Personally, I'd like to have a 6 gigabyte external (usb/firewire) harddrive that I could boot MacOS9 from AND share it between Linux and Windows computer. I guess I'll keep dreaming for a while
Re:Windows is your limiting factor (Score:2)
Re:Windows is your limiting factor (Score:2)
Re:Windows is your limiting factor (Score:2)
Apple has two ipod versions, one for MacOS and one for Windows. The one for Windows is formated with FAT32, whilst the Mac version uses HFS+.
I don't think MacOS can boot from the FAT32 version, but I could be mistaken.
Re:Windows is your limiting factor (Score:2)
I agree, with the lists of currently supported systems HFS sounds like the best option, it is already supported by two of the systems, and it is AFAIK far better than FAT. However it might be, that something else is better. Minix is very simple, and is much better than FAT. But maybe minix has some size limitations. Another choice I would consider is HPFS. AFAIK HPFS was developed for OS/2 while Microsoft was s
Re:Windows is your limiting factor (Score:2)
http://gamma.nic.fi/~lpesonen/HFVExplor
This means that 2/3 operating systems (Linux and MacOS) can read the format natively, and one can do it with a free 3rd party utility.. not too shabby.
As others have said, ext2 can be read from all three but requires 3rd party utilities on MacOS X and Windows.. plus, you can't boot MacOS from ext2
Re:Windows is your limiting factor (Score:2, Interesting)
Linux OTOH can boot from almost anything. However I allways boot Linux from ext2. A 31MB
Re:Windows is your limiting factor (Score:2)
My calculations say it should only be 48.5kb, but let's not discuss that minor detail any further. The problem you describe is very similar to one of the problems FAT16 has with large partitions. HFS is slightly better than FAT16 because it doesn't require cluster sizes being a power of two. And HFS allows partitions larger than 2GB (but maybe not much). Anyway
Re:Windows is your limiting factor (Score:2)
Somebody suggested to use the HPFS driver from NT4 with later Windows versions, so you might be right about the support being removed. Do anybody know why HPFS support was removed?
Nope, you are wrong. Sorry. (Score:2)
Re:Windows is your limiting factor (Score:2)
Re:Windows is your limiting factor (Score:3, Informative)
The problem is not so much the fraternizing, but the fact that the installable file system interface documentation is not available to your average open source hacker. The IFS Kit costs $899 + S&H [microsoft.com]. You just can't integrate other file systems cleanly without these docs.
In fact, there are utilities to read ext2 [swin.edu.au] and ISO9660 [isobuster.com] FSs, but they are stand-alone and require you to extract the files to your native partition before they can be used.
Re:Windows is your limiting factor (Score:2)
Now the questions are:
But I guess its considered fratenizing with the enemy.
I don't think trying to make ext2 the best cross-OS filesystem is fratenizing with the enemy. Using the enemy'
Re:Windows is your limiting factor (Score:1)
Re:Windows is your limiting factor (Score:1)
Re:Windows is your limiting factor (Score:1)
Re:Windows is your limiting factor (Score:2)
Re:Windows is your limiting factor (Score:2)
Interesting. From what I have read, I got the impression, that loosing all of your files would be very likely, if you actually used NTFS write support.
Re:Windows is your limiting factor (Score:2)
Uh, no. (Score:2)
Re:Uh, no. (Score:1)
Re:Linux is your limiting factor (Score:2)
#
# File systems
#
CONFIG_AUTOFS_FS
CONFIG_REISERFS_FS
CONFIG_ADFS_FS
CONFIG_AFFS_FS
CONFIG_HFS_FS
CONFIG_BEFS_FS
CONFIG_BFS_FS
CONFIG_EXT3_FS
CONFIG_JBD
CONFIG_FAT_FS
CONFIG_MSDOS_FS
CONFIG_UMSDOS_FS
CONFIG_VFAT_FS
CONFIG_EFS_FS
CONFIG_JFFS_FS
CONFIG_CRAMFS
CONFIG_TMPFS
CONFIG_RAMFS
CONFIG_ISO9660_FS
CONFIG_ZISOFS
CONFIG_JFS_FS
CONFIG_MINIX_FS
CONFIG_VXFS_
Re:Linux is your limiting factor (Score:2)
You conspicuously left out UFS, which is the "native" filesystem for BSD derivatives.
Don't forget FreeVxFS which is binary compatible with the commercial VxFS available on both HPUX and Solaris (others too?).
Then there's HFS and HFS+, which Apple use
NTFS (Score:2, Funny)
Re:NTFS (Score:2)
No, the really sad part is the fact, that it is true.
Give it a rest! (Score:2, Informative)
Re:Give it a rest! (Score:4, Interesting)
I myself am very interested in the answer. I wonder if the solution may be to have an ext3 (or xfs, or jffs) shim for Win32, also?
Man, if only there were an *open*, *journaled*, *fast* and *efficient* filesystem which all 3 OS's were allowed to play well with.
Seems to me if the answer to this "Ask Slashdot" ends up being "just use FAT32", then there's an opportunity for a decent OSS project: completely open, cross-platform, fast, journaled filesystem, with code tarballs for all major platforms.
Hmmm...
... followup ... (Score:2, Interesting)
- Windows is the problem: what open source filesystems are there for Windows, anyway?
Re:... followup ... (Score:2)
Port EXT2FS, UFS, whatever over to Windows?
D.
Re:... followup ... (Score:2)
A better and simpler solution is to just get rid of windows, problem solved.
Re:Give it a rest! (Score:2)
Write-limited card (Score:2)
Not to knock journaled filesystems, but wouldn't they pull extra writes in order to store the journal on the removable device? I'm not 100% sure about journal writes, but I think this require at least 1 extra write to th
Re:Write-limited card (Score:2)
--The OP has an external HD, which doesn't have the same kind of write limits.
fat32 is your best bet. (Score:5, Informative)
Your other options include Paragon's Mount Everything [mount-everything.com] and their Ext2fs Everywhere [ext2fs-anywhere.com] (which is really just a subset of `Mount Everything'.) These programs let you mount ext2/ext3 under Windows, or let you mount NTFS under Linux (I don't know how good that is -- I know that Linux has some NTFS support itself, but know it's not very mature.)
If that's not clear enough -- if you want to spend some money, spend it with Paragon and you can use ext3 or NTFS. If not, stick with fat32.
Flame or Praise? (Score:2)
I'm probably gonna buy a copy myself. Thanks for telling me about it. Keep up the good work, and please read more carefully.
Re:Flame or Praise? (Score:2)
Who's jumping to conclusions? I never said he said `free'. But this is /., and more people than just the original `anonymous reader' read these responses, and many /. readers prefer not to pay for software -- some out of principle, some because they prefer the free alternatives, and some because they prefer the local warez site.
Also, he says `I do a lot of database work' not `I'm a well-paid database wonk' -- for all
Why not XFS? (Score:2)
In other words, look at the problem from a tools perspective, not a filesystem perspective.
Re:Why not XFS? (Score:2)
Re:Why not XFS? (Score:2)
ext2 is a valid option (Score:2, Informative)
See here for general info:
http://www.ibiblio.org/mdw/HOWTO/Filesyste
And here for windows tools, but read the link... First.
http://www.it.fht-esslingen.de/~zimmerma/
And finally for OSX:
http://www.versiontracker.com/dyn/moreinfo/
Play nice and have fun!
Re:ext2 is a valid option (Score:1)
I don't know about the windows version of ltools, but the C command line utils it provides do not work very well under Unix. It doesn't allocate addtional blocks for a directory after a relatively low number of files are copied into a directory and a few other things. Maybe it's been fixed in a recent version or the guy is concentrating on the Java/C# implemenatation. I was very disappointed in it, so I wrote my own set of tools using the ext2fs library to read & write to an ext2/3 filesystem on a pa
I think you're stuck... (Score:1)
I too have an external (FW + USB2) drive that I like to keep quick backups on from my WinXP desktop and my MacOS X laptop. Unfortunately, while FAT32 works across platforms, it doesn't support any of the permissions, etc. that are native to each platform. One more thing to consider is that FAT32 doesn't support large files (>4GB per file).
What I've done so far is to use FAT32, but then use "disk images" on the MacOS X side to emulate the HFS+ file system (to keep p
Re:I think you're stuck... (Score:3, Informative)
I think you've about nailed it though: while there are a lot of valid criteria for selecting a good filesystem (security, permissions, metadata, etc), one of them in this case has to be portability, and without the help of third party software, no version of Windows has support for anything other than FAT* or NTFS. And while NTFS isn't such a bad filesystem, incomplete support for NTFS's s
Re:I think you're stuck... (Score:1)
Re:I think you're stuck... (Score:1)
Re:I think you're stuck... (Score:2)
Just a wild thought.
Eh, pay no mind ta me, I'm rambling.
Re:I think you're stuck... (Score:2)
That is not a good choice. You will still suffer from one of the worst performance problems in FAT32. Besides it is not much better than just making three partitions on the device (actually I think it is worse). And finally it is not much help, when you want to transfer files between the systems.
stuck (Score:2)
Re:stuck (Score:2)
You could use a small linux bootable cd, like knoppix. That one already has fdisk, etc. Hopefully USB mass storage supoort as well.
Try this (Score:2)
I suggest looking into that, as all oses should be able to read one of the standard CD formats...
Re:Try this (Score:2)
CD-ROM/CD-RW filesystems are multiplatform. If you could figure out a way to trick the PC into thinking a hard drive was a CD-ROM/CD-RW (a giant one obviously), it would solve the problem.
I never said I knew how to do it, I just posed it as an alternative. Do you know if it is possible or not? If you don't, then it would actually be you that doesn't know what they're talking about.
Why are you asking this? (Score:1)
Why? It's the only real choice you have.
Re:Why are you asking this? (Score:2)
That's easy (Score:1, Redundant)
UDF and iso9660, duh! That's all that's left.
repartition or you're stuck (Score:1)
(why ext2 and not ext3? many tools for other OSes can't handle ext3)
Re:repartition or you're stuck (Score:2)
Unfortunately, same goes for ext2 in general. You can compile your kernel with large file support, but your applications and filesystem drivers need to support it as well.
Re:repartition or you're stuck (Score:3, Insightful)
It is true that applications need to support large files, however the most important applications already do. And ext2 does support files larger than 2GB. I just created a 17247252480 bytes file on my
ext2 supports large files (Score:2)
any correctly written application compiled with a glibc >=2.2 will support large files.
Windows support for ext[23] (Score:1)
http://uranus.it.swin.edu.au/~jn/linux/explore2
Ext2 or fat32 (Score:2)
I really wish people would give more support to those who are trying to develop cross-platform filesystems. If
Ext2/3 on Windows (Score:2)
Re:Ext2/3 on Windows (Score:2)
Formatting the partition (Score:2)
Other than that, any of the other platforms available should be able to format the partition if you so desire
And why do you want a single monolithic partition? (Score:2)
Still need a more elegent solution than FAT32, put zipslack in a smallish FAT32 and boot into Linux there (or knoppix li
How about 2 out of 3? ;) (Score:3, Interesting)
Okay -- this one is purely for the "FWIW" file...
You could run NT's other original filesystem, HPFS. Linux has decent HPFS support available, the allocation unit is a 512b sector, and it's organized for fast searches and minimal fragmentation. It can also format up to 64GB partitions (although it still has a 2GB filesize limit).
The trick is to get HPFS support for Windows. To do this, you need to get the driver files from Windows NT v3.x (something that, admittedly I'm not sure works with Windows versions > NT 4. I don't do Windows personally, so I haven't tested it -- like I said above, FWIW). That will give you two of the three OSs supported. HPFS has been around for a while, (circa 1988), so you might be able to find something from the Intel FreeBSD world you could port to OS X.
I use HPFS for my 100MB ZIP disks (which I admittedly rarely use anymore for anything than quick archival purposes). It's not journalled, but it uses a bidirectional sector pointer system, so chaining errors are amost always fixable. The big downside is that if the filesystem is dirty, checking it can take a huge amount of time.
It's probably not a practical solution (I didn't and won't claim it is), but it's still a slightly more constructive answer than "Your stuck with FAT32" :).
Yaz.
ext2 (Score:3, Informative)
Re:Samba to the rescue!? (Score:2, Insightful)
Does that answer your question?
Re:Samba to the rescue!? (Score:2)
Re:Samba to the rescue!? (Score:2)
It might make some sense if he DID use samba, netatalk, and NFS.. as part of a portable NAS device... why doesn't someone make something like that?
Oh right, they do [procom.com]... it is a bit large, but I could imagine worse.
Thinking about it.. why couldn't he get get a laptop with a large harddrive and connect that via USB, firewire
Re:Samba to the rescue!? (Score:2)
--40GB over 100MBit Ethernet still takes quite a while. Altho if the wire speed were faster or his files smaller, your idea would be a good alternative.
Re:Samba to the rescue!? (Score:2)
Re:Samba to the rescue!? (Score:2)
Anyway, as I stated before.. UDF is the way to go if he can get writing to work; otherwise, FAT32 is the only solution.
Re:Samba to the rescue!? (Score:2)
Samba is a very viable option since it allows him to use any file system he choses.
Re:Samba to the rescue!? (Score:2)
As stated before, I'd think that having a portable fileserver would be a great idea.. but not a central one.
Re:Samba to the rescue!? (Score:2)
Re:Size of enclosure (Score:2)
Dude, you're stuck in a rut (Score:2)