Can Unix Mount .TAR.GZ and .ZIP Files?

rhadc asks: "We can mount filesystems. We are constantly downloading .zip files and .tar.gz's. Why can't we mount .tar.gz's or .zip files? Mounting compressed packages and archives would make installation of software a little easier. Depending on how it was implemented, it could also require less disk space than the current method: Download, unzip, work/play. I've been a C programmer for a while, although taking on a large filesystem-type project isn't something I have time for, right now." It's just a matter of some enterprising (or bored) hacker writing file system drivers which use /dev/loop*, isn't it? Are there any projects looking to do something like this? Programs that cheat and actually untar the file to a teporary directory don't count.

Roxen kind of does this.

[Pathwalker]

It's funny this question should come up now - the web server Roxen [roxen.com] added support for mounting a tar file in it's virtual file space earlier today [roxen.com].
--

AVFS

[Bazman]

Its on freshmeat, it took me two minutes to find it.

http://www.inf.bme.hu/~mszeredi/avfs/

Baz

dunno

[LWolenczak]

I guess its possible to be done in the kenrel with a driver. But I personally think it would be better if it was done in userspace with a tool. Kinda like winzip. Still, maybe you could write a driver for the kernel that a userspace app could plug into to access files in a compressed archive.

I'm sure if its thought out and done right, it could be done read only, but read write access probily would be harder to do. I bet there are some restrictions on archive file structure. That could be gotten around by just rewriting the file All together.

-LW

Compressed Ext2fs on loopback + other solutions

[BitMan]

There is a patch that should make it into Linux 2.4. It is included in VALinux's unsupported kernels from HJL [valinux.com] (as well as Ext3 and NFS, and don't forget the losetup/mount RPMs as well).

Otherwise, I just use Midnight Commander (mc) to peer into and even modify TAR, TAR.GZ, CPIO, RPM and other files. In fact, if you're serious about working on compressed files, why not get into RPM where you can script (among other things)?

-- Bryan "TheBS" Smith

Feasable, but practical?

[LionMan]

It's perfectly feasible, and would work just like a normal *nix character device, in almost every way. The only problem is quite slow access. For one thing, it is originally compressed (gz'd) so if every read/write went through a process optimized for say memory usage then the file would have to be decompressed every time. If it is kept as data in memory then writes to the physical data on disk would not be synced with the memory. So there are more practicality problems than feasibility. This are all managable and hack-around-able, but like many things that failed do to their reliability upon well-written programs using the API, it is likely to not work if someone does not know what they are doing (but then again it is *nix so that's true more often than not). Cooperative threads are one of those things that trust that "threads" will play nice - but not always. The same way, this fs would have to rely on people playing nicely, eg making sure that the memory is synced with the disk and so on. Well ANYhoo someone can always do something - Where there's a will, there's a hack.
-The LionMan

Re:The User File System

[mlepore]

The latest userfs code would compile on kernels 1.3/2.0, although it looks like someone has made it compile with 2.2/glibc. The author's page on userfs is at: http://www.goop.org/~jeremy/userfs/ [goop.org]

The old days...

[vs]

Back in the old days it used to work on an Amiga. Strange that this shouldn't be possible today (at least not simple...).

Read only or R/W?

[BLBishop]

If you all you want is read-only access, I don't see too much of a problem - it would be sort of like a CD ROM. Don't expect a timestamp on last access....

But read/write would be tough, because then you'd have to deal with fragmentation of files and gaps after a delete. Neither tar nor *zip would like that at all....and copying over to a new archive file defeats the purpose.

Re:/dev/loop* with gzip?

[dougmc]

The problem with that idea your application can do things like use seek() to jump around the file -- and this is completely legal, but it would give gzip/gunzip fits. If you allow seek() to be broken or a no-op, then it might be a lot simpler to implement.

But yet things like this *have* been done, because we *do* have compressed filesystems -- but I'm not sure exactly how they work. Perhaps they compress based on blocks rather than on files.

To answer the original question about mounting a tar.gz file, well I think the Hurd can mount a .tar file. To mount a tar.gz file would be a good deal more difficult, because you need to be able to seek() around in there. Mounting a .zip file might be a little better, because you have a table of contents at the end of the .zip file, so you can go directly to any file. The problem then becomes seek()ing within that file.

Midnight Commander

[Inoshiro]

Midnight Commander has in it a nice user-space VFS layer (which is set to become the Gnome VFS layer). You just hit enter on a tgz or zip file, and you 'view' it much like RAR did in DOS years ago.

Another cool thing is you can also have ftpfs, undelfs, and other interesting file systems through this MC VFS. The power is really apprent when you need a single file from a archive on an FTP site. Just connect to the FTP, browse to the dir, hit enter on the archive, browse to the file, and hit F5 to copy to the other pane (which has the proper CWD :)). Then just exit. Very clean and fast.
--

Re:Addendum:/dev/loop* with gzip?

[lunatik17]

Hmm... utilizing zcat perhaps? This would be a cool weekend project. Too bad I don't know how to code for zlib...

The User File System

[SIGFPE]

Many years ago when the Linux version number was less than one (approx 0.99) there was a rather cool user file system - a filesystem that ran in user space. It was easy to add custom file systems and the source came with an example for entering tarred archives exactly as you ask. Another neat example were symbolic links that were *executed* rather than followed. So 'ln -s "echo hi" hello' followed by 'cat hello' would give 'hi'. Another example was mounting the entire ftp world so you could just 'cd /ftp/ftp.slashdot.org' or whatever. It was fun to play with.
--

doesn't userfs do this

[spongman]

i believe that there was a 'userfs' module that supported things like this. read-only, though.

i use a product on windows called ZipMagic which supports read/write operations on .ZIP files at the file-system level. it even works in a command prompt - really smart stuff.

Re:Uhhh

[King of the World]

Nope. Read the tar.gz as a directory containing a collection of files.

/dev/loop* with gzip?

[SagSaw]

As a previous poster pointed out, it would probably be hard to have R/W aceess to a tar.gz file or a .zip file. What would be nice (maybe the tool already exists, I don't know) would be to mount a file to /dev/loop* and have everything going to/from that device pass through gzip, bzip, bzip2, or whatever your favorite compression program happens to be.

Does this already exist in some form?

Addendum:/dev/loop* with gzip?

[SagSaw]

You would, though, still need a user-space utility (probably actually a shell script) to create a file, run mke2fs, and copy the data over.

seek() compatable compression algorithms?

[SagSaw]

I have zero knowledge in this area, but is it possible to write an algorithm with would allow easy seek'ing withing a compressed file? It may be as simple as caching a block either side of the current position. Maybe the format would have to include information stating what block or other unit in the compressed file contains. (i.e. block 3 contains data for the 3004-7094'th bytes in the decompressed file). When seek'ing the file, you would have to scan throug the blocks to find which block contains your data.

Then again, I may be talking out of my ass.

Re:Midnight Commander

[iMMersE]

This VFS is not real

Good job they called it the virtual file system then, isn't it?

.iMMersE