Software SSD Cache Implementation For Linux? 297
Annirak writes "With the bottom dropping out of the magnetic disk market and SSD prices still over $3/GB, I want to know if there is a way to to get the best of both worlds. Ideally, a caching algorithm would store frequently used sectors, or sectors used during boot or application launches (hot sectors), to the SSD. Adaptec has a firmware implementation of this concept, called MaxIQ, but this is only for use on their RAID controllers and only works with their special, even more expensive, SSD. Silverstone recently released a device which does this for a single disk, but it is limited: it caches the first part of the magnetic disk, up to the size of the SSD, rather than caching frequently used sectors. The FS-Cache implementation in recent Linux kernels seems to be primarily intended for use in NFS and AFS, without much provision for speeding up local filesystems. Is there a way to use an SSD to act as a hot sector cache for a magnetic disk under Linux?"
isn't 40 GB enough for applications? (Score:4, Interesting)
Is there really a need for this? Intel 40 GB SSD still has a read speed of 170 MB/s and costs about 100 euro here in NL. Why have some kind of experimental configuration while prices are like that? OK, 35 MB/s write speed is not that high, but with the high IOPS and seek times you still have most of the benefits.
I can see why you would want something like this, but I doubt the benefits are that large over a normal SSD + HDD configuration.
Re:ZFS L2ARC (Score:2, Interesting)
Not Linux per se, but the same idea is implemented nicely on ZFS through its L2ARC: http://blogs.sun.com/brendan/entry/test [sun.com]
Swapcache on DragonFly BSD 2.6.x was implemented for this very reason IIRC.
http://leaf.dragonflybsd.org/cgi/web-man?command=swapcache§ion=ANY
ZFS' example (Score:1, Interesting)
What a waste of time. Just put /home on a magnetic disk and everything else on the SSD. This way, you can get away with a small (very affordable) SSD for your binaries, libraries, config files, and app data, and use tried and true magnetic for your important files.
Solaris' experience with use SSDs as a read cache shows 5x to 40x increase in read IOps:
http://blogs.sun.com/brendan/entry/l2arc_screenshots
while still getting the advantages of bulk storage with SATA drives (in various forms of RAID configuration).
This may not be a big deal for home stuff, but if you're serving homedirs, VMware VMDKs, or databases over NFS for work, it could save a lot of money in equipment and power/cooling just by adding a few SSDs.
Similarly using write-optimized SLC SSDs can help synchronous write operations (12x more IOps, 20x reduction in latency in some benchmarks):
http://blogs.sun.com/brendan/entry/slog_screenshots
More on the general concept of "hybrid storage pools":
http://blogs.sun.com/brendan/entry/hybrid_storage_pool_top_speeds
I believe parts of this functionality has been ported to FreeBSD as well (they're a few ZFS revs behind Solaris).
Re:I don't get it (Score:4, Interesting)
A fast SSD is not 1 Gb/s under ideal conditions. A fast SSD is up to 2 Gb/s (about 250 MB/s) under real life conditions (while reading). Anyway, it still makes sense to cache network content to disk if the other side of the connection is slow or not reliable.
preload (Score:1, Interesting)
I'm surprised no one mentioned "preload":
"preload is an adaptive readahead daemon. It monitors applications that users run, and by analyzing this data, predicts what applications users might run, and fetches those binaries and their dependencies into memory for faster startup times."
http://sourceforge.net/projects/preload/
Development seem staled, but i think the idea is there. Well, they attacked the problem of using unused RAM, but it could easily be adapted to use a SSD partition.
Sebastien Giguere
FSCache would work except... (Score:5, Interesting)
I have a similar problem and I tried the FSCache approach:
I've got two raids.
One is optimized for big ass files read contiguously and has raid6 redundancy.
The other is a much smaller JBOD that I can reconfigure via mdraid to anything that linux supports in software.
The problem is that 5% of the big ass files need read-only random access and that kills throughput for anything else going on. It takes me down from ~400MB/s to 15MB/s.
So, I thought I'd use the FSCache approach and use the JBOD as the cache.
I did an NFS mount over loopback and pointed the fscache to the JBOD.
It worked great got practically full throughput for contiguous access, for about 10 hours and then crashed the system.
Apparently NFS over loopback is well known to be broken in linux and has been since, essentially, forever.
I was stunned, it had never even occurred to me that NFS over loopback would be broken. Its freaking 2010 - that something I had been using on Sun0S 3 a bazillion years ago didn't work on linux today had not even entered my mind.
I've also tried replicating the files from the raid6 to the jbod, but that quickly turned into a hassle keeping everything syncronized between the files on disk and the applications that create the files on the raid6 and the apps that use the files on the JBOD. Plus, it doesn't scale out past the size of the JBOD, which I also ran into.
So now, I'm looking at putting the apps that need random access reads to the data in a VM and NFS mounting it with cache to the VM hoping to avoid the NFS-broken-over-loopback problem. I haven't had time to implement it yet, and personally and leery of doing so since I have to wonder what new "known-broken" problems will bite me in the ass.
So, if there is a better way, I am dying to hear it, unfortunately solaris/freebsd is not an option...