Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Linux Support for Hybrid Hard Drives?

Posted by Cliff on Fri Mar 03, 2006 11:35 PM
from the better-storage-tech dept.
Data Storage
christoofar asks: "HHDD (Hybrid Hard Disk Drive) technology has been receiving some buzz lately. The concept is not new, but Samsung has been working on a consumer version of HHDD that everyone can use. HHDDs are disk drives that carry onboard RAM (in this case, NAND flash) which is non-volatile and offers to speed boot times and writes to the disk. This carries enormous benefit to laptop users who need to keep their disk activity to a minimum in order to preserve battery life. Given that Microsoft is adding support for Hybrid Hard Drives in their upcoming Windows Vista release, what efforts are being undertaken in the Linux realm to use this new storage technology?"
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Linux Support for Hybrid Hard Drives? 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • It should just work (Score:5, Insightful)

    by saskboy (600063) on Friday March 03 2006, @11:43PM (#14848390) Homepage Journal
    Why aren't the drives designed in a way that the drive technology worries about the details of retrieving and writing files, and sends them the way USB, IDE, SATA, or SCSI drives already do? Why would these drives need special drivers?

    • Re:It should just work (Score:5, Insightful)

      by mrchaotica (681592) on Friday March 03 2006, @11:48PM (#14848416)
      I would guess that it's because having a software-based queuing and caching system is more flexible. For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

      Or if you've got a laptop, you maybe want to cache writes to the flash if you're using the battery, but write directly to the disk if you're hooked up to the mains. Since the operating system knows about this but the drive doesn't, the operating system needs to control the drive.
      • For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

        Still sounds pretty much like disk caching. During bootup, the system's going to be asking for the OS anyway*, so if you've got a big enough amount of RAM to load it into, you can just do that, can't you?

        *I realize that this does not explain current boot times. Windows, Mac OS X et cetera are obviously spending some amou

      • The simple solution is to just make a specific part of the hybrid drive be RAM based. It would default to being sectors starting at sector 0. Then the method to change it would be to partition the drive with a special partition code for a partition that will change the sector locations to be in RAM to what they are for that partition, up to the end of the partition or the RAM capacity, whichever is first. Then RE-partition it back to what the OS needs, which will not make changes on the hybrid drive (it

          • Maybe someone doesn't want their operating system on the RAM-ified part. Maybe they want swap space there instead. Maybe they want the reserved journal space of a journaling filesystem that is frequently updated to be there. Of course all of these needs could be addressed in some way (at least I know some journaling filesystems let you put the journal in a different partition). You'd have to choose which sectors (don't think in cylinders anymore, it's all fake anyway) would be RAM-ified in a way that wo

      • I would guess that it's because having a software-based queuing and caching system is more flexible. For example, if you want to optimize operating system startup, then you need some way to specify that the operating system is what you want in the flash memory, instead of some other data.

        You could easily have a drive that appears to the OS as two drives; one "drive" would always write to NAND, while the other would be the real disk, with any free space on the NAND part being used as the persistent cache. It

    • Re:It should just work (Score:4, Interesting)

      by AKAImBatman (238306) * <akaimbatman.gmail@com> on Saturday March 04 2006, @12:49AM (#14848624) Homepage Journal
      I'm guessing that it's not so much that it can't work that way, as it is that it would kill performance.

      Today's operating systems try to do exactly what this drive is doing by pumping all reads and writes through the paging system. The problem is that the OS can't take a full performance boost from this or data will be lost in a crash or power loss. You may remember that EXT2FS could easily lose all the data you'd recently written if it wasn't unmounted properly. (It ticked me off when I tried to move RedHat 5.2 packages to a special partition, then found out that they had all disappeared on reboot because I hadn't explicitly unmounted the disk. *grumble*)

      Since this technology has little to no chance of losing data in a power failure, the OS can be modified to write the blocks immediately. This could easily result in a performance increase of 2 to 3 times what you normally see today. The improvements in writing meta-data alone could easily provide this increase.
      • ...then found out that they had all disappeared on reboot because I hadn't explicitly unmounted the disk. *grumble*

        The SYNC command forces the system to flush all buffers to disk. Once it returns, you know that all currently cached data has been at least scheduled to be written to disk. (current documentation claims that it doesn't return until the data has actually been written). Once data goes to the HD, it may still take a second or more for it to clear out of the internal drive cache.

        for older kern

        • On W2K3 there is an option for enabling proper syncing (if your hard disk is enterprise suitable). It is enabled by default and the performance drop from W2K to W2K3 is shocking, until you disable it. Which means: don't use W2K for anything important - upgrade your servers to W2K3, take the performance hit, but keep your data safe. Microsoft ought to give you a refund on your W2K Server purchase as it is not fit for the purpose for which it is sold.

          Having flash in front of the platters will allow this same
          • If you do it properly, you shouldn't have that much of a performance hit. Linux also has subroutines for explicitly flushing data on a specifid File Descriptor to disk (for situations where the order is important, like databases).

            Between the horrific speed differences between RAM and disk and the importance (nay -- criticality) of permanent storage, disk caching is an especially touchy space to work the tradeoffs.

            If you don't cache at all, then you get horrific speed hits. If you cache too eagerly, th

            • I saw it under email loads, so it could well only be syncing for important data (ie, ensuring the email is in the queue folder an on disk before telling the SMTP client that it has queued it).
              • That barely makes sense. You should be able to write to disk far faster than you read network data -- unless you're reading from a RAIDed mail server over a gigabit LAN.
                • unless you're reading from a RAIDed mail server over a gigabit LAN.

                  Which is how most businesses are set up these days...

      • It would not kill performance at all. It might fail to improve it if the OS is paying no attention to what is going on with the drive. If the OS queues the writes to the RAM-ified sectors, then you get the old performanc as if this was not a hybrid drive. Reads can't get better if the page is already in system RAM anyway. But I could argue that an OS that queues writes for a reason other than the drive being busy or powered down is doing queueing badly, anyway. If the drive is idle and a process writes

    • There are a couple good replies to you already, but I'll add one more place the OS might be able to improve its performance if it knew about the hybridness of the disk.

      I have a hunch (this is unsupported by data; you'd want to do some profiling before making a decision to do this) that you would want swap reads and writes to go right to the platter rather than flash. One reason is that flash does "wear out" after a while, and moving swap so that it doesn't use the flash would likely increase its lifetime su
        • I regularily see the amount of memory allocated to Firefox alone over 500 MB...

          Currently Windows reports 920 MB pagefile usage. I think that's probably total memory use, not the amount that's swapped out, but I can almost guarantee that there's something on disk. Windows seems to be very bad with swapping; before I added another GB of RAM (for a total of 1.5) it paged fairly frequently. After coming back from work and unlocking it, it would take probably 10 seconds to page everything in from disk.

          A laptop (
    • If access to drive contents are bound to runtime drivers then that content can be made bound to DRM. Am I the only one noticing a nice little trend happening...

      --Neth

  • What to put in there? (Score:5, Interesting)

    by aspoon (794081) on Saturday March 04 2006, @01:18AM (#14848695)
    The question is not whether Linux will support the drive -- someone will eventually write one. The question is what files to put in there to make it boot faster. Perhaps should be done is to put the entire swap partition in there, and put those "hibernation" files in the swap. So it would resume fast, and would also make normal operations faster with the faster swap.
    • would also make normal operations faster with the faster swap.

      If you're swapping while running normal applications, something is wrong with your system. If an increase in swap speed is immediately noticeable as better system performance, you need more memory. I do like the idea of suspending to flash though.

      For the pendantic: sure, OCCASINALLY you might have to dip into swap. And sure, OCCASIONALLY you'll have to pull in some really stale pages. But these are supposed to be rare events. But if these ar

      • If you're swapping while running normal applications, something is wrong with your system.

        Either that, or you're just dirt poor and trying to surf the net on 5-year-old equipment.

      • how about 'boot state' recorder to flash.. as in, unless you install software or add hardware, your system memory at bootstate can be written to a flashchip the same size as your ram, and you can boot, or reboot, as fast as you can re-dump the chip into ram.

        add new hardware? fine, as soon as yer system is booted for real, save it to the bootchip.

      • If you're swapping while running normal applications, something is wrong with your system. If an increase in swap speed is immediately noticeable as better system performance, you need more memory.

        And of course, memory is free, so we'll just all run out and solve that problem right away. :)
      • It sounds like they want to mostly cache files that get read every boot-up, which would tend to not get changed much. This would avoid the however-many-writes limitation of flash.
        • That is exactly how this work. The reason it's so useful for a Windows box arises from one of the ways in which Windows makes interesting use of files on disk.

          When a DLL is loaded into memory from disk, it's only relocated if there's already another DLL loaded in the same slot in virtual memory. If there's no conflict with the DLL, it's not relocated; code pages are read directly from disk and never written back. Instead, when they're swapped out (it the machine doesn't have enough RAM), the page is simpl

  • Why wait? Hack one up yourself! (Score:5, Interesting)

    by Myself (57572) on Saturday March 04 2006, @01:36AM (#14848741) Journal
    You shouldn't have to wait for the drives to come out. Laptop drive controllers can address a master and a slave, they just don't have a slave drive connected in normal usage. I was just looking for an adapter [logicsupply.com] to let me put a pair of CF cards in place of a 2.5" hard drive. (I can only find the single-card version in a 2.5" form factor, all the dual-card ones are for 3.5" mounting.) I figure, put a solid-state card in one slot and a microdrive in the other, and I've got a hybrid-drive laptop, right?

    I just got rid of an old toughbook cf-25 [wikispaces.com] that would've been perfect for this, as the drive mounting is gel and would easily accomodate an oddly shaped adapter instead of a regular drive. Or for the truly insane, a CF card piggybacked on a regular 2.5" drive! All I need is the ability to home-brew those little flex cables, and I'd be in business.

  • Nice to see (Score:3, Insightful)

    by Telvin_3d (855514) on Saturday March 04 2006, @01:37AM (#14848744)
    With all of the improvements in RAM, on-die cache and processor speed, the bottleneck in performence is HD speed. Anything that helps boot that is a welcome improvement.
  • I'm not sure if write caching in the flash chips is a good idea...

    Flash chips write at around 1Mb per second. Tops. Modern harddrives write around 50Mb per second. You'd need quite a lot of flash chips in parallel to top that.

    Of course, you can cache acouple of writes on the laptop drive to prevent having to spin up the disk. But it is not going to be a speed issue.

    • Re:Write caching in flash... (Score:5, Informative)

      by greg1104 (461138) <gsmith@gregsmith.com> on Saturday March 04 2006, @05:45AM (#14849217) Homepage
      Samsung's OneNAND design uses some interesting implementation details to run faster than is normally expected with flash. Their specs at http://www.samsung.com/Products/Semiconductor/OneN AND/index.htm [samsung.com] suggest writes at 9.3MB/s and reads at 108MB/s; plenty fast for many applications even without running multiples in parallel. It's certainly much slower than writing directly to the drive if the drive is active. However, if the drive isn't spun up at the moment and the amount of data to be written fits in the NAND flash cache, I could see this being a net performance boost over the spinup/seek time combination of the hard drive.

      They also spec 100,000 erase cycles before it's worn out. As was noted by an underrated poster the last time [slashdot.org] this came up, intelligent flash controller designs like this can cope with bad bits and assuring level usage of the memory much better than what you normally see in random hunk of flash.
      An analysis at http://www.sudhian.com/printdocs.cfm?aid=686 [sudhian.com] suggests 33 years of usage for a typical worker. When you run the numbers it doesn't sound that difficult to create a design that would likely outlast the mechanical parts of a standard hard drive.
    • 50Mb/s yes, but only sequentially. As soon as you're seeking you get a few ms of delay on a HDD, but a few us or ns on anything solid state. So basically if you're encoding video a HDD or RAID array would work great, but if you're reading/writing lots of little files (say, in swap, or in a DB server) flash would beat it hands down.

  • Other good uses for this (Score:4, Interesting)

    by nial-in-a-box (588883) on Saturday March 04 2006, @02:52AM (#14848906) Homepage
    I think the real demand for flash-cache hard drives should be in the area of data loss prevention. Anyone who has any doubts about the fact that laptop hard drives are still way too fragile for the job needs to work at a help desk for a week. Any way to keep those drives spun down as much as possible should be seen as a good idea, since speed does not really matter much if you lose your data and/or need a new drive every three months simply because you like to move your laptop while it is running.

    I know this just adds another point of failure to the mix with the addition of flash memory. However, with the apparent improvements in the quality of flash memory, I would expect one of these drives to outlast a current laptop drive by at least 50%. (Note: this is just idle speculation, but I don't know of any solid real-world statistics on laptop hard drive lifespan. I'm guessing laptop makers don't want us to know, either.)

    I think that the point of drivers is, therefore, valid. There needs to be some sort of intelligence behind this system to allow frequently needed files to be held in cache in order for this to work effectively. Sure, you could build a drive that could try this on its own, but odds are you would totally throw out any performance or power advantage by doing so.

  • Another use? (Score:3, Insightful)

    by agentcdog (885108) on Saturday March 04 2006, @03:12AM (#14848940)
    So I see a completely different picture here. There are a lot of files on the HDD that are never rewritten. System files, etc. This is where having hybrid drives really helps. Put the main boot files, executables and libraries which are accessed when staring things up in there. So no fast saves, but no load time.
    • This has nothing to do about speed, you will not gain speed by using the flash, it's just a way to write data to the harddrive without spinning up. This will earn you
      1. power consumption (alot)
      2. less wear on the HD

      nothing else.
  • Seeing as I can't buy one of these drives from my usual shopping websites, why would I care? Obviously the technology is very new. I would much rather let Samsung and Windows sort out all the defects at their cost and embarrassment than risk buying a kludgey piece of hardware for my linux computer.
  • Its not a hybrid drive, but the clever one I saw was a hard disk-backed 2 gig "solid state" drive. It was a board in a 1U rackmount box with 2 gigs of regular ram, a battery and a 2 gig hard disk.

    All reads and writes are to the ram. The hard disk isn't even running. When power is lost for more than 15 seconds, the hard disk spins up and the contents of the ram are written out to disk. When power comes back on, the drive spins up, the contents of the disk are loaded back in to ram and operation resumes.

  • It looks like the answer is "yes" (Score:3, Informative)

    by NetRAVEN5000 (905777) on Saturday March 04 2006, @08:30PM (#14851881) Homepage
    I've got the 2.6.15.4 kernel and there's a spot to enable support for "Memory Technology Devices".

    Here's the description:
    "Memory Technology Devices are flash, RAM and similar chips, often used for solid state file systems on embedded devices. This option will provide the generic support for MTD drivers to register themselves with the kernel and for potential users of MTD devices to enumerate the devices which are present and obtain a handle on them. It will also allow you to select individual drivers for particular hardware and users of MTD devices."

    It's got options for a whole bunch of things, such as "FTL (Flash Translation Layer)", "NFTL (NAND Flash Translation Layer)", and "INFTL (Inverse NAND Flash Translation Layer)".

    Given that it says the hard drives will use "a one-gigabit OneNAND flash chip" (according to the article), it sounds like it will work.

  • I would think the easiest way to accomplish the goal of HHDDs is to simply have 2 drives, one made of Flash RAM and one traditional.... You use the flash drive for your OS and programs, and then store all your data on the traditional drive.
    • But this is non-volatile memory we're talking about, so when it writes to the cache it is done writing the data. Subsequently transferring the data from the flash to the disk is more akin to defragmenting than "completing the write."

      • Re:Is it me or does this seem pointless? (Score:3, Insightful)

        by Anonymous Coward
        Right like, perhaps in addition to the kernel, or other essential boot files, it would also store the journal for journaled filesystems - it will cache the writes and commit them the next time that power supply and demand warrant. Consider a 1 Gbyte flash, in addition to your 100+ Gbyte drive, that's alot of non-volatile write cache.

    • Well, they only produce the isolinear chips for Star Fleet.
      Seriously, the NAND flash memory stores the data immediately. It is a perfect write cache, because it does not forget the data. Almost perfect, because its write cycles are limited. But a more reliable file system would do the trick, too. Journaling is one way of making things more reliable. But, you have a good point. It is important that some data get written ASAP onto the medium. Unfortunately Linux file systems are not exactly the right choice i
    • This could IMPROVE the situation you're talking about.

      First of all, you know that right now your writes are cached in your computer's memory for up to 30 seconds before they are flushed to disk. This is done so that synchronous read calls are less disrupted by writes.

      It's possible that this new technology would allow the OS to commit writes to disk immediately (or at least much sooner). The disk writes to NVRAM or flash, at which point it should be stable through power loss or crash. The disk doesn't need t
    • No, the new "Hybrid" hard drives don't just add more cache, they actually have built-in flash memory similar to what you find in USB Flash drives.

      Flash memory lasts quite a while - even longer than a hard drive, from what I've heard.

      The Mybrid drives load commonly-used files into their flash memory - not only for faster access, but also to prevent wear-and-tear on the hard drive's cylinders.

      Whatever is in this flash memory is already committed to the flash memory, and it'll stay there until it's overw

    • As long as your ram has a battery, no worries.

      Some ( all? ) SAN devices have write cache, as well as read.
    • Windows doesn't open executables for writing when you run them. In fact, when you run a program, it prevents writing entirely. Windows loads executables through the memory-mapped file interface. It does this so it can share code pages between different processes, and get away with paging out part of a program without actually writing to the page file first.

      Windows programs that modify their own EXE have to do some crazy tricks. While you cannot modify or delete a running program, you *can* rename or mov
    • I'd like to see linux have support for (and have hardware makers create) what SUN used to call NFS Accelerators. Basically add-on NV (or battery backed) memory sufficent to transparently cache write information. On my laptop I'd put it in my ExpressCard(tm) slot.

      Then again I have been considering using an SD type card to contain the journal for my ext3 file system.

      Actually, that combination; an ext3 file system in full data write-back journal mode, a solid-state flash device for that journal, and a "large

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.