Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Technology

Large Scale Solid State Memory Storage? 76

Posted by Cliff from the terabytes-and-petabytes dept.
spacechicken asks: "I am doing a theoretical study of an extremely secure large scale data storage concept. Due to the nature of the (theoretical) location of the (theoretical) warehouse some of our constraints include very few (if any) service visits, complete remote administration and no moving parts. Does anyone have any experience using or information on large scale (on the order of 10^12 - 10^15 bytes) deployment of solid state storage? And to preempt those who will say it - I have Googled."
This discussion has been archived. No new comments can be posted.

Large Scale Solid State Memory Storage? More

Large Scale Solid State Memory Storage?

Comments Filter:
  • I believe that this might be what you are looking for.

    The technology is quite mature.
  • You want almost a THOUSAND TERRABYTES (10^15 bytes is 909.5 terrabytes)... of SOLID STATE storage?!? Are you insane?

    That's about a quarter of a billion dollars in PC133 ram. I'm sure wholesale prices would bring that down, but you're still talking many millions of dollars.

    • Heh (Score:1)

      by itwerx ( 165526 )
      Since when has Uncle Sam worried about cost? :)
      • Heh, yeah, not often, but I still think you could make significant savings by using non-solid-state. You could get good reliability using RAID 0+1.

        The biggest consumer drive I know of is Maxtor's 320GB drive. It would take a minimum of roughly 3000 drives to cover ~900TB with no redundancy. The MTTF of these drives is over a million hours according to Maxtor, though that's over a hundred years so I question this figure. I wasn't able to find a price on the 320GB drives, but assuming double the cost of the 160GB drives, we're talking 3.2 million dollars for 2x redundancy, much lower than the 250 million price for single-redundancy in memory.

        As you can see, the cost is still enormous even with non solid-state. Maybe the article submitter needs to delete a few (theoretical) things so he doesn't need so much (theoretical) space ;-)
        • Remember that MTBF stands for mean time between failures. That's mean as in average. Half of the drives will fail in less than the mean time. The more drives you have, the greater the change that one of them will fail at any given time. If the MTBF is one million hours, and you have 1,000 drives, then unscientific but useful arithmetic says that the MTBF for your entire system drops to 1,000 hours, which is just over six weeks. As I said, the math isn't entirely sound, but it is very useful for estimating the overall reliability of a system.
          • umnn, it's even worse than that. The MTBF is calculated based on REPLACEING the drive before its end of service life. So drives may actually need to be replaced every year to make that series of drives get its million hour MTBF
            here is [pcguide.com] a good article on MTBF

      • Re:Heh (Score:1, Insightful)

        by Anonymous Coward
        Since when has Uncle Sam needed to rely on Ask Slashdot?

    • If you go at the other side of his range, it's only 930GB. Knowing that I bought 256MB PC133 RAM for $CAN32 last week, that makes it ~$CAN120,000, or ~$US75,000. And that price is street price, not wholesale.

      Now, of course you'll need backplanes, electricity, etc., so the cost will be higher. But with a range of 3 orders of magnitude, it's normal to have a price 3 orders of magnitude bigger.

      Back to the topic... With such a big deployment, you're bound to have some failures every few days. Do you have a way to replace the defective parts, or since there's no moving parts and no human intervention it's not possible?

    • That's a petabyte of longterm storage, not working memory. Ram and ionizing radiation don't mix. What he needs is perhaps holographic storage, both for the density, and for the environment he's talking about, an orbital archive. Sounds like somebody's getting pessimistic about the future of the earth, and wants to put a copy of the LOC where it may be found someday.
      I'd suggest you also add lots of impact shielding. Even if you put up a bunch of replicas, over time, they're all going to get little holes poked in them.
      The alleged "joke" posted earlier (probably modded to oblivion... I don't know, i surf noscore, -1, hidemods) about clay tablets bore a grain of truth. Large area/byte, physically robust, soft enough to not shatter on impact from extremely small, high-velocity projectiles, large reading interface tolerances.

  • Should be easy enough... (Score:3, Funny)

    by node 3 ( 115640 ) on Monday November 11, 2002 @12:20AM (#4640407)
    Just go the the place where you got your 10^6 processor smp server. I'm sure they deal with 10^15 bytes of solid state storage all the time and will be happy to get you what you are looking for.

    Hope that helps.
  • Uhm, isn't 10**12 to 10**15 bytes rather a lot?

    *pulls out a calculator* Hmm, 10**12 bytes is approximately 931 TB.
    WHATEVER FOR?!

    And that's on the low end, too. Since you're ranging a couple orders of magnitude, you might be pushing the limits of recognized abbreviations. All in solid state, too? That could be costly.

    • Actually, 10^12 bytes is one TeraByte and 10^15 bytes is one PetaByte.
      • Wrong. One terabyte (what's your fascination with InterCaps?) is 2^40 bytes. One petabyte is 2^50 bytes. The difference between the decimal expansions and the binary expansions become very significant at higher powers. For example, your statement that one terabyte is 10^12 bytes is wrong by 11%, or more than 99 billion bytes. That's quite a rounding error you've got there.
        • You can't really blame people from messing this up. The hard drive companies caused this when they decided that megabytes and gigabytes were based on powers of 10, just to make their drives look bigger.
          • No, the hard drive makers got it right. It was everyone else in the semiconductor industry who got it wrong by using SI prefixes (ie. the mega, giga etc. things) to mean various powers of two, when they really mean powers of ten. Nowadays there is a NIST standard which specifies prefixes for powers of two (the mebi, gibi, etc. things), but few people use those prefixes.
    • WHATEVER FOR?!

      Ummmm hello?!? alt.binaries

  • How about a combination of commercial IDE/SCSI RAID technologies chaining many individual SDRAM based solid state units along with some sort of SAN?

    For example

    http://www.storagesearch.com/ssd-3.html can hold up to 8gb per unit, and at 15 per SCSI card, 6 cards per system, you'll end up with 720GB per 'server'.
    Figure gigabit fiber (at the minimum) between machines and you'll end up with quite a bit of storage.

  • Question (Score:4, Interesting)

    by Thrasymachus Online ( 624213 ) on Monday November 11, 2002 @01:06AM (#4640565)
    The interesting part about your question is the complete lack of background on why you would possibly need all this storage.

    So I'll make a guess. 10^12--10^15 bytes is a large range. And I can only think of a few ways to generate that much data. The most probable is video cameras, but I can't think of any reason why you would need it secure in that fashion.

    Secure without human intervention is interesting. I mean, if all you want is security, the easy way is distributed networks and encryption. And really, that would be more secure in the event of nuclear war or other similar events.

    So I have two guesses:

    A) Given the similarity of your numbers to the 10^11 neurons in the human brain (and each neuron has as many as 1000 connections to its neighbors) this is some sort of screwy immortality thought experiment.
    B) Given the security requirements, this is some screwy thought experiment involving the preservation of the sum of human knowledge over a vast stretch of time without human presence. That could be interstellar travel, say, or large disasters wiping out the human race.

    • Re:Question (Score:2, Informative)

      by spacechicken ( 462919 )
      I've been waiting for someone to ask what it is about.

      Rest assured it is neitherone of the two screwy thought experiments you have postulated. It is another one entirely. Albeit one with a commercial bent.

      As I said earlier the reason for so much storage is that the site needs to be built in one go. There will probably be no ability to increase the size of the facility as requirements grow. We are not pulling numbers out of the air (or counting neurons). We are simply planning ahead

      • This have anything to do with the computer co-op?
      • Something out of this atmosphere?(in either direction)
      • Come, now! Surely you can give us more than THAT!

        You said that mechanical mechanisms would survive neither the final environment nor transportation there. I can think of a few places which would stress drive mechanisms, and which would be difficult/impossible to reach for repairs. Ocean floor, Arctic or Antarctic, deep space, ... The only one which would be hard on drives during transport would be a deep space launch (maybe? what about the drives they use on the shuttle/space station...?) Or if you were doing an airdrop or something.

        But I can't think of which one of these would have a "commercial bent" (at least not with a good ROI for a one-shot deal).

        <voice type="Dr. Evil">
        Throw us a frickin' bone, here!
        <\voice>

        -Ster
      • I think you're a meta-troll. And if so you've done a damn fine job of it
    • i was thinking genetic data storage
  • You want 1 Terabyte-1Petabyte of solid state storage, so here are some "back of the envelope" type calculations.

    Assuming slow, but commodity CompactFlash cards of 1Gigabyte each (currently $800 retail, your price may vary). You'd need 1000 to 10^6 of these puppies for approximately 800,000 to 800,000,000 $US (Retail). It would be fairly compact, fairly reliable, and fairly slow.

    So, with a price of $800,000 today for the low end, in 3 years (more or less) the price for 1 Terabyte of CompactFlash will be $100,000. This drops further to the point where I can afford it, in about 2 more orders of magnitude (7 years?).

    Bottom line, it's feasable, would be $1Million to $1Billion to implement 1 of (at retail, buy the fab, prices WILL drop dramatically). I'll be able to afford the same thing 10 years later.

    --Mike--

    • I'm sure the limited amount of times that CompactFlash can be rewritten would be a big problem for this project. What kind of latency do they need? More details on the actual nature of the project would really help in its design.

      Okay, so here's what you need to do. If you can stand a little latency and have a lot of money to burn. Launch a satellite into geosynchronous orbit that's always visible from the theoretical warehouse, that just echoes the signal back (and amplifies it of course). Broadcast out the data stream at as high a bitrate and over as much of the spectrum as you can cover aimed at the satellite with the intention of the satellite bouncing the same signal(s) back to you in a little amount of time. Thereby, you can get the following amount of storage out of the system (theoretically, just hope for no planes, you might want another satellite for redundancy):

      2 x total bitrate x distancebetweenstatelliteandwarehouse / speedoflight

      and a maximum latency of:
      2 x distancebetweenstatelliteandwarehouse / speedoflight

      of course if this "warehouse" is going to be moving, then you'd probably want a pilot signal to detect and compensate for any shifting. Now how easy is that :-P!
  • i got it! i've figured out all possible uses for this much storage:

    • bill gate's savings account
    • national debt ticker
    • google hit counter

  • Ok, what about other problems? (Score:5, Informative)

    by Klaruz ( 734 ) on Monday November 11, 2002 @01:50AM (#4640686)
    Using solid state drives you could probobly do it on the order of about a half billion dollars. Possible indeed. But there are other things you haven't thought of. You didn't say if you need to have this thing on 24/7, but I'm going to assume you do.

    1: Power. Solid state drives tend to forget stuff when shut off, so you'll need a UPS in the data center to handle it. No biggie, except when you realize that the batteries are going to need maintence. They do go bad after a while, I know of no batteries that don't. In theory you could do flash memory instead of volitile for about 3 times the cost (512meg ATA flash storage is $300 on pricewatch, add raid, san, etc, pricey but possible)

    2: Cooling. Massive amounts of solid state chips are going to generate massive amounts of heat. This means water chillers (most likely) and fans. Both involve moters. Moters go bad. You can build redundent, but eventually both cooling systems will go out.

    3: The hardware it's self. CPUs go bad, controllers blow up, ram chips go out, power supplies blow, etc. You can only leave a redundant system alone for so long until it's no longer redundant.

    4: Acts of god. Floods, fire, lack of fuel for power, emi. These things happen. You can build two data centers in seperate locations and write one off when something bad happens, but then you're back to no redundancy.

    5: Murphy's law. Don't forget, Murphy always wins.

    Having worked on a LARGE scale redundent system (Think uncle sam), I can tell you these things do require maintence. Building a system that large without bugs that creep up in a few years time is going to be next to impossible.

    That said, it sounds neat. Let me know when you guys need an engineer to build it, it'd be fun.

  • Remotely administering solid-state storage is no different than remotely adminstering spinning storage, just less chance of failure, so less visits. The data size you require can easily fit in a single rackmounted disk array. Vendors like E-Disk sell highly reliable solid state drives in IDE and SCSI, up to at least the high 10's of gigs in size per disk, 2.5 and 3.5 in form factor.
    • The data size you require can easily fit in a single rackmounted disk array.

      I think your calculation may be broken. Spacechicken is asking about storage in the range of one terabyte (lower bound, very easy to achieve) to one thousand terabytes (definitely possible, but bigger than damn near everything). You can squeeze about 1.8 TB into a rackmounted disk array using 160 GB drives; the new 320 GB drives will double this to about 3.6 TB, at least in theory. You're still 996.2 TB short of Spacechicken's high number.

      • Oops, I was off by 10^3 when I read his post :)

        Still, that just means you need several racks, and the problem against isn't solid-state specific. Check out EMC and Hitachi for better storage density in a normal array, but they may not offer a stock solid-state option (although I bet you could retrofit it with drives yourself). In any case, managing a large amount of storage, and managing solid-state storage, are two different things, both of which are pretty standard question with standard answers.

        I will say one thing - if you're thinking of SANing all of this storage to a cluster of servers - don't buy the existing market leaders' products. Cisco is on the verge of coming out with their second-gen SAN switches that are in catalyst chassis and support virtualization (read: veritas software-raid) inside the switch, making for a very nice solution.
  • If you're planning to shoot this into space, I'd think again. You're going to have trouble finding solid state storage that both holds a petabyte AND is radiation-hardened.

    The only other practical place I can think of right now where hard drives wouldn't survive the trip is some sort of undersea base. Sealand already has an offshore data haven, and they can visit theirs.

    Tim

  • Look At Me! (Score:4, Funny)

    by Llama Keeper ( 7984 ) on Monday November 11, 2002 @02:21AM (#4640783) Homepage
    SARCASM

    Look at me Mom, I got my question posted on Slashdot by makin up some bogus and totally illogical question. Then I made tons of vauge references and quasi-logical posts to make people wonder. Ooh ooh ooh look at me!

    /SARCASM
  • You're going to have to pull your submarine up to that underwater fiber optic cable much more frequently than you planned to. Why not just run another fiber from the underwater tap to a secure listening facility on land?

    (If it's not clear to most people reading this, the questioner wants to build a device that can listen in on a fiber optic tap and record a lot of data for a long time. Since fiber is vulnerable where it lies on the bottom of the ocean, and that position would necessitate the other requirements, that's the best guess.)
  • I assume this is for a space mission from previous comments, so talk to the people with the most space experience. For this type of application the only system I would use would be a raid 1 utilizing IBM's shipkill technology. Basically you would have RAID 5 at the dimm level and raid 1 at the system level. Would it be cheap, hell no. Would it be the most likely to survive radiation hazards, yep. IBM has nearly a half century of studying the effects of radiation on computer components, so if you want to put a complex computer system into space talk to them. For some information to digest before talking to them see the chipkill whitepaper here [ibm.com].
  • Did anyone think in terms of *real* solutions, rather than just pricing out parts and assuming this guy could magically make them work together?

    Anyway, at least one solution to this request appeared on Slashdot just a few days ago: The solid-state RocketDrive [cenatek.com].

    Perhaps not the ideal solution (I honestly can't say if OEM solid-state storage exists on a much bigger scale), but something that you can concretely say "this would work". Proof-of-concept, if nothing else.

    Granted, for the size you want, at $5k/4G, This would cost USD $1.3 billion just for the storage itself (not counting the array of 32k+ PCI slots you'd need to hold all these and the hellacious network to RAID them), but this sounds like a gub'mint project anyway, so cost presumeably forms the *last* of your concerns. If cost *does* matter, you can get the unpopulated controller boards for $800 each, and certainly a *much* better bulk deal on RAM then what Cenatek offers (basically they charge $1k/1G? Perhaps 10 years ago!).

    Checking Pricewatch, the average non-volume-buyer can get 1G of PC133 for around $100). That would lower the storage-only cost to only USD $315 million, before considering volume discounts.
  • This whole thing is a massive troll. Anyone who was really considering such a thing would be able to do the back-of-the-envelope calcs that would show that:
    1. Power and heat: either you don't have enough juice, or you melt down - ram also produces heat;
    2. Errors: the more parts, the more errors, so a project this size would be the worlds' biggest random error/number/data generator;
    3. MTBF: you have parts, you have failures. For example, Maxtor drives may have a million hour MTBF, but the company recently reduced their warranty period from 3 years to 1.
    If this was real, and your boss was even half-way competent, you'd be out a job immediately.

    The only storage that would begin to meet SOME of the requirements would be molecular/biological. Try again in a decade.

  • Have you thought about perhaps a custom many million mile long "spool" of fiber optic cable? What I am thinking of is something akin to a nickel wire delay line - except using fiber and laser pulses, photomultiplier tubes to pick up pulses on the other end, and high-power lasers to pump the thing - basically with the data going around in a very long circle, so that the data takes a half minute or so to make the trip - you could pack a lot of data in that length, much more with a multi-mode system (I think that is the right term). Of course, I am talking out my ass here, so it may not be that good of an idea...

Slashdot Top Deals

1 + 1 = 3, for large values of 1.

Close