Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Hardware

Storage Area Networks vs. Local RAID Arrays? 43

Posted by Cliff from the comparing-storage-technologies dept.
Noxx asks: "My department is purchasing several new servers for an intranet website project. We are under pressure to store our content on an existing Storage Area Network accessed over a fibre connection rather than on a local RAID-5 array, to cut purchasing costs on the new hardware. Have any Slashdot readers evaluated the pros and cons between the two storage technologies, and are there any points of concern we should address? How does performance compare between the two, and is this a proper use of the SAN? If multiple servers access the same content from the SAN, is the possibility of introducing a single point of failure (ie: the SAN crashes) a valid concern?"
This discussion has been archived. No new comments can be posted.

Storage Area Networks vs. Local RAID Arrays? More

Storage Area Networks vs. Local RAID Arrays?

Comments Filter:

  • SAN is probably better (Score:4, Interesting)

    by crow ( 16139 ) on Monday January 28, 2002 @05:56PM (#2916260) Homepage Journal
    Generally, you will find that using a SAN is better. The sort of equipment that is deployed in a SAN is typically higher-end than what you would get with a stand-alone RAID array.

    You probably also get a number of other advantages. Your SAN is probably already backed up. Your SAN is likely already part of any disaster recovery plan.

    And while you could view the SAN as a single point of failure, you could also view your local RAID array as a single point of failure. Any decent SAN implementation has redundancy at every level.

    Of course, I'm biased, as I work for EMC, a big SAN company.

    You probably need to sit down with your IT people and discuss with them exactly how the SAN is set up. You'll probably find that it has more than enough reliability and performance for any web server application.
    • Ive set up fiber channel SANs..and i have only one thing to say -- DONT DO IT!. EMC and other vendors might claim to set up SANs and get em up and running in a flash. in reality you spend an ungodly amount of cash to get the fiber hardware in, pay EMC and other vendors a huge amount to plug it all in and fire it up only to find that firmware needs to be reflashed because one EMC box has firmware problems and is at a different revision level than the other EMC box. once you get past the firmware nightmare (after paying EMC or any other SAN vendor a huge consulting fee per hour of course) you end up with a freaking unstable SAN nightmare which takes months to fix. unstable equipment, firmware bugs, login problems, you name it. use RAID5 -- its fast dependable and millions of companioes have used it without any problems. its also mature. i use ICP Vortex RAID-5 cards for low end boxes or Sun Storedge A5000 , A3500 or equivalent for midrange.
      dont do SAN. at least not till it becomes mature.
  • San is always the better way to go...since it tranparently integrates into your network Topology. Therefore, for needs of quick off site, or offserver anyway access, backup etc, everything just works faster, and better.

    If your really need to get the low down on what is better, esspecially for your situation, to figure out what you really need I suggest you look up your local EMC office, or visit www.emc.com....its never failed me where I work...Nobody ever got fired for buying EMC.
    • If your really need to get the low down on what is better, esspecially for your situation, to figure out what you really need I suggest you look up your local EMC office, or visit www.emc.com....its never failed me where I work...Nobody ever got fired for buying EMC


      I don't know if anybody ever got fired for buying EMC, but if you are going to consider EMC as a solution, you should talk to the people at GunBroker.com [gunbroker.com], they use an EMC solution. They recently had a problem with the disk array, and here is a quote from their News and Announcements [gunbroker.com] page:

      "Our EMC disk array, which is supposed to guarantee 100% uptime, failed. It took EMC 24 hours to get it back online, and when they got it back online they corrupted our database"

      Althought the reason for the failure has not been disclosed yet, I would think twice about buying EMC.

      • Here is an email I received on the 21st of this month:

        These last two days have been the worst two days we have ever experienced at GunBroker.com. Our EMC disk array, which is supposed to guarantee 100% uptime, failed. It took EMC 24 hours to get it back online, and when they got it back online they corrupted our database. Although we have tape backup the tape runs at regular intervals, the crash occurred at the worst possible time. Everyone here worked 48 hours straight to restore the damaged data as fully as possible.

        We are extremely sorry that this happened. Downtime is extremely rare at GunBroker.com because we spend a lot of money on an extremely high quality infrastructure. This never should have happened at all because EMC disk arrays are never supposed to fail. We have a contract that guarantees that the EMC will not fail and that it will not cause us data loss. We are of course going to find out why this occurred and do whatever it takes to make sure that it never happens again.

        To ease the financial loss of our sellers who use this site as business income, we are waiving the Final Value Fee of any item that is listed Monday, Jan 21 through Wednesday, January 23 (eastern time). The FVF will appear on your real-time account info but will be removed before you get your monthly statement so you will never pay it. If you had not already noticed we did away with the fraud insurance fees (but not the Fraud Insurance) starting Dec 1, 2001.

        We have extended all auctions that were to end in the down period to be fair to our buyers and sellers. If you placed a bid or listed and item on the morning of Jan 18, please read the following and check to make sure that you bid or listing is still there.

        As a consequence of the crash it is possible that a small amount of data was lost from the time frame immediately before the crash. If you placed bids or listed an item after 10:00am on 1/18/2002, please check 'My Auctions' from the top of any web site page to make sure that your listing or bid is there.

        Once again, we are extremely sorry for the downtime. If you have had a problem that has been caused by this incident please 'Reply' to this message and tell us what we can do to help you and we will do everything we can to resolve the issue.

        GunBroker.com
        The Web's Largest Hunting and Sport Shooting Auction and the ONLY firearms trading place that guarantees your purchase against fraud.
        http://www.gunbroker.com

  • Security concerns (Score:4, Informative)

    by hectorh ( 113198 ) on Monday January 28, 2002 @06:26PM (#2916474) Homepage
    One thing that you should consider when connecting many servers to one shared SAN is the issue of security.

    Most security designs involve using "concentric circles" of security.

    Each ring contains a set of applications and data that have a common security concern or priority. The closer to the center that you get, the data becomes more valuable and therefore the security measures are stronger and more protective.

    The outer layers of the circle usually contain internet web servers, incomming mail servers, etc. The inner layers could contain such things as source code, payroll, billing, R&D, etc.

    If you share a SAN across layers of security, an intruder could use the SAN to bypass any security measures that protect the inner layer.

    And if you think that this is not possible, think again, I have read the results of a SAN security risk assesment performed by a large security firm, and they were able to plug in a laptop into the SAN and gain access to the SAN by making the SAN controller believe that the laptop had the WWUI (world-wide unique identifier) of a critical server that was down for maintenance.

    Can't give any more details, since I am under NDA and I cannot reveal the exact method used, or specific company names or brands.

    • sure....thats what they all say
    • And if you think that this is not possible, think again, I have read the results of a SAN security risk assesment performed by a large security firm, and they were able to plug in a laptop into the SAN and gain access to the SAN by making the SAN controller believe that the laptop had the WWUI (world-wide unique identifier) of a critical server that was down for maintenance.

      If you gain physical access: plug in a laptopor put a boot floppy into it, it will seldom be safe. You can get adminstrator access on most NT4 servers just by using a specailly crafted boot flop.

      I know the story about a security consultant telling that there was no plan for a failure of power. To prove it he unplugged the (AS/400?) box. He was right, it took them a long time to get the system in good order again. Last thing i heard a lawsuit was started. (So far for a single point of failure.)

      The point: A SAN can fail. a local raid5 can fail.
      e.g.
      SAN: plug a laptop in (your example)
      RAID: Steal the backup tape.
      replace te redundant disk(s), an put them in an other system, tadaa, copy of all data!

      Security does not become better or worse just because there is a SAN station.

  • Depends on what kind of a "SAN" you mean (Score:5, Informative)

    by foobar104 ( 206452 ) on Monday January 28, 2002 @06:39PM (#2916586) Journal
    The word "SAN" can mean two very different things: switched access to storage, or shared access to storage.

    The simplest kind of SAN has a number of computers and a number of storage devices all connected to a fibre channel switch. Each computer gets some of the storage for its own private use. No two computers ever mount the same filesystem at the same time.

    The advantages of that kind of SAN are mostly physical: buy a bunch of storage and put it on the SAN, then allocate it to the computers "softly," by changing LUN mapping and such, rather than by running new cables.

    If that's the kind of SAN you're talking about, I'd say go for it. The IS group that manages the SAN will take care of some of your problems for you-- maintaining the RAID hardware, namely-- but in all other ways it'll be just like direct-attached storage.

    The other kind of SAN allows multiple computers to mount the same filesystem at the same time and access its data over fibre channel. This is a lot more complex, obviously, because your storage software has a lot of work to do: keeping buffer caches consistent, managing file locking, propogating metadata updates, and on and on.

    This kind of SAN requires a special driver, like Sanergy or Centravision or CXFS. (Google 'em.)

    They're often more trouble than they're worth, especially if you start talking about large storage clusters (8 nodes or more). I'd avoid these.
    • "The other kind of SAN allows multiple computers to mount the same filesystem at the same time and access its data over fibre channel. This is a lot more complex, obviously, because your storage software has a lot of work to do: keeping buffer caches consistent, managing file locking, propogating metadata updates, and on and on."

      Isn't this what NFS is? This is coming from somebody who doesn't know better, so please keep the flames to a minimum.
      • Close, actually. The difference is that the computer sees the drives in this case as hardware physically attached where in NFS it's a mount point. Big difference when you think about it connection wise, but the difference to the user will be that you can "cat /dev/zero > /dev/san_disk_0" - you ought to not be able to do that with NIS...

        Shortly, you could IE use either ext2 or reiser on the NFS server but the clients would see it as NFS type. Via SAN, it would mount as ext2 or reiser directly.

        Hope I said that right. Flame on!

      • Re:Depends on what kind of a "SAN" you mean (Score:5, Informative)

        by foobar104 ( 206452 ) on Monday January 28, 2002 @07:37PM (#2916912) Journal
        Isn't this what NFS is?

        Yes, that's EXACTLY what NFS does. Shared-storage SANs try to do the same job in a different way.

        Despite what you might think, the primary difference between NFS and a shared-storage SAN isn't the medium; one uses gigabit Ethernet and the other gigabit Fibre Channel. The different is the presence of the server in an NFS environment.

        The server listens for mount requests and grants or denies them, and it responds to requests for data by reading the data from the disk, marshalling it, and shipping it off to the client.

        In a shared-storage SAN, these functions have to be performed in some other way. A common approach is to nominate one machine on a SAN to be the "metadata server." Any disk operation that doesn't involve reading or writing actual data blocks goes through the metadata server over Ethernet.

        For example, if you were doing a "cat" on your workstation, the "cat" program would first do a "stat()" to see if the named file is there, then a number of "read()"s to get the data. The "stat()" call would result in the disk driver sending a set of SCSI commands to the disk to get data out of the file's inode, and the "read()" calls would get blocks of data off the disk.

        In a shared-storage SAN environment, these two calls would be handed differently. The "stat()" call would be handled through communication with the metadata server over Ethernet, while the "read()" calls would access the disks directly with SCSI-over-FC commands.

        In an NFS environment, the NFS server would take care of both of those things; the NFS client would have to worry about neither.

        Some SANs use a dedicated metadata server (like Sanergy) while some have a complex and pretty darn cool scheme for nominating a metadata server dynamically (like CXFS).

        Maybe that helps shed some light on why my opinion is that shared-storage SANs are more trouble than they're worth.
      • >Isn't this what NFS is?

        No, and it isn't even close. A typical SAN presents itself to the host as a disk controller. It has no knowledge of file systems (those who tell you to plug into a SAN because it's already being backed up are smoking dope - you *can't* back up the storage if you don't understand the file system). A SAN is a very expensive disk subsystem, and yes, you do get what you pay for. Because the SAN has no knowledge of file systems, any volume sharing is totally up to the clients to coordinate. Present the same disk to two systems that don't run some sort of clustering or file sharing software, and you *will* corrupt your data.

        NFS, on the other hand, presents itself to your host as a file system. You share that storage with multiple hosts, and there is usually locking happening in the background. You don't have the same level of control or access on an NFS server like you'd have on a SAN.

        .../Ed (who manages an EMC SAN and multiple NFS servers)
  • Your requirements are probably different from these, but I think anyone thinking about storing a large amount of data cost-effectively will benefit from reading about Brewster Kahle and the Wayback machine:
    http://www.oreillynet.com/pub/a/webservices/2002 /0 1/18/brewster.html

    and at www.archive.org

    I was lucky enough to get a tour through the facilities several weeks ago. It can reset your perspective to understand that fast archive and retrieval in a multi-terabyte database is being reliably done using cheap networked clone PCs with IDE hard drives and (mostly) free software. Buy a system that can handle terabytes, and fast, for the cost of one high-end database server!
  • If you want to share devices between boxes, you'll want to check into GFS for linux (it's in the standard kernel) which will coordinate concurrent access to physical disks at the filesystem layer. GFS will work just as well on either a local (SCSI Y cable) disk array, or on a SAN, but the SAN disk will likely be easier to manage over time, especially considering you won't have to do anything strange to put more than 2 boxes in your HA configuration.

    Also, linux-ha has STONITH capabilities (Shoot The Other Node In The Head) to prevent a split cluster from corrupting your filesystem(s).

    As far as reliability, HP-UX (I don't think Linux can currently do this) can have dual paths to the same physical device on a SAN. This means you can pull the plug (or trip on the cable) on one of the fibre cards and the other will pick right up. It is possible that the qlogic cards will do this, but I haven't checked.
  • While I agree in principal with the numerous posts placing a SAN as the preferred modus operandi, I believe that most individuals are commenting purely on a performance basis as determined by their own environments and needs.

    As is the case with most questions concerning I/T infrastructure, I believe that several other factors much be fully considered prior to making an educated decision...

    Funding/Resources - As with any project, two of the utmost concerns should be the cost and resourcing of any proposed solution. In your case, budget sounds to be a major motivation in going with the SAN technology. As well, if you currently have a SAN in place, one can only assume (hope?) that you currently have an appropriate support structure in place.

    Goal - Again, any project must have a well defined goal to ever be successful. Under the described circumstances, I would equate this to the purpose of your proposed Intranet. The intended uses and content served via your intranet could heavily way on the decision of which technology to choose. If you will be serving simple information web pages a SAN is more than capable and an obvious choice. Alternatively, if your goal is to provide web-based CBT's, web-enabled applications, employee web-portals, etc... then a dedicated server(s) may be the appropriate path to venture down.

    Scalability - Scalability is an obvious strong suit of a SAN. A SAN invasively allows for the transparent addition of processing power, memory, and not least of disk space. If you expect that the intranet will grow quick or expect it to become large, a SAN is the optimal option.

    These are but a few of the key concerns that I would suggest that you further examine prior to making any type of commitment - financial or otherwise. Additionally, it is always a good idea to do your homework and investigate what other companies of similar size and market are choosing as their solution. Best of luck!
  • "Noxx asks: "My department is purchasing several new servers for an intranet website project.""

    You are paying for a solution from the vendor, I hope, not jsut the barebones hardware.

    Your vendor should be supplying a SOLUTION to your storage needs, not jsut dumping hardware on your loading dock and expecting you to do the rest. Get some people who have experience with SAN/NAS to give you an opinion, and get a 2nd or 3rd opinion if you have to. But you should pay for a solution, not for a DIY homebrew "Some assembly required" one size fits all package.

    Your vendor should atleast have a consultant/rep give you an idea of what they think your needs are, and give you a quote you can understand and see is value for money. Otherwise get a new vendor.
    • and what if they want to purchase the hardware only? It's actually cheaper to have in-house IT people that can put it together who the company is already paying rather than hiring some consultant firm to do it for you.

  • NAS is better for many things (Score:3, Interesting)

    by smoon ( 16873 ) on Tuesday January 29, 2002 @08:16AM (#2918760) Homepage
    NAS, or "Network attached Storage" is often better for maintaing large collections of data to be accessed by multiple computers. You can simulate NAS by exporting some filesystems via NFS (Unix) or CIFS (Windows). Network Appliance "Filers" are said to be very good. On the lower end are the Maxtor MaxAttach and Quantum Snap! devices.

    The big advantage to NAS is that dozens of web servers can mount the NAS volume and all serve up the same content. Developers, Administrators, etc. can also mount the NAS volume and do updates etc. Compared to a SAN and buying a fibre channel card, cabling, switch ports, etc. for anything but non-essential components gets very expensive very quickly. Although a previous poster indicated that multiple computers can mount the same SAN volume, It's much more difficult than with NAS since you're essentially operating at the same level as a SCSI bus, wheras with NAS you're operating via TCP/IP.

    A Fibre Channel SAN is good for multiple computers running I/O intensive processes, e.g. a SQL database. It's also good as a foundation for clusters since (usually) LUNs can be re-mapped w/out a reboot. SANs really shine for fully redundant storage as well -- multiple loops, switches, controllers, etc.

    Many products in both categories suffer in support for backup -- the typical low-end devices require you to mount the data on a server then use a server-attached tape device. Some products feature built-in tape drives or offer ways to back up the entire storage unit to a fibre channel attached tape drive, however this option tends to get very expensive very quickly.

    One major bonus in the backup arena is the "snapshot" feature many products have (SAN or NAS). This lets you freeze 'the drive' so that no updates happen to the drive for your backup, but the system still stays up and allows updates. See vendor propaganda for more details.
    • The big advantage to NAS is that dozens of web servers can mount the NAS volume and all serve up the same content.

      Being able to use of-the-shelf shared filesystems (NFS, SMB) is one big advantage when you introduce this into an environment, form the admin point of view.

      However, shared filesystems can have vastly different performance characteristics compared to local, non-shared filesystems, and NFS is well known to be hard to get to grips with in terms of both performance and consistency across a cluster. There a quite a number of scenarios where NFS can be used very effectivly, and everything I hear about Filers is very good, but one should be generally cautionary when jumping into these klnds of setups.

      As to the original question: if your organization already maintains a proper SAN, then by all means use it. Most likely, they can help you with backup and recovery as well, which in itself can be a major hassle.

  • Little to add that hasn't already been said, but...

    We are having an excellent experience with our SAN. It's fiber (fabric logon, not arbitrated loop), runs through Brocade Silkworms (dual, actually, for failover) to HP XP256 chassis (actually made by Hitachi and a competitor to EMC's almost as good Symmetrix). We run HP-UX and NT boxes from shared storage.

    We don't, however, cluster so no one is accessing files from the same place. If we were, the locking would be handled by the application (notneeded for serving read-only html files, for example). But, your mileage may vary.

    We also have a significant investment in SCSI-attached RAID arrays. My unscientific observation leads me to say the SAN access is generally faster along with much less wasted storage. It is more expensive, though, and can take quite a bit of fiddling to get the fabric logons to work.
  • silly name....awesome hardware

    www.netapp.com
  • I've been mucking around with SAN's for a couple of years. Since it has become a buzzword here we have even used them in places where they made no sense at all simply because higher ups thought they were the way to go. The drawback I see to SAN's are that they are expensive, complex, and the underlying technology is still evolving. On the other hand we have achieved far better performance out of our big SAN storage arrays than we have ever got from local disks. Also, for most things I deal with RAID5 is evil. We mirror and stripe almost everything. We only use RAID5 where the volume is read mostly (RAID5 has poor write performance) and static. We have had large RAID5 volumes die due to multiple drive failures; this sort of thing doesn't happen every day but it is a nightmare when it does. My motto is that disks are cheap, data are expensive.
  • As said by other people before:

    NAS = easy to manage (compared to SAN)
    SAN = high performing (compared to NAS,
    but complex.

    You might also consider to purchase a fibrechannel RAID chassis and attach it directly to your server. This allows you to bypass a lot of config-trouble. It also saves a lot of money while you get the performance of SAN.
  • Anyone using a SANS system with diskless workstations and booting from the SANS?

    I saw a demo of a Xiotech Magnitude that looked pretty interesting from an architecture standpoint.

    Any machine could become any other machine, and backup was truly centralized

    Anyone know how one could make on like this on the cheap ?
    • Repeat after me: a SAN presents itself to the system like a disk controller - nothing more, nothing less. You typically run SCSI over fiber, so if your host can see the host bus adapter as a bootable device, then you can boot the system from it. The SAN disks look like SCSI disks.

      A local company here has most of their mission-critical Windows servers (is that an oxymoron?) booting off their EMC SAN. If a single system has a hardware failure, they roll in a replacement, boot off the same SAN-based disk, and they're up in no time flat.

  • San (Score:2)

    by mindstrm ( 20013 )
    Just curious. Anyone got an actual definition of a SAN?

    I've seen some pretty different observations.

    Some people think a SAN is just networked storage.... file server, etc.

    The real idea behind a SAN, as far as I thought.. is that it is a 'network' in the sense that different hosts can access the same storage.

    What does this mean?

    SCSI or FC-AL connections to a disk array, generally.. but from more than one host at a time. Plus.. a shared filesystem.

    Why is it a storage area network? because the machines are networked via storage.
    • >Just curious. Anyone got an actual definition of a SAN?
      >[...]Plus.. a shared filesystem

      Please see my earlier article. SANs typically do *NOT* offer a shared filesystem. My EMC-based SAN does not offer a shared filesystem to the majority of the hosts. What it offers is raw disk that look like SCSI disks, and then the hosts are responsible for sharing a filesystem on top of that (via Veritas Clustering, Windows (ughh) clustering, etc).

      NAS, on the other hand, does offer shared filesystems, typically via NFS or CIFS.

Slashdot Top Deals

"No matter where you go, there you are..." -- Buckaroo Banzai

Close