UPS Setup For a Small/Mid-Size Company? 260
An anonymous reader writes "We're a small company employing ~30 people and we are becoming increasingly reliant on virtual servers. Unfortunately, the hosts they are on don't have redundant power supplies because we simply don't have the capacity. We currently have one UPS per rack, which gives us about two minutes. This may have been enough time when they were put in — they've been there for some time — but it isn't really enough time to shut everything down in the event of a failure. Domain Controllers alone may take up to 15 minutes. So I'm looking at upgrading the UPSs to ones that would preferably give us around 15 minutes of breathing space and send an email or text alert when a failure is detected. Something that could trigger shutdowns automatically would also be nice. Of course cost is a key factor too. so given all of the above, what does Slashdot recommend?"
Network UPS Tools (Score:2, Interesting)
The open source world has NUT [networkupstools.org] to offer (Network UPS Tools).
We've been using it at work for all our critical servers. It works with pretty much all UPSes, and on pretty much any production OS, so you can use your existing servers and just buy whatever hardware the budget affords.
The linux/unix servers and clients are excellent, and there is a reasonable Windows port for the client (which we've modified a little to suit our needs).
The cost is just your sysadmin's time, as with all F/OSS solutions.
Re:Generator (Score:3, Interesting)
Out company got to the point of almost bribing building management to let us put a generator on the roof for our use, since the cell companies already had diesel generators up there to power their cell phone antenna equipment. They still wouldn't let us though.
Larger UPS (Score:3, Interesting)
It sounds like you may have outgrown the traditional "UPS". They're fine and dandy as long as you're only powering so much equipment. There are some huge options (large in physical size, and more so in price).
A decent alternative may be a DC power room, with generator backup.
Basically, you have banks of batteries, with true sine wave power inverters on them. The power coming in goes to charge controllers. Depending on how you set up, these can get pricey too. There are some nice (and expensive) units that handle both the charge controlling and inverting, and will automatically switch between the incoming power and batteries. Look at the higher end Xantrex units, made for on/off grid purposes.
The less expensive way would be to break up your battery banks by power circuit. Say a 15A power circuit per set. Put a dependable inverter on the rack side of the batteries, and a good charge controller on the line side. Separate inverters for each circuit may not seem like the best idea, and the overall efficiency will hurt because of it, but an inverter failure will only mean one circuit goes down, not the whole place. It's affordable to keep a few spare $300 inverters on hand, where it's harder to ask for a few spare $3,000 inverters.
You'll also want an automatic crossover, if your line power should fail, you can bring up a generator. The batteries shouldn't be intended to last for hours. They should only last as long as it takes to bring up the generator (say 1 minute). Expect that there may be generator problems though. In a prolonged outage, you may need to shut down the generator to refuel, so the batteries may need to last for hours. At very least, if your generator fails, and line power doesn't come back up, you have that hour to gracefully shut down your equipment.
Such a setup can be made to make your company more "green" too. Are you in a situation where you could put a large array of solar panels on the roof, and have enough battery power to last you through the night and then some? You could bring your power bill down to almost nil, or possibly feed back to the power grid (with the appropriate permission and power meter), and make a little money in the process. The long term savings may warrant a raise for you. :)
There are plenty of consultants that can evaluate your needs, and provide the appropriate solutions. As you talk to various consultants, several will say the others are giving you bad advice. Look at all of them, and research them for yourself before making a decision. Remember too, it's in *their* best interest to sell you the most expensive units possible, while you probably want the most reliable and cost effective.
Re:A second site (Score:4, Interesting)
There's an awful lot to be said for redundancy. I think he's talking in-house applications, but I'm not positive.
One company I worked for, we maintained equipment in multiple datacenters, that were fully redundant. Normally, we served from all of them (no warm-standby sites). Over the years, we'd lose datacenters for various reasons. Sometimes it was power. Sometimes it was connectivity. Sometimes it was simple things, like our own hardware died. We've all seen where portions of the Internet can't reach other portions. Such redundancy will save you. It's better to have the reputation of "they just always work", rather than "they're down every time there's a problem in [insert area]".
Most users won't say "thank you", but they'll be more than happy to complain when you're down. If you have such a presence, you're probably making money on it, so an hour of downtime can easily cost more than the cost of a couple redundant datacenters. With say 3 datacenters, I always made sure we had capacity at each datacenter, in case we had two sites fail simultaneously. While it seems like an almost unheard of event, we did have it happen a couple times in a decade. The providers will apologize profusely, but that doesn't make up for the money lost during the outage.
Re:Generator (Score:5, Interesting)
Re:Cost of UPS vs Colo? (Score:5, Interesting)
I think he was talking about in-house servers, but I could be mistaken. it's good to be in a *GOOD* datacenter that has the proper redundancy. Most of the good ones have multiple generators and tens of thousands of gallons of fuel stored. They can stay running indefinitely, assuming they can get fuel supplied before they ran out.
I did work in one good one. They had a DC powerplant to supply at least 24 hours of power. They also had two diesel turbine generators, and something like 10,000 gallons of fuel, which would provide power for 7 days. In talking to the senior techs who had been there an awful long time, they said the generators had kicked on quite a few times. Only once in about 20 years had they needed to refuel. It got touchy. The power was out for about 14 days. It took 6 days to get a refueling truck in, because it was a nasty blizzard, and all the roads had been closed for days. They were starting to notify the customers of a potential power outage, when two fuel trucks finally arrived. One refilled their tank, and the second was left parked there, in case power wasn't restored in time.
That was a huge facility, and they had the power to say "bring us trucks now", and not be put off for larger customers.
I wasn't impressed by the advertised specs of the site. They were good, but it's easy to lie about the specs. I *was* impressed by the site, when I walked through, and was allowed (with an escort) to see their primary data room (many OC192's), the DC power room, and generators. I wasn't getting the sales tour. I was getting the tech tour, because the senior guys wanted to tell me all about their stuff, and we had a chance to talk about all of it.
I've been to many datacenters over the years, and many have failed to be as good as their advertising made them sound. N+1 generators can be a few 11Kw generators out by their dumpsters, or massive industrial generators. Maybe they test them once a year, or once a week. Maybe they work, maybe they don't. It's less than impressive to see the generators sitting outside, covered in rust, and looking like they were purchased 2nd hand and hadn't been maintained since 1950.
At one site (again, an impressive site), they had an absolutely huge DC room, and I was there a couple times when the received phone calls to turn on their generators because the power company needed the extra capacity. A couple 1Mw generators may make the difference between constant power, and widespread brownouts.
The impressive datacenters were way beyond anything I could possibly talk my management into doing in-house.
Eaton is excellent (Score:2, Interesting)
Re:stationary bikes with alternator (Score:2, Interesting)
I second this. Get small consumer sized UPS units that can keep a server up for a few minutes. Then get a backup generator with an automatic transfer switch to keep the whole building alive until power returns.
When sizing your generator, don't forget to include ancillary equipment such as routers and emergency lighting if you decide not to wire up the generator for the entire building. Otherwise, sizing should be easy. Just watch your watt-hour meter during a normal business day and add 25% or so to that as a safety margin. That will be the capacity you're looking for.
Before you go contacting a vendor who will undoubtedly quote you an absurd price, do some legwork and look at the used market for diesel backup generators. The market right now is pretty good for them. Once you have the generator, you can price out the UPS's you'll need and the transfer switch(if the generator doesn't come with one). The cost to have an electrician wire it up should be pretty reasonable given it's a relatively quick job.
The initial cost to this type of setup is likely higher than equivalent battery based UPS units, but over the long term is significantly cheaper than any large UPS setup.
Re:Generator (Score:1, Interesting)
I've done this at my house. My wiring closet contains servers, satellite receivers, whole-house audio, power supply for the plasma display in the den and a split-unit air conditioner. I put the UPS in myself in a day and it supplies about 7-8 hours in the event of a failure with everything on. The backup generator operates on natural gas and supplies the UPS and the A/C during power failure and was installed by an electrician in under a 1/2 day and automatically starts. The generator also self-tests once a week. It's not complex and doesn't require a lot of work. A reasonably competent person could coordinate and manage the configuration without too much difficulty.
Colocation (Score:3, Interesting)
With the current availability of fairly inexpensive bandwidth, why are you running servers at your location? There simply isn't much justification for any business not in the fortune 500 to go the route of "build your own" Catacenter. If it must be up, look at the option of renting rack space from a Telecom provider that takes care of generator power for you. Most of these will do a rack for a couple hundred a month that includes the generator backup. You may need to get a small UPS that handles the "blip" until the generator kicks in (they usually tell you that you need a few seconds of UPS), but it sounds like you already have units to put at the bottom of the rack that will handle that. You then have servers that will survive as long as the provider has fuel. Anything else is going to cost you far more. Most likely you can find one that will provide decent bandwidth from your location to theirs and provide you with an Internet connection at the Colo that is less expensive because it doesn't have the local loop to your facility. This probably would offset much of the cost for bandwidth that you will need from your office to your servers at the Colo.
Re:Inverters (Score:3, Interesting)
Re:stationary bikes with alternator (Score:3, Interesting)
Diesel is a liquid too, so why doesn't it break all the time?
We'll leave aside the fact that what gets compressed prior to injection of the fuel and the power stroke is air, which is compressible.
The fact is that the water is denser and sinks to the bottom, which is where where the pipe is - and you can't run a diesel engine on water. Most diesels have a trap or U-bend to catch the water, which you need to bleed off from time to time.
Re:stationary bikes with alternator (Score:3, Interesting)
Be careful sizing a backup generator when a large part of its load will be switching supplies and UPSes. UPSes in particular have _very_ strange load waveforms due to the rectifiers that are used in the charging circuit. The harmonics passed back on the line can cause the generator to 'seek' trying to lock in to 50/60 Hz, which can cause significant damage.
A permanent magnet generator can help, but they're a little more expensive.
http://ecmweb.com/news/electric_ensuring_generator_ups/ [ecmweb.com]
http://www.physicsforums.com/archive/index.php/t-186969.html [physicsforums.com]