Ask Slashdot: Little Boxes Around the Edge of the Data Center? 320
First time accepted submitter spaceyhackerlady writes "We're looking at some new development, and a big question mark is the little boxes around the edge of the data center — the NTP servers, the monitoring boxes, the stuff that supports and interfaces with the Big Iron that does the real work. The last time I visited a hosting farm I saw shelves of Mac Minis, but that was five years ago. What do people like now for their little support boxes?"
ESXi (Score:3, Interesting)
No little unsupportable boxes here.
Get a real time server. (Score:5, Interesting)
Go get a GPS satellite receiver/time server. Actually, get two. Don't screw with time.
THEN, virtualize the rest of the stuff. Monitoring, syslogging, management, patchers, etc.
We've virtualized everything except for
- a Windows DC so that it stays up if the vmware datastores or SAN eats itself in a horrible way.
- The NIS server we have to use on our UX environment due to an ancient regulation. I'm not willing to put up HP-UX VMs for this right now, otherwise it'd be safe in a VM as well.
- Anything we can't virtualize due to licensing/contract/support issues. So our VOIP environments, phone call recording, access control systems for the doors,
My datacenter is getting a lot nicer to look at, and a lot easier to upgrade. I can shift servers or volumes all over the room so I can do live maintenance during the day.
"Obsolete" hardware (Score:5, Interesting)
Those support tasks don't exactly push hardware to its limit, and most of those tasks are the kind of thing that demands a bunch of redundant servers anyway.
Throw a bunch of "last generation" hardware at the task -- stuff from the "asset reclamation" pile. Leave a few more around as spares. Less disposal paperwork. Works just fine. By the time your last spare fails, you'll have a new generation of obsolete hardware.
amazon (Score:2, Interesting)
For little boxes that deal with DNS, time, etc - put them in amazon. They're critical servers, but don't really need to be at your site. Put the primaries outside, and slaves on the inside. That way if you have an outage you can always repoint DNS to somewhere else...something you can't do if your primary DNS is on a dead network.
Re:VMs (Score:4, Interesting)
There are good reasons to separate functions. Mainly security. That way, if someone hacks the NTP server, they don't get control of DNS, nor do they get control of the corporate NNTP server, or other functions.
The ideal would be to run those functions as VMs on a host filesystem that uses deduplication. That way, the overhead of multiple operating systems is minimized.
What would be nice would be an ARM server platform, combined with ZFS for storing the VM disk images, and a well thought out (and hardened) hypervisor. The result would be a server that can take one rack unit, but can handle all the small stuff (DNS caching, NTP, etc.)
Re:HP Proliant MicroServer N40L (Score:3, Interesting)
It's not rack-mountable. No IPMI either. That should be a deal-breaker for anyplace serious enough to have a rack.
We try to virtualize anything that can be virtualized. But for those few tasks that really need to run on bare metal, we've had good luck with little Atom D525 Supermicro rackmountable boxes. We bought a few complete boxes (minus ram and storage) that Newegg billed as fanless (which was a lie). Those ran hot enough to develope problems after a few months. Ever since we've built ours up from parts (SUPERMICRO CSE-510-200B 1U rackmount server case, SUPERMICRO MBD-X7SPE-HF-D525-O server motherboard, SUPERMICRO MCP-220-00051-0N single 2.5" fixed HDD mounting bracket, GELID Solutions Model CA-PWM 350 mm PWM Y Cable, RAM and storage). About $400 and have been really reliable. Only thing I don't like is that they don't have IPMI on a dedicated port.
But honestly, if there is any virtualization going on, there shouldn't be much need for these.
Re:VMs (Score:4, Interesting)
Well, one of the reasos is that some services get hold of port 80 (or, a few times other ports), and don't want to share it. With virtualization you can share resources with those too... But yes, those services are a minority, and probably won't need a lot of resources...
Another reason is that you may want to give different people permission to administrate different machines... But again, except for companies that sell hosting, that's an exception.
A third reason is that you may want to replicate your environment for backups and testing... Except that you don't need a VM to do that on Linux. You just copy the files, add two devices to /dev and run the bootloader again. It's easier than backing-up a VM in Windows.
And I've never heard about any other reason for virtualization. I can't also think about any other. I'm lost about why sudenly so much people wants it so badly... Ok, all datacenters added specialized machines for decades because of those first two reasons I gave you above, and get some benefit virtualizing them... But the core of a datacenter (the main databases, web servers - the machies that actualy spend the day working) should run on the metal, and altought I've met several people that arguee otherwise, I've never heard any argument for virtualizing them that holds any water.
But now, I think, maybe the HA people should try to virtualize their clusters. They have a huge amount of redundancy, and consolidating several virtual machines in a single real one can help them reduce their costs. (Ok, if you are in doubt, no, I'm not THAT stupid, it's a joke.)
Re:performance? (Score:2, Interesting)
We use two of our Windows domain controllers for our time source. Those 2008 R2 machines are running on a 10 node ESX farm with about 450 other virtual machines. Those two domain controllers provide time services for about 2000 devices in our worldwide network (not just windows machine either, our switches, routers, SAN, etc). We have NEVER had a problem with NTP and synchronization.
NTP is network time protocol. It is designed with random latency in mind. If you are going over a network, there is random latency. That latency inherent to any network is many orders of magnitude higher than any latency a virtual machine sees running on a hypervisor.
Why not hypervisors? (Score:4, Interesting)
Re:performance? (Score:4, Interesting)
I have had best results on bare metal indeed.
I run ntpd on bare metal along with other apps but I run ntpd in a jail (chroot like), just in case. I do reply to public requests but I do not allow queries, ntpdate and other stratum servers requests work fine but you can't ntpq -pn me for example.
From ntp.conf:
restrict default noquery
By the way, I am a maniac but I am still satisfied at +/-5 ms. Please do not close my door to hard so it generates a gust of wind towards my ntp server and make it go above +/- 5ms error margin. Not maniac enough to buy a GPS although...
Re:NTP servers are NOT about consistency (Score:3, Interesting)
NTP servers are NOT about consistency, they are about making badly designed protocols, such as NFS, capable of limping, instead of just falling on their face.
If the requests on these protocols used a client timestamp for the client's idea of the current time, then the server on receiving the request could look at its idea of the current time, and arrive at a delta before it actually did anything other than enqueue the request locally.
Then when the server responded with a non-"now" timestamp in any client response, it could apply this delta to the response value, and as far as the client was concerned, it and the server would have synchronized ideas of "now", without resorting to all of this NTP BS or worrying about clock drift, or anything.
I lobbied very strongly to try to get this fixed in NFSv4; maybe we will get our collective heads out of our butts by NFSv5.
Are you all mad? What does improving NFS have to do with intentionally letting PC clocks drift?
Could I go out on a limb and suggest there are reasons besides NFS to keep clocks in sync? Wow.
Re:VMs (Score:2, Interesting)
Well modern hypervisors like VMWare allow you to prioritize virtual machines so that they get a higher share of scheduling time in an overcomittment scenario. Assign your ntpd server a high priority so that it doesn't have to wait in a long queue to get run time.
Yes running time-sensitive stuff on a hypervisor is tricky but not at all impossible. It's not stupid unless you don't know what you're doing.
Re:SOLVED: Little Boxes (Score:3, Interesting)
Re:SOLVED: Little Boxes (Score:2, Interesting)
I tend to plan for the worst case scenario, which is a restart from a dark data center. Given that a hurricane just passed awfully close by one of them, that seems like a valid assumption for me to make.
I have a couple of thousand physical servers. They very much need to sync their hardware clocks via. NTP. I need reliable NTP servers. NTP running on a virtual host is not reliable (the clock drifts horribly, although ESX5i is better in this regard).