Ask Slashdot: 802.11n Bake-Off Test Plans? 125
First time accepted submitter Richard_13 writes "I am seeking a bake-off test plan for an enterprise size deployment of 802.11n wireless. We are about to go to tender for a large scale deployment of 802.11n controllers and APs — and I need a bake-off (benchmarking) test plan that is focused on testing the *maximum number* of clients that an AP can handle before it falls over, in addition to the throughput for each client. We intend to test the latest products from the major vendors, Aruba, Cisco, HP, Xirrus, Ruckus, etc.; not consumer products like Linksys, D-Link or Netgear. Any bake-off test plans or useful links to multi-vendor wireless focused web sites would be greatly appreciated."
You need to say Bake-off (Score:1, Funny)
one more time~
Re: (Score:1)
420 4eva bra! let's get baked!
Bake sale (Score:5, Interesting)
Maybe you could hold a bake sale and offer free wifi for all your customers. Just give them places to sit.
Re: (Score:2, Insightful)
Paraphrased from Pulp Fiction:
[brandishing gun]: "Say 'bake-off' again..Say 'bake-off' again! I dare ya..I double-dare ya, mothafucka! Say 'bake-off' one more god-damn time!!"
Re: (Score:2)
Re: (Score:2)
Give Richard_13 a break. He didn't use solution or cloud in his request. ie. I'm open to cloud solutions in the bake-off.
"Bake-off" (Score:5, Insightful)
Just say "benchmarking" and you won't need to explain that "bake-off" means "benchmarking". Wait, what was the question?
Re: (Score:3)
Re: (Score:1)
I am allergic to peanuts you insensitive clod!
Re: (Score:2)
A bake-off and a benchmark aren't exactly the same things. Price, ease of configuration and deployment, general vendor responsiveness, and other things may come into play, besides the raw performance numbers.
It's a term the sales droids and CTOs seem to use a lot; not so much other people.
Re: (Score:3, Insightful)
If I hear a CTO say "bake off" instead of "trade study," I'm talking to the CEO about making the CTO a CUO (completely unemployed oaf).
Re:"Bake-off" (Score:5, Insightful)
The correct meaning of bake-off is when many different makes of a device that are supposed to support a standard are paired off to see if they actually inter-operate.
As usual, the sales droids and CTOs (that didn't come up through the ranks) abuse the term to try to sound like engineers.
Re: (Score:1)
The correct meaning of bake-off is when many different makes of a device that are supposed to support a standard are paired off to see if they actually inter-operate.
Nah, that's called a "hoedown".
Once you've identified the interoperable parties, you can then invite them to a "barn raising".
Remember to provide lots of lemonade.
Re: (Score:2)
Re: (Score:1)
Except that a bake-off and benchmarking are not at all the same thing, but can happen in the same event.
Re: (Score:2)
I thought I was the naive guy who didn't know and didn't want to say anything.
What the hell does bake off mean? I know he explained it but that's not good enough, who the hell calls it a bakeoff?
Re: (Score:1)
who the hell calls it a bakeoff?
The Amish, mostly.
But they typically shun computers in general, let alone 802.11n deployments.
It's possible that the OP is an Amish youth in his "rumspringa" wilding phase. His clueless use of buzzwords (eg "enterprise size") supports this theory.
On the other hand, he might just be another brainless corporate shmuck who is way out of his depth in a technical decision-making role. A prime candidate for immediate firing.
Make the vendors work for you (Score:5, Informative)
Call a meeting of the competitors engineers. It's important you get them all in the room at one time with not too much advance warning of the topic.
Tell them what you think you want. Ask them as a group what you're missing. Then make them as a group come up with an eval plan and cook 'em off according to the plan they come up with.
If you need an independent judge, go to one of the labs that does independent third-party assurance and contract them to provide oversight.
Disclaimer: I've worked for one of those labs for the past 15 years.
Stand back and watch the fun......
Red
Re: (Score:2)
Yes. Absolutely. You need to tell them what you want, and it's their job to prove to you that they are capable of providing it.
If the gear falls over, then you can say 'we're not paying for it, because it has to meet this specification before we do, and it obviously isn't', rather than 'we tested it ourselves, and it should be working'. You want to reduce the risk that this project isn't going to meet expectations.
Re: (Score:3)
Sounds like a Texas Faraday Cage Death Match.
Re: (Score:1)
Re: (Score:1)
There's no "load balancing" at all built into wi-fi. Your laptop will latch on to the first system in its list, if it can. Even if the construction of the list is fully randomized every time the laptop is told to find an AP, the number that find the same AP first is also a random number, and can vary from 0 to All. And if the SSID broadcast from them isn't coordinated to split the interval between broadcasts equally, then one of them is more likely to be in the top of the list every time, because it come
Re:Cisco... (Score:5, Informative)
See here [perihel.at]
Re: (Score:3)
Load balancing, automatic power level adjustment to avoid excessive overlaps or voids, triangulation of clients and nearby access points, and various other stuff that can be quite handy; but m
Re: (Score:2)
Man, if I could have one that does adaptive beam-shaping...
Re: (Score:2)
If you know how to write firmware, you could probably use the MIMO access points to do some phased-array fun. Write the code to determine the approximate direction of the client (which, you should be able to do with multiple antennas for triangulation), and then increase the power using multiple antennas in said direction.
*goes to do some Googling*
It looks like there already is an access point that does this:
http://www.tomshardware.com/reviews/beamforming-wifi-ruckus,2390-3.html [tomshardware.com]
The incredible thing is that chip-based beamforming, like MIMO, has been compatible with 802.11a/b/g for years. In fact, the technology is an optional part of the 802.11n standard. Despite its benefits, though, Cisco is the first to deliver on-chip beamforming to market. The enterprise-oriented AIR-LAP1142N access point is Cisco’s first and so far only product to feature beamforming, which it brands as ClientLink. It arrived in the first quarter of 2009, but the firmware that enables beamforming capability didn’t arrive until July. We tested with this firmware literally within days of its release.
Re: (Score:2)
My company makes some wifi testing gear. (Score:5, Informative)
We have a box that can emulate up to 128 stations, including wpa, wpa2, etc. It can do
DHCP or static IPs and generate Ethernet, udp, tcp, http, and other higher level protocols,
including IPv6. Multiple systems can be clustered together for additional throughput and
radios. Each system can run on only one channel at once, but can talk to multiple APs
on that channel.
One of our systems can saturate any of the consumer grade APs we have, and some folks have
used it to stress very big systems (conference centers, etc).
Runs on Linux of course!
http://www.candelatech.com/ct520-128_product.php
Thanks,
Ben Greear
Re: (Score:1)
Re: (Score:2)
.. I should really get me one of these for FreeBSD 11n testing. :)
Re: (Score:3)
But how do we know that your testing gear is the best. Maybe we should have a "bake-off" between you and another test gear vendor...
are you the cluster guy? (Score:2)
Unless you're going to deploy like 100 APs or more i an skeptical that the vendors will work with you for such an effort.
Actually doing this correctly is going to be hard and expensive. Anyway, i'd read up on smalnetbuilder's methods and just run, say 10 or 20 concurrent client machines o a 3 or 4 AP set-up. make some of those clients mobile and walk around the space to see that hand-offs happen ok.
graph it all and look for major priods of drop-out etc. Again, though, unless you're doing a massive de
Re: (Score:3)
Lmao..he said he needs to test a big deployment with non-consumer grade gear. Your approach outlines the least of his worries.
What is the building(s) architecture, power and existing cable plant like? (concrete and steel, stick; adequate/sub adequate/surplus power; CAT-3, CAT-5, CAT-5e, CAT-6, fiber, thin-net, coaxial) Access points need power and users need to be able to connect to something worth connecting to wirelessly
How is he doing authentication? (802.1x with cert?, challenge response?)
Is it again
Re: (Score:2)
Doesn't sound like you're familiar with Xirrus which was one of the manufacturers in the list. Their product is an array of access points, up to 64 APs in a UFO shaped container with controllers and everything built in, just give it a network connection consistent with the performance level you're expecting and go, it even has built in radius.
I only priced out their 16 AP option but it comes out to about a grand an AP so it's quite affordable for enterprise offerings that need lots of connections in a smal
Re: (Score:1)
"The Pillsbury Bake-Off is a cooking contest, first run by Pillsbury Company from 1949 to 1976 as an annual contest. Since then, the contest has been held biennially."
https://secure.wikimedia.org/wikipedia/en/wiki/Pillsbury_Bake-Off [wikimedia.org]
Re: (Score:2)
Can I has? (Score:1, Funny)
Radio power (Score:1)
If you're running a bake-off with access points, you're probably running just a bit too much power into the radios.
On-topic, I do like my redundant-controller, centrally-managed 160+ AP Aruba system.
No consumer or no small biz? (Score:1)
Im not sure what you mean by no consumer stuff.but netgear has started offering small biz gear.including wifi setups supporting up to 150 APs. So not sure how big you thing big is, but they are one of the few mid size deployment shops. While my prior opinion of netgear was low they seem to be trying to break into the enterprise markets. Of course if you need mor APs than that, they are still too small.
VeriWave in Portland (Score:3)
The (not so big) secret is that most WiFi AP rolls over with 8 or so clients. Only a few manufacturers themselves test their products beyond that, and those work all the way to over 100.
The company selling the test equipment you need is called http://veriwave.com./ [veriwave.com.] You can buy the equipment from them and test all the vendors, or even better, just ask them.
They do of course know, since that is how they test their own test equipment. Problem is that they can/will not tell you because then 1. you would not need to buy their product, and 2. AP mfg would fix their products, and Veriwave would not have a market for their products.
Maybe just do some social hacking to get it out of them.
Re: (Score:3, Informative)
There have been large tests, and it can be done by using Linux boxen that allow one to change out the raw socket to emulate numerous concurrent IP/MAC address pairs concurrently.
Then you decide what kind of duty cycle of transactions will be typical. All surfers checking Facebook? Or are their apps with sockets?
The biggest part of this is the backhaul; what's behind the AP in terms of next hop to a thick layer 2/3 switch/router. How skilled is the person that programmed it?
Are you going to use bi-freq N? If
Re: (Score:2)
Aruba Rocks (Score:2)
They may not be selling the cheap one anymore, which would be good, because it was vastly inadequate for any real N-style usage.
Don't forget about Juniper/Trapeze (Score:2)
Don't forget about Juniper's new wireless solutions, from their Trapeze acquisition.
I've heard a lot of good things about Aruba and Xirrus.
Having actually done Cisco wireless support and new deployments, I would highly recommend against Cisco. They call it a "Cisco caveat" for a reason. Sure that feature works... you know... under the right conditions which will never be met.
don't forget IPv6 capability (Score:2)
I looked at equipment recently for wireless using the 'n' protocl - but noticed no mention of IPv\6.
No point of gertting new communications equipment, if it cannot be suicessfully usec with IPv6!
So make IPv6 part of the requirments.
Re: (Score:1)
Hint: don't use the access point he's using, it can lead to strange character corruption.
Re: (Score:1)
You insensitive clod! His local wireless connection works fine, but he's posting from Mars! ;-)
Re: (Score:2)
Which is why we shouldn't be looking for life on Mars. We should be looking for open wireless access points.
Re: (Score:1)
I completely support this - I want to see where each one is w/ IPv6 support. If something can't support IPv6 both in & out, it's worthless. Has to be part of the requirements.
One advantage of supporting IPv6 is that router vendors can then have a range of n routers supporting IPv6 for different ends of the market - @ one end for the home user, just 1 SSID, and @ the other end, support up to 4 SSIDs if possible (beyond that, I'm thinking they'd have to go wired to avoid too much congestion here). In
Been there done that... (Score:2, Interesting)
Here are the results:
Vendors were tested with 30 then 60 wireless clients and 1 then 2 access points. So, 1 AP with 30 clients, 1 AP with 60 clients, 2 APs with 30 clients and 2 APs with 60 clients.
1: Cisco - Somewhat surprising. Great client density/bandwidth. Good load balancing between APs. Good management interface.
2: Trapeze (now Juniper) - Great client density/bandwidth (just a little slower (read less bandwidth to client, and just slightly less) than Cisco). Good load balancing between APs. Buy
Something to consider: (Score:5, Interesting)
Dicking around at home and the wifi cuts out for a second? Reload the webpage and quit your whining.
Running your basic "enterprise" client configuration(documents directory is actually on a fileserver, authentication through AD, etc, etc.) and the wifi cuts out? Be prepared for frustratingly erratic appearances of apparently disappearing documents, authentication fails, not automatically reconnecting to the fileserver, Finder just twiddling its thumbs and thinking about infinity until that server either times out or comes back, etc, etc.
Even before any APs show up, you can start identifying the likely areas of sheer pain by using netem, switch jiggery-pokery, or just a $20 consumer AP and flicking your laptop's RF power switch: If your environment has client applications that don't play nicely if the network goes all to suck for a second or two from time to time, wifi deployment is going to be Fun.
Honestly, for most applications where wifi isn't a totally terrible idea(ie. heavily throughput dependent stuff), that would be the big focus of my testing(along with how useful the management tools and interfaces are). High throughput is far less valuable than stable connections.
only the ELA matters anyway (Score:2)
The only thing that matters is the bean counters and any ELA you have with existing vendors. Cisco might be good, or it might be crap, but if you have a pimp contract with them and good support, they're getting the contract. Live with it.
Ask the vendors about bufferbloat too (Score:3)
All the major vendors should be aware of what is going on at www.bufferbloat.net and have something in place to ensure that their products will reflect new updates soonest when things get fixed. This is an ongoing problem that crept up on the internet tech community and there is work in progress to deal with it but it will take time.
See (for example) Bufferbloat - Dark Buffers in the Internet, 1/20/2011 [bufferbloat.net]
What's your budget? (Score:2)
Plenty of companies out there make tools for testing this sort of thing. Spirent, Ixxia, and Agilent, to name a few all have layer 4-7 traffic generation appliance type products for stress testing.
http://www.spirent.com/Devices-and-Equipment/Base_station_testing.aspx [spirent.com]
Re: (Score:2)
Here's a more appropriate link http://www.spirent.com/Solutions-Directory/Avalanche.aspx [spirent.com]
Though I would certainly research their competitors products as well before making any purchases, but this is the type of equipment that ISPs and large banks might use to test their infrastructure.
Re: (Score:1)
enterprise (Score:2)
Set some params. (Score:1)
What does enterprise size mean?
What class of construction is the
building? Straw, sticks, bricks... this wolf wants to know
Are regions RF isolated from each other
do you have multiple floors and multiple
buildings? Can RF pass between these
odds and ends.
First you need to wire the building and also decide if
your WiFi boxes will get power from the ethernet wire
links or from the wall.
Do you need secure access for all or is this install
wide open and plan to let VPN do the security.
i.e. guests will always want W
Wi-fi Testing and Benchmarking on Tom's (Score:1)
Re: Wi-fi Testing - MOD PARENT UP (Score:1)
Aruba (Score:3)
We run a thousand or so Aruba 125s at school here, covering all 600 or so acres of campus. Those are probably overkill for you (at about $750 a pop), but AFAIK even the lowest-end ones have the same essential features.
Basically, the network architecture puts the whole wireless network on a separate segment, all the way back to the aggregation points. They're gigabit wired into the building routers, but placed on a separate VLAN all the way back to one of the three aggregation points. Each AP is assigned to a controller, and will fail over to a second one if needed. The controllers pass the traffic to the rest of the network.
The controller architecture means you can do some pretty interesting things. Particularly, it means new APs are trivial to install - stick them into the controller's DB and plug it in to an Ethernet cable (it's PoE); it'll go and find the controller and pull down the config and any upgrades to the software. It also allows IP roaming between APs, even if they're in different netblocks. I can walk from one end of campus to the other (7 city blocks) while keeping the same IP and getting all my traffic, through about 150 different APs - much like a cell network. You can also do spectrum analysis through their management console - I once saw them find a broken microwave from all the interference they were seeing across the 10 APs in range a la Dark Knight.
The APs we have will band-steer clients over to the 5GHz spectrum if possible, which can support a huge number of clients, but you need the density for it to make a big difference. If you do, though, you can easily get 30 people per AP, with a few doing massive downloads/uploads and no hiccups. They don't recommend more than that, and in any case it's difficult to fit people densely enough that you wouldn't need a new one for signal purposes.
No, I don't work for them. I don't even work for the Networking department. I just really like the toys - though I suspect I might feel differently if I had to make the purchase! Quality isn't cheap...
PWNED (Score:2)
Re: (Score:2)
Re: (Score:2)
I knew a guy who did that once. He spoofed one of their Internet gateways with his Linux box. Two minutes later, they shut the port down because it threw up a flag on their NOC screen in 30 seconds, and it took them another 90 to find the port on his nearest switch. Then he got an email suggesting that he not do that again (they knew it was intentional) and reminding him of their TOS, and that they'd turn it back on after he acknowledged that he'd read and understood it. Total downtime was a few minutes, fo
Re: (Score:2)
Because it is. Here on the network where I am, moving from one AP to another (they're Cisco, but on the same controller - admittedly the configuration of the two isn't quite the same as we have fiddled with it to fix a problem with Apple's shitty 802.11 implementation, but I digress) is guaranteed to result in your device disconnecting - handover is very manual and there's a chance the device will fail to re-associate entirely.
We've done this bake off (Score:1)
The result was Cisco but this was 3 years ago. The Aruba gear is poorly made and it took about 70% more APs for the same coverage. We used 900 Cisco APs as opposed to over 1500 Aruba. That being said we also like the Ruckus Gear which also feels flimsy compared to the Cisco 1042 APs but the beamflex technology is second to none especially if your trying to carry a voice or video signal.
Meru Networks (Score:2)
Thank you to all who replied with useful info... (Score:1)
Great comparison on Tom's hardware (Score:1)
Another option (Score:2)
Why Use Large Vendor? (Score:1)
herve leger dress (Score:1)