Designing a Municipal Wireless Service?

EvilTwinSkippy asks: "I am on a team generating a proposal for the Wireless Philadelphia Initiative. In short I have to figure out how to cover 135 square miles of city with Wifi. I'm reading through the requirements. (Not linking to them, no fair slashdotting the customer, or my employer.) I have already figured out that supporting Wireless B and G simultaneously has to go. As does supporting cars traveling at 60mph. And getting 1MB sustained across the network is a pipedream. In the end, I'm looking down the barrel of designing a network this is projected to have 160,000 users in 5 years, over at least 3000 nodes. I know that Rooftop mesh networks are going to be a large part of the design, as will Linux boxes acting as routers and access points. What massive network issues has 4 years of electrical engineering, and 10 years of hacking routers and servers not prepared me for?"

Re:3000 unattended servers?

[rednip]

the real problem isn't just the access, it's the handoff, and coordination with other nearby access points. Cellular networks do this already. But what happens when you take a trip down the street with your little handheld PC. You need to get another IP address, those dumb access points won't coordinate with each other to provide service. I believe that is what each of those servers will be intended to do. Sure you could set the DHCP to some very low timeout, but somehow I don't think that would be a great idea. Eventually manufacturers will catch up and build specialized equipment to solve the issues, but I don't believe that any of them have so far (I could be wrong...).

What I think that the submitter is trying to do is to find strategies to minimize 'network churn'. I have a couple of ideas on what is needed, but my lunch hour is almost up and I need to get back (my company block my attempt to post to slashdot a couple of weeks ago and I have been avoiding it at work ever since.)

What you need ot be prepared for

[Anonymous Coward]

What massive network issues has 4 years of electrical engineering, and 10 years of hacking routers and servers not prepared me for?"

In Philadelphia: union thuggery, municipal corruption, and pay to play.

Is this for real?

[snorklewacker]

Are the good citizens of PA shelling out tax dollars to fund a setup of someone who has to Ask Slashdot how to set up a municipal wi-fi network?

3000 full-blown linux servers? Jiminy Christmas. Probably COTS PC hardware, right? Please tell me there are competing bids from experienced networking outfits?

Warning Will Robinson!!!

[moorley]

Just some thoughts from days from working at an ISP.

Know your scope, technically you are setting up WiFi but you need to forget about the technology for a moment and have AT LEAST a prioritized list of what this network is to be used for. Without that guiding light it will do what it does, but it may not do what anybody (or perhaps a particular high ranking somebody) will want it to do. You won't have anything to guide your decisions or your priorities.

Second, LATENCY!
I haven't played with WiFi meshes so this may not come in to play but from past experience with Wireless solutions ala ISP you have to remember that cabling and bandwidth is VERY IMPORTANT. Donot be tempted to use wireless repeaters with abandon. You need to be able to have a greater amount of backbone, node to node bandwidth than the nodes themselves will provide. If the wireless nodes get overloaded and TCP retransmissions (or retransmissions by the WiFi repeaters themselves) will climb and there will be a point no packets will move. The latency of WiFi will cause this packet storm (if you will) way quicker than wired solutions. Without a good amount of bandwidth behind the nodes, or even a backup landline for administration bringing it back could be quite a pain.

The ISP I worked for tried to deploy point to point wireless bypassing the telco. Rather than run cable to the tower, they used point to point to the tower and then point to point to their customer. It didn't take long (since all of the point to point links were rated the same) for the whole solution to get snarled up bottlenecked on the point to point between the ISP and the tower. With the latency of wireless it would be unusable REALLY quick. (If only we had SQUID and bandwidth limiters back then... SIGH...)

Lastly, you have the greatest opportunity to win through control. Your watchwords should be metrics and design. As you roll out your nodes you best be pulling metrics so you know how your design will handle load and how it will fail. This will be knowledge that is good as gold, and will allow you to re-design and re-deploy. Your first attempt will be a guess but if you capture the metrics and track as most information as you can, whether that be the temperature of the wireless nodes (do they overheat, are they sheltered, is there a pattern to failure) or the packet retransmissions; all of that information will be vital to learn how to tune it up, engineer it and deploy it.

Have fun... I'm envious...