Testing and Mapping a Cellular Data Network? 114
bgsneeze writes "In order to resolve an ongoing issue with a vendor, I have been trying to find a way to test different 3G data devices empirically. I would like to be able to chart signal strength, latency, and bandwidth. I would also like to create a map of the coverage area. I have a test 3G card from three different providers. I would like to be able to travel with the setup to several different locations and run tests. What software or techniques would Slashdotters use to test the different devices? Are there any free or open source software packages that will do this?"
Re:hack something in perl (Score:5, Funny)
Good Luck With That (Score:2, Insightful)
Your provider won't care even a Tiny Iota [futurama-madhouse.com.ar] about your results. If you show that their network is fine, they'll ignore you. Even if you prove their network is not fine and all the packets are routed though a Sinclair ZX80 in Clive's basement they'll still ignore you. If you're really lucky, you might get a nice, polite "Fuck you very much, We're not fixing it."
Knowledge is knowing tha
Re:Good Luck With That (Score:5, Insightful)
Re: (Score:2)
However, if you have 3G cards from three different providers, and your company wants to know which one to use to deploy their new fancy device, and they don't actually trust any of those providers (for good reason), it may be a good idea to go out and make sure with real world tests before you jump to one provider or another.
Yeah, you'd think that. Until the owner's son sees a compelling ad on TV and requires you to switch via fiat.
Re: (Score:2)
Re:Good Luck With That (Score:4, Funny)
I'd tell you stories, but my therapist says I'm close to recovery...
Re: (Score:1)
Unless you have the ability to inflict significant monetary impact, they could care less.
Re: (Score:2)
Right! And from the way the original question is posted, I'm thinking he has at least SOME monetary power over the providers...
Re: (Score:2)
Unfortunately true.
I encountered the same issue, was naive enough to figure proving my scenario would mean something to them. (I had a cell phone which had no coverage where I lived - in a large city 0.5 km away from the largest university).
In the end, what did work was going to Visa and making a claim to refund charges on the basis that I was a sold a service and device that didn't provide the services and work anywhere near as advertised. Despite a "blah blah blah we can't guarantee 100% coverage clause
Re: (Score:1)
Use results for public shaming? (Score:2)
I'd mod you Insightful, but if this telco has any competitors, a nice heaping pile of public shame could motivate them to take care of the issue...Post the results of the "cellular wardrive" online and send links to the competitors, ideally to someone in marketing if you can find the contact info. Again if there are competitors, maybe you can get a SIM from them and record data from that device simultaneously so you can compare the signal quality. The company with the best signal quality in the area would b
Re: (Score:2)
Agreed. Wouldn't it be smarter to deploy devices from more than one carrier in a large deployment anyway?
I know the carriers make this difficult with "bulk" deals on service, but for cryin' out loud... if you at least choose TWO vendors, and use both equally -- you'd be able to hand the one guy who lives out in the one coverage area that Vendor A doesn't cover, one of Vendor B's USB sticks and say, "Go try this one."
And if they both don't work, you have a lot more leg to stand on to say, "You just live out
3G (Score:1, Interesting)
Do you mean a 3G SIM for use in mobile devices?
If you mean the USB/PC card 3G adapters for computers, I doubt you'll be able to run them on Linux without using WINE or a VM or whatever, which may interfere with latency readings. Those adapters require you to install software(ATT Connection manager in their case) that's only supported on Windows and Mac, and that software is required to "authenticate" your computer and use the network.
Anec
Re: (Score:2)
Re: (Score:2)
Even without Linux, it should be pretty trivial to whip up an app on Windows to track signal strength. I'd install the drivers and check if any new COM interfaces appear or .NET components for interacting with the software. If so, I bet finding out the current connection status would be pretty easy.
As far as actual throughput, there's a lot of software out there for that - even on Windows. You could also use native Windows logging facilities to track network packet rates, Mbps, whatever you want to.
Re: (Score:1)
Re: (Score:1)
Re:3G (Score:4, Interesting)
If you mean the USB/PC card 3G adapters for computers, I doubt you'll be able to run them on Linux without using WINE or a VM or whatever, which may interfere with latency readings...........Note: too lazy to dig up hacks and testamonials from Google.
Sigh.......this is the kind of misconception that happens when you are too lazy to use Google. Those USB/PC card 3G adapters for computers work quite well on Linux, here is one example tutorial [wordpress.com]. Try not to give advice if you are too lazy to verify it.
Re: (Score:3, Informative)
Dan Williams, the guy behind Network Manager, does a lot of work to get cellular modems working in Linux. There seems to be lots working and steady process on others.
His blog http://blogs.gnome.org/dcbw/ [gnome.org] is informative and frankly pretty hilarious in a geeky way.
Props to Dan for doing a great job.
Re: (Score:2)
As well as Network Manager, I'd recommend umtsmon and wvdial for USB 3G modems - the former tracks your data usage, the latter is just a standard PPP dialler that can be put into scripts.
Re: (Score:2)
Anecdote: I had ATT 3G in the metro San Diego area around 2007. Speeds and coverage were decent enough to browse the web, but too spotty for anything hardcore.
There's nothing wrong with that. Softcore on the bus is cool, but people always bitch at me when I'm watching hardcore.
Protip: Use headphones and sit in the back.
Re: (Score:2, Interesting)
Those adapters require you to install software(ATT Connection manager in their case) that's only supported on Windows and Mac, and that software is required to "authenticate" your computer and use the network.
(Disclaimer - I work for Vodafone, albeit as a lawyer)
I have never tried with one of the PCMIA cards, but, most Huawei modems will work fine under Linux - you might need usb-modeswitch to flip it into modem mode (as opposed to mass storage mode), but, otherwise, it should not be a problem.
If u
Re: (Score:2)
You'd better tell my employer. They think that the USB 3G units they're using for a Linux embedded device work just fine, thanks.
Are you kidding? (Score:2, Funny)
Obviously this can only be used for terrorism.
Funny Story (Score:2)
Back in 2001/2002 I was doing drive testing for US Cellular. One day we were cruising through some random suburb of northern Illinois. My driver that day was an Iranian American. Nice guy, but not the usual bleached white skin tones folks are used to seeing in such a location. Someone called us in to the police as possible terror suspects, driving the neighborhood around with "computer equipment".
So we got pulled over by a fleet of police. Took about 5 seconds for them to realise what we were doing and we w
Don't waste your time (Score:2, Informative)
http://www.cellularmaps.com/coverage_finder.shtml
Re:Don't waste your time (Score:5, Interesting)
These are simply links to the coverage maps advertised by the providers, in which service quality is frequently exaggerated, if not blatantly false.
For example, AT&T (my provider) claims "good" coverage in two neighborhoods in my home town, Pasadena, where I know for a fact that there is no coverage, or worse, sporadic one-bar-then-no-bar coverage that drains my cellphone battery in an hour. And even worse, they show "best" coverage throughout San Marino, a town in which I can never make or receive calls on AT&T.
Re: (Score:2)
The data is outright modelled and cannot take into account a bunch of factors that limit the actual signal strength. I had a somewhat heated discussion with a saleperson who was convinced that their telco had actually sent someone with a meter to measure the reception at 100 metre intervals across some wild and rugged terrain. It would be funny to take some samples and quantiatively demonstrate the exaggeration in each provider's coverage maps, the tabloid media would lap up the right analysis.
Xix.
Re: (Score:2)
Typical of marketing. If one person once managed to make a call, the coverage is "good". If it wasn't dropped in the middle of "hello", it's "excellent".
MyTrueCoverage/Root Mobile (Score:3, Informative)
http://www.mytruecoverage.com/ [mytruecoverage.com] is a good site to show coverage. You can install an app on your phone then manually run tests. The results usually take 12-36 hours to post to the website.
Re: (Score:1)
Multiple ways to do this... (Score:2, Informative)
Re: (Score:2)
I assume that you are not working with a network operator?
It looks like he is trying to independently verify/disprove his cell provider's claims. They are about as likely to send a network engineer to evaluate his situation as BP is likely to send a scientist to evaluate the situation at the bottom of the Mexican Gulf.
Re: (Score:1)
Big Money; Lots of variables (Score:5, Informative)
Empirical testing with wireless CELLULAR networks can be very tricky; A few things that you will need to keep in mind is that for testing, you want to make sure your transceiver setup is perfectly reproducable; Same card, same ANTENNA, same position of antenna. When performing your testing, your signal strength will depend on several factors: Distance from the site, antenna type/gain, and specifically what sector/node on the site you happen to be on. while driving in a straight line, you may find that you approach steep nulls near the border of a cells sector boundary. Alot depends on the ability of your particular card to handle the handoff between sectors/sites.
Your latency measurements will also vary according to the individual usage of the sector/site that you are currently on, and additionally vary with time and also variable bandwidth allocation to the sites from the main switch.
There are quite a few test sets and software suited that are commercially available and are tailored for this use, and are used heavily by the mobile data/cellular industry in their drive testing and coverage verification methodologies.
Good luck to you on your testing.
--ToO
Re: (Score:3, Informative)
Please mod parent up: He's completely correct, and his simple methodology is a good example of best practices.
However, there are more impromptu ways to accomplish similar feats:
One obvious possibility is the test mode built into many (perhaps all) cell phone handsets. These will typically display a number of different datapoints, such as signal strength and error rate. If the asker is troubleshooting only troubleshooting GSM or CDMA (so that switching carriers does not necessarily entail switching radios
One idea (Score:3, Funny)
What software or techniques would Slashdotters use to test the different devices? Are there any free or open source software packages that will do this?
I would suggest the "grad student technique".
Find a nearby university and convince a professor to lend some (under)grad students to your cause.
It helps if you have friends that can refer you to professors/students who'd be interested in the challenge.
P.S. The only thing better than the "grad student technique" is the "summer intern technique"
Re:One idea (Score:4, Funny)
P.S. The only thing better than the "grad student technique" is the "summer intern technique"
Especially if she's hot...
Test / Debug Mode (Score:1)
Latency, and bandwidth are easy (Score:2)
There's plenty of network monitoring tools around.
Signal strenght might be harder if the manufacturer of the device doesn't give easy access to that data (many cellphones can be switched into diagnostic mode; or some smartphone OSes have diagnostic software available). Maybe you can find some electrosensitive person?
Placelab? (Score:2)
Obligatory reference: (Score:5, Funny)
How about now?
Stats The Cellular Provider Sees (Score:3, Informative)
At least, this is what I see working with SNMP data coming off dumps from the cellular base transceiver stations attached to our towers.
Every 15 minutes, every GSM handset measures the perceived strength of the tower signal it is using, this is reported to the tower, and we record them all. Also every 15 minutes, the strength of the handset signal, from the perspective of the tower is sampled and recorded.
These readings go into buckets, for example if a reading showed a -78 dbm signal, it goes in the -78dbm bucket. Before long, a histogram can be generated with the datain the buckets, and we can see typical distribution for receive performance - for both handsets and the tower. Different towers have very different "signatures" in this data.
You may go around sampling receive performance, (which would be interesting) but I don't think you'll be able to map how well the cellular system is receiving from you.
Re: (Score:3, Informative)
You may go around sampling receive performance, (which would be interesting) but I don't think you'll be able to map how well the cellular system is receiving from you.
Which is most often the limiting factor.
The base station is transmitting at 50-100 watts ERP, but your handset (modem) is transmitting at less than a watt.
Sure the base station has pretty efficient recieve antennas, possibly with tower mounted pre-amps, but if it can't see your signal, it can't do much with it.
And a signal path needs to work in both directions to be able to do something useful with it.
Modelled data (Score:2)
Most providers use modelled data to estimate signal strength as it's too expensive to actually measure every point within their coverage. You might be able to obtain and use modelled coverage (which is mostly based on putting virtual towers into a digital terrain model) and undertake some validation. This would provide a meaure of the optimism usually present in modelled coverage. Actual coverage is usually going to be less due to buildings, weather, trees and other rubbish you rarely see in a DTM. Validati
After what happened (Score:1)
After what happened to wardriving be sure you legally have your butt covered. Especially if you prove the network sucks >.>
3G Coverage != Good Speed (Score:3, Insightful)
I wouldn't worry about charting the signal strength for 3G. You can be in a densely populated area showing five bars of 3G and your speed and latency can still be dog shit depending on how many people are hitting the tower, similar to your cable modem. It might be worth it to record whether or not you have 3G just to help map out your general coverage, but that doesn't mean you'll have great speed. Although, you can find something like that here [cellularmaps.com].
As for speed I like to use a util called iperf [ucf.edu] for measuring speed from one device to another across a network. You may have to open ports on your firewall or setup a VPN, which will add unwanted overhead, but you will get a good idea of which carriers have the best speeds. You can also run the simple tests using other websites like here [speedtest.net] or here [pingtest.net].
Re: (Score:3, Insightful)
I wouldn't worry about charting the signal strength for 3G. You can be in a densely populated area showing five bars of 3G and your speed and latency can still be dog shit depending on how many people are hitting the tower, similar to your cable modem. It might be worth it to record whether or not you have 3G just to help map out your general coverage, but that doesn't mean you'll have great speed.
Also, if your 3G network is based on CDMA (WCDMA, HSPA, HSDPA) then signal-to-noise ratio is as important as raw signal strength. SNR in both the downlink (what you receive from the base station) path - which you might be able to measure, and the uplink (what the base station receives from you) -which you have no way of measuring directly.
And while you might be able to control the signal level somewhat, you have very little control over the noise part of the equation.
Re: (Score:2)
"Also, if your 3G network is based on CDMA (WCDMA, HSPA, HSDPA) then signal-to-noise ratio is as important as raw signal strength."
Actually, this is true for ALL communications, bar none - S/N is what matters. You could have a signal that is barely 6 dB above what your receiver can detect and have a good S/N ratio, or have a signal that is so large it is driving the receiver into compression, thus distorting the signal and driving the S/N ratio down into the weeds. Shannon's Law.
However, reporting signal le
cdrouter (Score:4, Informative)
Industry standard?
http://www.qacafe.com/cdrouter [qacafe.com]
You can use linux (Score:2)
A lot of wireless modems have AT commands for signal strength and other info, you can do a Google search for documentation about AT commands for your specific modem. A war
Hack something together in Python (Score:1)
If you're on Linux you can use gpsd for interacting with your GPS. If you're on Windows using the new Location API in Win7 would be nice, but, more realistically, you'll be listening on the serial port your GPS uses and parsing NMEA packets. No worries, this isn't that hard to do. After a either a certain interval or time or distance traveled, send a few pings to a server and download a reference file. Dump your data into a CSV or use OGR to make a shapefile, KML, PostGIS layer, whatever.
Re: (Score:1)
Sensorly on Android (Score:2, Interesting)
Re: (Score:1)
_most_, if not ALL operators use TEMS at some point. All the equipment vendors I've dealt with won't even respond to a radio related trouble ticket unless there's a TEMS log.
What about standard tools? (Score:2)
I'm a bit rusty, but since your network is probably TCP/IP, can't you do this with standard TCP/IP tools?
The only thing that could be a problem is measuring "signal strength". However, but a lot of mobile chipsets work under Linux. Even tools running under Windows, there may be access to this information (netstumbler)
For latency, you could just use ping:
Charting? Gnuplot (Score:2)
I had to do something similar. (Score:4, Informative)
In the end, the project was cancelled before we got a chance to get to test 3G coverage. But we did get to think about it. Our customer was a fleet of cargo ships, going through a fixed path on the Rio de la Plata (River Plate). Basically, we were going to install our CCTV system in there, and have it push images and other information to our servers whenever it had signal. We wanted to know approximately in what areas of the river we would have signal. We were going to base our system on the Vodafone mobile connect driver. It's a set of Python scripts. Of course, it communicates with the modem using simple AT commands. It's released under the GPL. It is capable of measuring signal, sending and receiving text messages, and other nice stuff (like, well, actually dialing and calling PPP to stablish the connection). We had it working with several Huawei devices, but I know it works with other brands too.
Our idea was to modify this scripts so that they would try to maintain a connection, auto-dial every time it disconnected, and log the signal at certain intervals to a MySQL DB. We were also going to run download tests all the time automatically. Since there was no chance we would go on the ships with the devices (the ships were cargo ships that transported and extracted sand, and there weren't very comfortable, not to mention their average trip was at least ~72 hs.), so we wanted to do all of this automatically. The devices would also inform their IP to a web service every time their IP changed, so we could SSH in the machine running this tests in case we needed to change something.
We were going to add a GPS to this system, that would also log its position at certain intervals, so that we could then generate a color-coded signal map.
I hope this helps. It's really fairly simple. I would be happy to provide you with source code, but we didn't get that far into the project as to produce actual source code, since the customer changed his mind due to budget restrictions real early. Feel free to contact me if you have other questions {almafuerte (at) gmail (dot) com}
Academic paper on that topic (Score:3, Interesting)
ABSTRACT
We describe our experiences in building a city-wide infrastructure for wide-area wireless experimentation. Our infrastructure has two components — (i) a vehicular testbed consisting of wireless nodes, each equipped with both cellular (EV-DO) and WiFi interfaces, and mounted on city buses plying in Madison, Wisconsin, and (ii) a software platform to utilize these testbed nodes to continuously monitor and characterize performance of large scale wireless networks, such as city-wide mesh networks, unplanned deployments of WiFi hotspots, and cellular networks. Beyond our initial eorts in building and deploying this infrastructure, we have also utilized it to gain some initial understanding of the diversity of user experience in large-scale wireless networks, especially under various mobility scenarios. Since our vehicle-mounted testbed nodes have fairly deterministic mobility patterns, they provide us with much needed performance data on parameters such as RF coverage and available bandwidth, as well as quantify the impact of mobility on performance. We use our initial measurements from this testbed to showcase its ability to provide an ecient, low-cost, and robust method to monitor our target wireless networks. These initial measurements also highlight the challenges we face as we continue to expand this infrastructure. We discuss what these challenges are and how we intend to address them.
Get android handset and/or iphone; write an app (Score:2, Interesting)
one possible solution:
Download the android SDK; write an app; run it in the emulator that comes with the SDK.
I'm not sure how much work it'd be to tie your 3G card(s) into the emulator (that comes with the SDK), but it's possible.
Linux would be my first choice, but the SDK also runs on windows or mac os.
Bonus for getting a useful app included in the app store.
Too many variables... (Score:2)
While the goal is an admirable one, there are just too many variables to even begin to consider really doing this.
First off, you aren't working with a stable environment. There are changes being made to the cellular infrastructure all the time. Changes to settings in the towers, changes to the tower equipment and addition of new towers. You will not be informed about any of this making your information out of date almost immediately.
Next there is the problem that most throughput problems are going to be
Google "UT Diagnostic Monitor" (Score:1, Informative)
...or "Spirent UTS"
I used to do this kind of programming all the time for Qualcomm and Kyocera Wireless. Cell phones have a User Terminal Diagnostic Monitor running on them constantly that can be used to get almost all data in realtime out of the phone through it's data port. There are a couple of areas of memory that are write only for things like passwords, but you have access to pretty much everything else.
Signal quality isn't a comparable measure (Score:5, Informative)
A bigger number on the same device probably means you have a better signal. A 'two' on one device could be equivalent to a 'five' on another or a 'one' on a third. There is no standard that is used across the industry, or even across all devices from a given manufacturer.
When a device manufacturer gets customer reviews that say "I only get one bar with your phone but two from company X" the device manufacturer can either try and explain repeatedly that their one bar is better than X's two bars and that unlike X you can still make phone calls on our device on one bar. Or they can just double the number of bars reported on the next model so they don't look worse than X.
Which do you think they do?
To do a real test you need to use a constant antenna and location, attenuating the signal gradually until each device stops functioning. The amount of attenuation it can take is a crude indicator of the quality of the radio.
Howto (Score:1)
1) Ping Test. A simple ping to the nearest visible IP address, usually the GGSN, will give you the round trip time for a session in progress. You are looking for the variability. A simple ping -t command for a few minutes and plot the results in excel. You can vary the packet
Won't be reliable data or give ability to compare (Score:3, Informative)
I was a 3G data engineer for Sprint PCS during the launch of 2.5G and 3G data. There are a few problems with the type of testing it sounds like you want to do.
First, I would suggest reading the specs from the IEEE on CDMA 2000, aka 3G. CDMA2000 allows the ability to allocate and de-allocate bandwidth on demand and based upon quality of service configurations. At night, with no one on the cell tower, you're going to get the full pipe for data. You'll see bursts up in speed but then several things can happen. First, voice takes priority. So at midnight in this scenario, I pick up my phone and make a voice call and I take priority. You're pipe just got smaller. The next variable is overall tower usage. Cellular towers shrink and expand RF power with regards to usage. As more users get on, the cell tower will reduce it's footprint. So even if hardly anyone is on the phone, but there are a ton of subsribers on a cell, it can drop it's power. So your bandwidth and RF are variables which change by the second.
So if you're just doing a "Hey lets just see what we see," type of test, then expect a huge array of data with not too many descernable results. If you're looking for data to be compared with something else (carrier vs carrier, region vs region), then it also won't be terribly useful. As a geek, it'll be cool to know you can set the test up, but as a quantitative analysis tool it won't be repeatable or statistically useful. You also can't really compare it to desktop loading times either. The images you "download" via a wireless carrier are not the same. Go to Yahoo and download the GIF for their logo using an aircard or wireless carrier device. Now, download the same with a desktop PC over a wired (or other non-wireless carrier method). Not the same file size, huh? :)
Just as an aside, I used to test it using Stick Figure Death Theater ( www.SFDT.com [sfdt.com] ) since you can start a really long download and watch the stats. Also, it was entertaining to watch at the same time.
CellMapper (Score:1)
Try the application CellMapper from the Android market. It creates maps of user-submitted cell signal strength. Some of the areas you are interested in may be covered already, and any new data you submit will help to benefit the community as a whole.
Do it yourself (Score:1, Informative)
I found it extermely easy to write a program myself for my windows mobile handset. It took me about an hour to write a program which a) collects network information (available techniques such as GPRS, EDGE or HSDPA) and signal strength along with tcp ping statistics (for some reason some Finnish wireless base stations block icmp ping) and b) gps data (time, speed, location). This gave me a possibility to gather quite interesting data about both coverages and the effect of speed as I ran the mapping program
Using old Nokia Network Monitor (field test) Mode (Score:2, Informative)
This is more a suggestion to help with mapping received signal strength (RSSI), rather than data network latency and bandwidth (you can argue that those data network metrics rely heavily upon your RSSI!):
Grab an old Nokia, use gammu [gammu.org] to enable Network Monitor mode [wikipedia.org], fire up a GPS and display the combined information streams on a map. I did exactly that as an experiment [spench.net] using a Nokia 3310 and a Navman GPS receiver. Interesting to then correlate the signal peaks to the actual [spench.net]
Go for the pros... (Score:1, Informative)
Warning: shameless plug dead ahead...
Look at http://www.p3-group.com/communications/en/home.html, it's their core business and they just opened an office in the US.
If your end users will run 24hours (Score:1)
If your end users will need to run 24 hours a day you will need to test 24 hours a day.
We have number of little (Linux of course) boxes using 3G USB modems here in Sweden, and have discovered that Telia does all sorts of maintenance at night and at weekends.
Furthermore the latency (as measured by ping) varies quite a bit depending on time of day, i.e. when the local farmers are downloading their porn.
Another thing we've come across is that the versions of the USB modems changes with great frequency! Which o
This is my job! (Score:2, Informative)
There are plenty of expensive software solutions for this that are used at a professional level (ROMES, NEMO TEMS etc).
If you are after a free solution, the AT commands of most units Huewai, Option etc will give you network information,
For example: Signal as RSSi. (look under 3GPP TS 27.007) AT+CSQ? gives a number 0 (-113dBm or Less) to 31 (-51dBm or greater)
Failing that a lot of the dashboards have open APIs (in the UK Vodafones dashboard gives you access to lots of information)
Most of the Samsung mobiles
Re: (Score:2)
I want to comment on Received Signal Strength, Eb/N0, Es/N0, SNR, etc.
A commercial data receiver is NOT a professional test system. I did a lot of testing on satellite receivers, and what I know is that the SNR numbers you get from commercial receivers represent the ability of that receiver (antenna, front end, demodulator) to receive the signal under your current noise floor and interference situation. The SNR numbers you get may be correlated with actual SNR, but it is not exact.
For example, if you appl
Re: (Score:1)
Good reply,
You're correct about a modem being non-professional equipment, there are some excellent PN Scanner/Decoders used in these system - Rohde & Schwarz TSMW, TSMU, TMSL, however a modem is representative of a customers experience.
SNR is not really used as Ec/No (Chip energy over noise) is more useful. The scanners mentioned supply a CINR figure - Carrier Interference Noise Ratio.
General testing is carried out with a user device modem/mobile and a Scanner as this is not controlled by network comm
Resolve an issue with your vendor? (Score:2)
Using SMSlib (Score:1)