Comcast Cheating On Bandwidth Testing?

dynamo52 writes "I'm a freelance network admin serving mainly small business clients. Over the last few months, I have noticed that any time I run any type of bandwidth testing for clients with Comcast accounts, the results have been amazingly fast — with some connections, Speakeasy will report up to 15 Mbps down and 4 Mbps up. Of course, clients get nowhere near this performance in everyday usage. (This can be quite annoying when trying to determine whether a client needs to switch over to a T1 or if their current ISP will suffice.) Upon further investigation, it appears that Comcast is delivering this bandwidth only for a few seconds after any new request and it is immediately throttled down. Doing a download and upload test using a significantly large file (100+ MB) yields results more in line with everyday usage experience, usually about 1.2 Mbps down and about 250 Kbps up (but it varies). Is there any valid reason why Comcast would front-load transfers in this way, or is it merely an effort to prevent end-users from being able to assess their bandwidth accurately? Does anybody know of other ISPs using similar practices?"

This is an advertised feature I believe

Doesn't Comcast advertise this "SpeedBoost" as a feature - the language in their ads is something like "get massive super speed for the first 10MB of a download, then it will revert to your provisioned line speed"... So, it actually *is* a good thing rather than something to pad bandwidth tests, and it does generally help your general user, right?

Re:This is an advertised feature I believe

I agree - I know that Shaw Cable (Alberta) offers a plan that does exactly this: for 5-20 seconds, you get increased download bandwidth. This is their PowerBoost feature, that costs an extra $2.95 above your regular plan....

Re:This is an advertised feature I believe

Yeah, the point is that you can get a webpage down in those first few seconds generally so browsing is much better than it would otherwise be.

Re:This is an advertised feature I believe

This is it, and it should be amazingly obvious to any "sysadmin" who should know something about general browsing habits. I worked for a wISP for a year and this was a standard feature offered by the company for no extra charge. Max subscriber speeds were 1.5 Mbps, but for about 20 seconds ALL traffic was burst to 2 Mbps, regardless of the subscriber paying for the 500 k/s speed or something higher. For general browsing and light email, it made all the customers quite happy to have things terrifically speedy.

Re:This is an advertised feature I believe

So... how can you tweak your Bittorrent client to fool Comcast into thinking it is making lots of small downloads?

Re:This is an advertised feature I believe

Torrents do that anyway. That is the reason why comcast have to beat them on the head. Each segment in the download is small enough to fit its "booster" criteria.

Actually, there is nothing wrong with this approach. This means that interactive services and casual browsing are favoured vs bulk downloads. That is what every ISP wants to do anyway.

PowerBoost uses a 30 second average, not filesize

Torrents do that anyway. That is the reason why comcast have to beat them on the head. Each segment in the download is small enough to fit its "booster" criteria.

No, that's not right.

PowerBoost only accelerates the connection if the average speed you've been getting over the past 30 seconds* is less than the speed you are rated at/paid for. So if you have a 6 Mbps connection, that's 768 KB/s max. PowerBoost will raise that to up to 2 MB/s for a little less than 15 seconds, making your average for the past 30 seconds equal to 768 KB/s. After that, no matter how many new connections you open, your connection stays at 768 KB/s. But if your connection gets interrupted/throttled for a few seconds, you may get another boost after it resumes, until you are back to 768 KB/s 30 second average again.

*it may be slightly more/less than a 30 second average. Boosts seem to last about 10-15 seconds, which would make sense with that number.

Can't be right

That can't be right. From your description, it sounds like a genuinely good and beneficial to the user idea. Where's the catch ?

Re:Can't be right

Any actually benefit to the customer is purely coincidental and unintentional.

That nice warm shower feels pretty good until you realize someone is pissing on you.

Re:This is an advertised feature I believe

I've made something of a game out of it, actually. With careful tactics, one can easily hit as much as 1.0 MiB/s upstream for short periods. I use Deluge to play. My present record is 2.4 MiB/s, on an Ubuntu 7.10 torrent for which I already had all the file data.

First, configure your torrent client to use a modest number of connections -- limit it to, say, 250 connections globally and 70% of your nominal upstream speed. Then, get on a very large, active torrent and build up a few minutes' worth of downloaded data. Once you're in the swarm, open everything wide up -- no global connection limit, no bandwidth cap, and no per-torrent upload slot limit. If your client has a bandwidth chart, watch it scroll by and enjoy the thrill as your upstream bandwidth surges to heights like you have never seen before. Of course, eventually the Power Boost will wear off and some connections will finish as their pieces are completely transferred, but it's fun while it lasts.

Re:This is an advertised feature I believe

They aren't shaping based on individual downloads. They are going to shape all the traffic going to your cable modem as a single stream. There may be other outbound queues within the shaper to provide fair queuing and such, but that gets too complicated for this post.

Most shapers (including the ones in their broadband routers) allow a variety of parameters.
You can set a sustained rate, peek rate, and burst size. For example a common implementation would have the following values (I haven't worked with Cisco QOS much, so it may be implemented differently but the principals are the same):
sustained rate: 2mbps
burst rate: 10mbps
Max burst size: 10 Megabytes

The burst size counter is depleted as you download over 2mbps, and replenished when you download under 2mbps.
When you download a 100MB file you will deplete the burst size at 8mbps. You can download at 10mbps for 10.5 seconds, at which point your download will drop to 2mbps and will stay there until you slow your transfer rate. If you stop downloading completely it will take 42 seconds to refill your burst counter.

Re:This is an advertised feature I believe

You are correct in your interpretation. The customer briefly receives more than they pay for after a period of inactivity, this throttles down to the 'purchased' bandwidth as the activity increases. For Read-Click-Load-Read web browsing this gets content in front of eyeballs quicker and is a "good" thing. If you are using a tiny file for a bandwidth test it screws up the results. HINT: USE A BIGGER FILE.

People are out with pitchforks and torches over the "bad" thing Comcast does, throttling Torrent downloads, which works completely differently. To throttle a torrent, they forge a "I'm dead" packet from remote host, and send it to the customer. This causes the customer's torrent application to shop elsewhere for a feed. The repeated connect-forge disconnect-search-connect process slows the overall transfer. This only works because of the multi-peer technology underlying torrents, and wouldn't work with web browsing or ftp*.

-Ellie
* technically it would reduce the bandwidth usage, because it terminates the connection. This would result in broken connections and half-downloaded files. Then the pitchforks would REALLY come out.

Re:This is an advertised feature I believe

To throttle a torrent, they forge a "I'm dead" packet from remote host, and send it to the customer. This causes the customer's torrent application to shop elsewhere for a feed. The repeated connect-forge disconnect-search-connect process slows the overall transfer. This only works because of the multi-peer technology underlying torrents, and wouldn't work with web browsing or ftp*.

Actually that is not entirely correct. If they were simply forging the RST packet and only sending it to their customer it would be a simply matter of having the customer's firewall filter out all RST packets on specified port that is used for torrent download/uploads. I in fact have such a filter rule in place. However, detailed testing has shown that Comcast is sending the RST packet to BOTH their customer AND the outside connection, not just their own customers. Unless both sides have the RST filter in place on their firewalls, the connections are still dropped and throttled. This is what is going to get them into trouble as they are not just sending forged packets to their customers whom they have it written down in their service agreements somewhere that they can do this to you, but they are also forging YOUR identity and sending those packets to outside entities to affect their service as well, something that those people have NOT agreed to have happen to them.

Re:This is an advertised feature I believe

If you read the whole of the comment (don't strain yourself, now), you'd see that the user is in fact paying for "insert speed here", but for a premium they can also get "insert speed here" for the first $size_limit || $time_limit of their download, which *is* a good thing when browsing webpages or other small files.

Powerboost

This is because of powerboost. As I understand it, powerboost makes the first 20MB download at a higher rate than your advertised bandwidth. Since bandwidth tests are done on such small files, you get a worthless result. The idea is that people who download lotsa of relatively small files get better performance, where as people downloading a lot of huge files like ISO images, full length movies, et cetera willg et initially good speed but after 20MB will feel like they are getting gipped.

Re:Powerboost

I suppose it depends on how much it drops for those larger files. If it goes from 10Mbps to 1 Mbps I could see the point, but if it only drops to something lik 7 or 8 Mbps I think that's a reasonable rate. We also have to remember that this is a residential connection. It is designed for the typical residential user. That type of person will download a lot of smaller files regularly. The result is that the web browsing will seem very fast. ISO downloads? Not so much.

I wonder how it deals with P2P or a multi-streams of data. What if I have 10x 30Kbps streams running simultaneously would that aggregate and trigger the throttle down mechanism?

Re:Powerboost

I have Comcast's advertised 8 Mbps service and I very consistently get that downloading large files off of Usenet. I get about 25 Mbps for the first 20-30 seconds after I start a download.

Wouldn't it help with browsing speeds?

Most internet browsing is with relatively small amounts of data, so wouldn't front-loading of this nature noticeably increase browsing performance? Since this kind of performance is noticed by the majority of users it would seem to be something that increases their perception of their connections' speed.

I'm not saying that Comcast might not be cheating on purpose for speed tests, I just think that there might be another reason behind it other than just to make their test scores artifically high.

Web browsing optimisation

Sounds like they have simply optimised their network to favour "bursty" usage, for example web browsing. This would seem a sensible thing for a consumer ISP to do.

SpeedBoost is the thing

Several months ago, New Englanders were the first consumers to experience Comcast Communications' latest high-speed Internet upgrade - PowerBoost Speed Enhancer. The speed upgrade is now being rolled out to Comcast customers nationwide. This new network technology temporarily doubles Internet speeds for consumers subscribing to the company's 6 megabits per second and 8 Mbps services, bringing download speeds to 12 Mbps and 16 Mbps, respectively.

Some consumers may not notice the speed increase when downloading smaller files, such as text-based e-mails and simple Web sites with few graphics. However, customers who frequently download large files, such as software, games, music, photos, and videos will now download at speeds that are faster than ever before. For example, PowerBoost significantly reduces the time it takes to download a one hour television program. Comcast subscribers at the 6 Mbps tier would reduce their wait time in half - from 4 minutes and 29 seconds to 2 minutes and 15 seconds. And MP3 fans will be able to download music files as fast as 2.2 seconds!

See more here [broadbandinfo.com]

Token Bucket

Um, this isn't a new concept, nor is it particularly sneaky:

http://en.wikipedia.org/wiki/Token_bucket [wikipedia.org]

It's true

I can assure that they do absolutely do this, and it is really annoying.

It's really bad on uploads -- I just ran a test and I got 300 KB/s for the first 5 megs, then it degrades 100 KB/second over the next few megs, so that by the time you have uploaded 14 megs you are getting close to 40 KB/S in upload speed, and the connection is so bad that the shared digital phone line does not have enough bandwidth to have a phone conversation. Stop the upload and start it up again, and you get 330 kb/second, with the same degradation curve.

For downloads they do the same thing, but not so severely -- I downloaded a 67 meg file and it ran at about 750 KB initially, but then dropped to around 350-400 KB/S (according to the FTP app) about halfway through.

So for anyone using the connection for smaller file sizes (like the speed tests) you seem to get "blazing" speeds -- I ran the test at a couple of the internet speed test sites and they both think that I have 12000-14000 kb/s download speed and 2700 kb/s upload speed.

So if I didn't have any other way to measure it, I would think that I was getting way more than I paid for, rather than something that in reality is very pitiful.

Consumer-grade Shared bandwidth

What part of "shared bandwidth" do you not understand? It boosts peak (short-term) rates by using other' users idle capacity. This has been done for 10+ years and is a feature of consumer-grade links that helps keep their costs down.

If you want a commercial-grade link you expect to saturate, pay for it! Otherwise, you are stealing from other users and the ISP should throttle you to be fair to them.

Shortest Job First

In operating system theory, it is well known that a scheduling algorithm called "Shortest Job First" yields the least total waiting time. The SJF algorithm is usually implemented by giving a "new" job high priority, and then reducing the priority gradually as the job accumulates resource usage. The algorithm was developed in the 1960's to allow time-sharing operating systems to provide rapid keystroke response, while continuing to process large batch jobs in the background.

For communication systems, the same principle applies. The only difference is that the network is sharing a different resource (circuit bandwidth), instead of cpu time. The "new" connection gets high priority, and then that priority is reduced as the number of bytes/packets transferred over that connection increases. This allows rapid response for interactive applications, like browsing or editing, while also allowing the network to process large data transfers in the background. To apply it to datagram traffic, the switch just keeps a priority for each source/destination address-pair in cache, and any pair that is not in the cache is regarded as "new".

This has been pretty much standard practice in packet communication switching for a very long time. There is no surprise here, at least not to those of us who have not been doing communications network programming for a few decades.

sheesh...

http://www.dslreports.com/shownews/Slashdot-Keeps-Rediscovering-Comcast-Powerboost-91976 [dslreports.com]