Slashdot Log In
Internet Partitioning - Cogent vs Level 3?
Posted by
Cliff
on Wed Oct 05, 2005 06:50 PM
from the balkanization-of-the-'net dept.
from the balkanization-of-the-'net dept.
slashmicah asks: "Internet partitioning and Tier 1 ISPs are something most people don't know much about (myself included). Today, however, some Slashdot readers might have run into some issues involving these two topics. Cogent Communications and Level 3, both Tier 1 ISPs, are apparently having some 'undisclosed' disagreements, causing an Internet partition by turning-off or deactivating their peering point. Cogent Co. has released a statement explaining their side of the problem, however they have no mention of when the problem will be fixed, or when they will sort it out. This partitioning is a problem because any [single-homed] computers that are connected through Cogent Co, can not connect to [single-homed] computers connected through Level 3. Having spent all day sorting out this problem, I ask Slashdot: Isn't there a better way that the issue of peering can be handled/regulated? If not, does the future hold a scenario in which the Internet is split into several separate networks, only to be connected at the whims of large corporations?"
Related Stories
[+]
Technology: ISP Dispute Causing Connectivity Issues for Customers 192 comments
I Don't Believe in Imaginary Property writes "A peering dispute between Telia and Cogent is causing routing and connectivity problems for many internet users. Cogent shut down their connections to Telia over what they described as a 'contract dispute' over the size and location of their peering points. Telia attempted to route around the problem, but Cogent blocked that, too. This has caused a lot of trouble for sites which are not multi-homed. Groklaw, for example, is on a Cogent network (MCNC.demarc.cogentco.com), so any Europeans connecting via Telia can't get through."
[+]
Technology: Peering Disputes Migrate To IPv6 111 comments
1sockchuck writes "As more networks prepare for the transition to IPv6, we're seeing the first peering disputes (sometimes known as 'Internet partitions') involving IPv6 connectivity. The dispute involves Cogent, which has previously been involved in high-profile IPv4 peering spats with Sprint, Level 3 and Telia. Hurricane Electric, which has been an early adopter on IPv6, says Cogent won't peer with it over IPv6. Hurricane has extended an olive branch by baking a cake bearing a message of outreach for Cogent."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Interesting scenario, though most likely untrue. (Score:5, Insightful)
Re:Interesting scenario, though most likely untrue (Score:5, Insightful)
Parent
Re:Interesting scenario, though most likely untrue (Score:4, Interesting)
Personally, there are several sites I can't get to from home now. I didn't have any problem getting them from work (UT Austin). I have effectively zero power to rectify this. Annoying.
Now, if Cogent offered me some way to connect to them for an additional $5/mo... would I?
Think... if the government allowed an additional $5/mo. for each Tier 1 my ISP (Time Warner) is connected to... my cable modem bill would instantly double.
That's a scenario that bothers me more than the dissolution of the Net does. Flip side, the Internet would get a whole lot more redundant really quick...
Parent
Re:Interesting scenario, though most likely untrue (Score:4, Insightful)
This will eliminate any internet performance anomalies for those customers so that they are not affected in a bad way by this issue. It's also a good PR move that might let them grab a few Level 3 customers who are impressed by the goodwill gesture.
Parent
Re:Interesting scenario, though most likely untrue (Score:5, Informative)
Read about it here [2sparrows.org] - warning: it's a 120-page pdf...
-David
Parent
Re:Interesting scenario, though most likely untrue (Score:5, Informative)
What they are essentially saying is: "We haven't done anything. We haven't made any changes on our side" Level 3 have terminated their connection to Cogent "Without cause". Now, that's probably legal speak on Cogent's side for we haven't got the letter in triplicate yet telling us what the reason is. Or otherwise whoever put up the notification about it doesn't know.
Now, Cogent may have tried to change the peering arrangement, or Level 3 may have too, one side probably didn't agree, or otherwise an agreement ran out and the switch got flipped. This has happened previously with Cogent in their peering arrangement with AOL. [findlaw.com]
What Cogent are trying to do is get business from Level 3 customers because Level 3 stopped the connection. Cogent is offering them connections to the Cogent network (And subsequently Cogent's customers) for a year with no fee on the amount of data they put through. That connection itself they will obviously have to pay for, but the customer can connect into (presumably) the closest of any of 1000 points across North America and Europe.
Now some people are already connected to both Cogent and Level 3. These people won't have any problems as they will be able to go direct into either ISP. These people would probably have never have used the interconnect between Cogent and Level 3 either, unless one of their connections into either Cogent or Level 3 went down.
I understand this is still rather technical, for a simpler version, take a look through the document that I linked to.
Parent
It's happened.... (Score:4, Informative)
That either network corporation allowed this to occur is without pardon.
What I'm afraid of, is when this is all over and people realize how singificant it was, the solution to mangers will be "buy service to each, so we never have to worry about being partioned". Which is exactly what both companies would like to see.
Parent
Really easy solution all net-ops already know (Score:5, Insightful)
Level3 is threatened by Cogent's bandwidth pricing model, and is using it's weight to threaten that model, forcing Cogent to buy transit if it wants to reach its network. THat's how things work: you can't get free bandwidth from everyone, you're going to have to be willing to step up and pay for your link.
quote attribution (Score:4, Informative)
Parent
Reminds me of... (Score:4, Interesting)
A quote about censorship. Come on, we all know it. The internet will see that as damage and route around it. The very fact that you mention that this affects single homed computers on one or the other network means that even at the onset of this "partitioning" it is ineffective.
Re:Reminds me of... (Score:4, Insightful)
A data carrier makes their money off the small guys that want to plug into the heavily funded infrastructure that the big guys have spent much time building up. If you have two equally sized carriers with equal data being sent/received to the other network, it makes perfect sense to peer them. Since they both have to bridge the gap to one another's network somehow, its cheaper to go directly to one another.
Now, lets say the data flow rate isn't symetric. TinyISP and UberISP. TinyISP uses 100Mb/s on UberISP's network, but UberISP only uses 1.2Mb/s on Tiny's network. UberISP wouldn't feel inclined to allow a peering agreement since most of the financial benefit is happening by TinyISP.
Now with all that said, your argument is only partially correct. Yes, "The internet will see that as damage and route around it" can happen, but it isn't the magical sugar plum fairy granting magical bandwidth to route this traffic. Its Cogent footing the bill to L3 or some other peer in order to get to their intended recipients. Thats if L3 hasn't blocked the Cogent Netblocks as well. In which case, Cogent would be forced to have a peer Source-NAT their traffic if they wanted to reach L3 resources. Thankfully, To my knowledge this crazy scenerio has never occured.
Parent
Re:Reminds me of... (Score:5, Insightful)
If your company or ISP uses only Cogent for bandwidth, it is currently impossible to reach L3 only connected services. I believe L3 to Cogent is being blocked as well.
Parent
PITA but move along (Score:5, Informative)
At least it's not like UUNET more than one, some years ago, wanting to charge other Tier 1's per packet for transfer when peering while their traffic they wanted to pass for free. They were a big dog and were trying to make everyone pay. No one did and threatened to or did kill off traffic until UUNET got the sh*t together. But the did try to pull it off more than once.
OK, WTF time here (Score:4, Interesting)
If my grandma can't check her email for a day, I don't really care that much. If my doctor is consulting with a cardiac specialist over using VoIP (V being either voice or video) concerning an acute health problem then I have a much larger problem with outages. As long as we have important economic or healthcare services running over the internet--which is the foreseeable future--this sort of thing needs to either be avoided or have a pre-planned workaround.
I guess this explains some of the unresponsive hosts I came across today. And here I was thinking it must be Bob's Worm of the Week.
Re:OK, WTF time here (Score:4, Insightful)
If you keep the following in mind, you will be a much happier person:
(a) The Internet is not guaranteed to be secure.
(b) The Internet is not guaranteed to be reliable.
Anyone making claims to the contrary is a charlatan.
Parent
Re:OK, WTF time here (Score:5, Insightful)
Okay, first off if your physician is using any IP-based service and ISN'T using a dedicated connection then no one's to blame except the fucktard who set it up in the first place.
Physicians are subject to the greater economic pressures than any other small business. Insurance companies, government regulations, litigation risks, patient scheduling, qualified and reliable staffing, emergency on-call, and obligitory hospital fundraising contributions. Given a choice between an $1,800/month point-to-point circuit PLUS provider termination and service fees, or a $59/month xDSL for probably 4x the bandwidth, which do you think most will use? If in doubt, give your physician a call and report back.
Parent
Re:OK, WTF time here (Score:5, Funny)
I envision such a system could be seriously robust and would possibly withstand a nuclear attack.
Parent
Re:OK, WTF time here (Score:5, Funny)
I envision such a system could be seriously robust and would possibly withstand a nuclear attack.
Wow, sounds like something that could be of great strategic military value. I wonder if the DOD would be interested in developing this idea...?
Parent
Re:OK, WTF time here (Score:5, Informative)
When Paul Baran [wikipedia.org] had the task of designing a network that could withstand a nuclear attack, he envisioned a "distributed network". By today's lingo, it's a mesh network where each router is connected to approximately the same number of other routers. But now that routing infrastructure is driven commercially, with tit-for-tat contracts between Tier 1 ISPs, we ended up with what he said was a "decentralized network" -- that is, power law. Not what Paul Baran had in mind. If the underlying topology were his distributed network, you wouldn't be reading this story.
You can read his paper here [rand.org]. The Internet could withstand one nuclear attack. Several well-placed nuclear attacks? That's debatable...
- shadowmatter
Parent
Re:OK, WTF time here (Score:4, Informative)
Cogent is not a bonified teir 1 as they still pay for some of there transit.
Parent
Been dealing with this all day. (Score:5, Informative)
public peering! (Score:5, Interesting)
In all seriousness, these private companies will work it out when they realize that their paying customers are pissed and leaving because they're no longer selling very complete connectivity. Just like in the past, it won't take long. If TV has taught me anything, these problems are usually wrapped up pretty nicely in about 28 minutes.
As a rule... (Score:5, Insightful)
Telcos suck.
ALL of them do in their own special way.
Cogent's message (via NANOG) (Score:5, Informative)
Level 3's official statement (Score:5, Informative)
They're saying Cogent is intentionally not advertising routes to them via other providers, presumably because they're upset about not having a peering agreement in place. Anyone affected by this presumably needs to harass Cogent.
http://ws.arin.net/whois?queryinput=AS174 [arin.net]
Seen it before in Australia (Score:5, Interesting)
Optus didn't appreciate that and promptly blocked all data between themselves and Telstra. Customers with Telstra were pretty much screwed because they couldn't contact anything and with their network going nuts even sites within Telstra sucked a lot. Still, for a couple of days there, it was two halves of an internet available in here. Was amusing to watch really.
Easy to Fix (Score:5, Funny)
Some thoughts on this mess (Score:5, Informative)
Second, it is dishonest for Level-3 to blame Cogent for this exclusively. Level-3 had a peering arrangement with Cogent for a long time. If you look at Level-3's interconnection policy page:
http://www.level3.com/1511.html [level3.com]
It still looks like Cogent and Level-3 could peer under these terms. It was Level-3 that pulled the plug, not Cogent.
What is really annoying is that this is only traffic from Level-3 to Cogent, not to other parts of the internet. Level-3 wants money for Cogent customers to connect to Level-3's network but does not understand that this is a two-way connection and that Cogent's customers and Level-3's customer both benefit from this equally.
Up until this point, I was very happy with Level-3. They run an excellent network and I pay top-dollar to be on it. This blatent disregard for the impact on their customers is a diservice to their customers, to their reputation, and only begs for regulation.
No Rules. (Score:4, Interesting)
This is bad. Very bad. (Score:5, Insightful)
But the action of Level3 is not merely an inconvenience to end users; it is hurting a great many small businesses, badly. There are thousands of small businesses that depend on single-homed Internet connectivity and that cannot afford dual-homing. There are dozens of low-cost datacenters that provide single-homed bandwith to tens of thousands of servers.
As we speak, the livelyhoods of thousands of entrepreneurs are being threatened. Many people depend on being able to offer internet services to any peer on the net. But today, Level3 has changed the rules of the game, and have split the Internet into two somewhat isolated internets.
This is happening on a very large scale. Sure, most of the affected people and businesses are going to get through it just fine. But given the sheer scale of the Internet, a small percentage of those depending on full connectivity will not escape this ordeal unscathed.
You can be sure that a few small businesses will close because of this, the reputations of a few persons will be damaged, and there will be a few bankruptcies - all because of Level3's evil actions. You won't hear about it in the media - nobody cares about such small-scale damage. But the damage is already done, and it is getting worse with every passing hour.
I urge you to join me in a five-minute hate against Level3 and all that their evil discriminative ways stand for. While Cogent is widely recognized for its shitty cut-rate network, they are the good guys here. In the past few years, Cogent has been a major driving force for lowering bandwith costs. Level3 is fighting back, and they long for the days where they charged 5000$/mbps. I say: down with Level3 !
How to Complain (Score:4, Informative)
720-888-2518 (Level3 Investor Relations)
and complain.
Or call 877-453-8353 (Main customer service number).
Parent
Re:This is bad. Very bad. (Score:5, Insightful)
Cogent's sells bandwidth for cheap
Parent
Actually, for small business the economics change (Score:4, Interesting)
Case in point: I've run my own data center for 12 years (18 if you could dial up bbs crap). This week, I'm shutting it down. I need more reliability for an important application, and it will be cheaper for me to outsource the public facing side to a data center (In my case, linux boxes at ServerBeach -- I can plug them, they've made me happy).
This is coming from someone with 13 years running his own shop; who owns good firewall, routing, and standby power equipment; as well as servers. Still, it will be cheaper from month 1 to outsource today. For less money, I don't have to buy (or maintain) hardware, get more bandwidth, multi-homed servers, way more reliable power and facilities, and a lower power bill.
The market is changing. More and more consumer broadband utilities (which is what they are) will have to drop out of the single homed dedidcated circuit market. Dissagree? Time Warner doesn't. Why do you think they're building state of the art colocation facilities and datacenters in the markets they serve?
Because soon public facing servers for any serious purpose will live primarily in big datacenters. The only companies to host their own, will be hosting them in their own big corporate data centers.
Parent
Peering 101 (Score:5, Informative)
Provider A and Provider B peer, be it public or private, normally they do this in several places and alternate who pays for the circuit, etc. Now, under normal circumstances, they both push enough traffic from one to the other to justify this mutual payment plan. However, in some cases, you find that B is either intentionally dumping traffic into A thinking A won't notice, or A discovers that its sending so little traffic to B in comparison to the amount B is sending to A that its not worth the continued cost.
When the first sort of thing happens, it usually gets resolved -REALLY- quick, that sort of behavior is not tolerated and will result in B getting de-peered by A (and potentially others once the abusive behavior is discovered and known) exceptionally quick unless B can show that it wasn't done knowingly or intentionally.
When the second instance happens
Based on Cogent's 'oh poor us' post from this morning, I'm leaning towards them having given L3 the finger when L3 said 'look, this isn't equitable, we're going to have to re-arrange the money'.
YMMV of course, but I'm betting I'm not terribly far off.
More Regulation != Solution (Score:5, Insightful)
The Internet has flourished without much control, run by Both large and small businesses for one reason: profit. Information is free yet its distribution is profitable.
If we give government control (taxation, censorship and worse(, we'll see less freedom.
Why did this jinx happen? Because the top tier providers aren't making a profit. But their calls for support go unheard, so they found a way to make it news.
When businesses that rely on the infrastructure paid for by private industries, they have high expectations. But they're not paying for that infrastructure!
Trust me, no one wants to bifurcate the Internet. Its a ploy to show a problem that needs to be solved. You will Never see it done for control, censorship or monopoly powers. You'll only see it when consumers don't pay for what they use. See California's old electric company that was forced to sell energy at a loss. They went bankrupt.
Common carrier status (Score:4, Insightful)
These guys suck. May capitalistic pressure force them out of business.
-ted
Level 3 = Failing Business = Retribution (Score:5, Informative)
The arrogance of Jim Crowe [workign on his 7th manion and 8th large layoff at Level 3] and Kevin O'Hara (President, CEO) is only matched by the jailbird Bernie Ebbers. They only reason they have yet to decalre bankruptcy and liquidate thier debt (and clear away their bad business model with a fresh debt-free start) is that all their Omaha cronies have tied up money in the company stock, which would be flushed.
That they would resort to stunts like this against companies that undermine their pricing is not surprising. Level 3 have amassed BILLIONS in debt that they cannot service at current pricing levels, while Cogent and other more nimble competitors can sustain operations and drain Level3 into bankruptcy. So Level 3 execs do what arrogant desperate people do: lash out.
Level 3 is playing the "Sampson" card - if they cant make people price it their way, they will take the internet down with them.
And they did this trying to kill XO and now Cogent. Watch for more until they finally admit their business model is a failed one, and they declare bankruptcy, wipe the debt, and then begin to price lower and rake in the profits that their debt service is now eating.
Re:Level 3 = Failing Business = Retribution (Score:5, Informative)
Putting on my investor's cap, and taking a quick look at financials, its obvious that Level 3's burn-rate on cash, and billions in debt is not looking good for them if they cannot start generating both higher margins and more revenues. Neither one individually will save them at this point. The debt service is eating what EBITDA revenue they have coming in faster than they can produce it. And with companies like Cogent undercutting them Level 3 is dying; it seems the only question now is how much interconnectivity they will destroy in fits of pique like this.
I think you may also be right on another point, after considering it and runnnign the numbers: if Level 3 were to reorganize in bankruptcy court, dump the current shareholders, turn the debt holders into stock holders to ditch the debt, then they would probably be very profitable at even lower pricing levels. After all, that is what a lot of their competition has done. If they do that, Level 3 will cut the throats of every company out there, and make a bundle doing it, free-market style. Pretty interesting scenario.
But first they have to drop the stockholders, and from your post, it sounds like cronyism is a big factor, so its only going to happen when there has been far too much damage to Level 3 as a company. Thats a shame, because looking at their web site, they have some good ideas, but the wrong time and place for them.
Thanks for the post AC (wow an AC that actually said something useful!)
Parent
Transit vs. nontransit service (Score:5, Informative)
When you buy Internet bandwidth from your ISP, you are getting transit service. This means that you can use the link to send traffic to that ISP and to other ISP's upstream from it.
Nontransit service means that the link is to be used exclusively for sending traffic to that one ISP.
All of the Tier 1 ISP's provide nontransit service to each other, because at tier 1 there is no such thing as "upstream." This is not people playing stupid, this is how it's done at the top. It's the reason why the major peering points exist.
Any ISP who wants to shut off a peering arrangement for stupid business-o-political purposes is creating a hole in its own connectivity, and therefore shooting itself in the foot, plain and simple.
Level 3/Cogent Deal (Score:5, Informative)
Re:Come on (Score:5, Funny)
The Internet's insecure enough without introducing race conditions into it.
Besides, that's so 1990s. The 21st century equivalent is to yell "Tier 1 ISPs don't care about spam victims!"
Parent
Re:Consider switching to someone less petulant (Score:5, Informative)
As to the notion by another poster of not expecting peering with someone bigger for free forever: 38.0.0.0/8 Class A is Cogent/PSI... how much bigger than being an entire Class A (and then some?!) does one have to be to be considered [ahem] "equal"?
It was a mutual arrangement: they both allowed transit for one another's packets... pretty fair given the size and stature of them both, I'd say.
Parent
Re:Consider switching to someone less petulant (Score:5, Informative)
Yeah, Level 3 is really out of line in my opinion. It's not that they shut down the peering link. That wouldn't be that big of a deal. The traffic would just flow through other providers on less efficient routes. It's not as though every single backbone carrier peers with every other. But I just checked my BGP sessions, and Level 3 is not advertising the Cogent route at all. And you know for a fact that Level 3 is receiving the Cogent route from many of it's other peers. But it appears that they are intentionally filtering out the Cogent route. Which is pretty much not playing by the rules. It's one thing to shut down a peering agreement. It's something else entirely to refuse to accept that route from any of your other peers.
Parent
Partitioning occasionally happens (Score:5, Informative)
I'd bet L3's argument is that they will not provide transit across their AS to Cogent. It's a play that's been made several times before. The first time I know of it being done was in 1995 when Sean Doran pulled this at the CIX-W router, preferring to take commercial traffic via NSFNET or Sprint reseller service. Not only didn't it work, but it caused some immediate political backlash as Sean's action (presumably made without his boss's approval, who was the chairman of the CIX board and took some political grief for Sean's latest stunt) caused several state's to literally drop off the map.
If my memory's right, I think this got pulled again around 1998 timeframe on Exodus by someone like Genuity (I may be wrong about the culprit), only for the higher ups at the culprit to discover they couldn't see half of the world's worthwhile websites and search engines. Much of this was in the transit battle - e.g. if you had consumers, you felt your eyeballs were the value of the Internet and all other ISPs should pay you to get to your consumers, while if you were a content provider, you had the stuff all those consumers were paying their ISP to get to and someone had better pay you for that content.
What can you do about it? Let your ISP know you're not paying them for 80% of the Internet. When UUNET considered pulling this stunt around 1997, I worked for a small software shop that had a couple bonded UUNET T1's and we let them know we were going to drop them the moment they were only selling partial Internet. Then follow through if they do (UUNET backed off). Bilateral agreements are weird things in the world of settlement-free IP exchange, so unless you want a settlement-driven Internet (which will have unusual effects you might not want, like driving a per-packet pricing model), just expect this occasionally and drop those who don't play well with others. When L3 drops customer base, even the Denver boys will figure out their customers aren't happy.
*scoove*
Parent
Re:Partitioning occasionally happens (Score:5, Funny)
I don't think that word means what you think it means.
Parent
Re:Consider switching to someone less petulant (Score:4, Insightful)
In this case Cogent has: as Cogent does, you're surely, surely not a small ISP!
The point is: as most of us are non-multi-homed end users of ISPs--even major ones like Cogent--we're now all subject to the whims of *other* ISPs as to whether or not we can see customers who aren't even hosted by them?!?! Grrrr.
For instance: right now no one on Comcast, Road Runner, or Verizon can see our sites or those of our customers. How does L3 get off doing that?
Parent
Re:Your statement is incorrect (Score:4, Interesting)
As seen on any network, a sufficient degree of inefficiency will result in DoS. How many peerage agreements would have to be cancelled for this to happen? While I freely admit I couldn't compute a number for an effective local DoS vs regional Dos vs global DoS, I would still be extremely interested in making sure it won't happen.
Parent
That's transit (Score:5, Informative)
Parent
Re:WRONG!!! (Score:4, Informative)
Parent
Re:Level3 has a reason to be scared of Cogent (Score:4, Informative)
Parent