Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
The Internet Businesses

Internet Partitioning - Cogent vs Level 3? 450

Posted by Cliff from the balkanization-of-the-'net dept.
slashmicah asks: "Internet partitioning and Tier 1 ISPs are something most people don't know much about (myself included). Today, however, some Slashdot readers might have run into some issues involving these two topics. Cogent Communications and Level 3, both Tier 1 ISPs, are apparently having some 'undisclosed' disagreements, causing an Internet partition by turning-off or deactivating their peering point. Cogent Co. has released a statement explaining their side of the problem, however they have no mention of when the problem will be fixed, or when they will sort it out. This partitioning is a problem because any [single-homed] computers that are connected through Cogent Co, can not connect to [single-homed] computers connected through Level 3. Having spent all day sorting out this problem, I ask Slashdot: Isn't there a better way that the issue of peering can be handled/regulated? If not, does the future hold a scenario in which the Internet is split into several separate networks, only to be connected at the whims of large corporations?"
This discussion has been archived. No new comments can be posted.

Internet Partitioning - Cogent vs Level 3? More

Internet Partitioning - Cogent vs Level 3?

Comments Filter:

  • Interesting scenario, though most likely untrue. (Score:5, Insightful)

    by versiondub ( 694793 ) on Wednesday October 05, 2005 @07:53PM (#13726204)
    While it is always fun to entertain such doomsday scenarios in ones' mind, I don't think that anything like this is possible. Current demands of most large corporations (Microsoft, Apple, any number of others) along with the internet-using public are for a universally-connected internet. Any company that simply creates its own network is going to face a huge revenue loss.

    • Re:Interesting scenario, though most likely untrue (Score:5, Insightful)

      by ewe2 ( 47163 ) <ewetoo@gmail . c om> on Wednesday October 05, 2005 @08:03PM (#13726255) Homepage Journal
      It's possibly on a par with the scenario of countries cutting others off their internet connections. Not that it can't be done, but the repercussions are akin to MAD. Although these days with the effective merger of state and corporate interests anything can happen...

      • Follow the Money! Cogent's lost before (Score:4, Informative)

        by billstewart ( 78916 ) on Thursday October 06, 2005 @04:24PM (#13733774) Journal
        Peering is about two carriers deciding that they both benefit from providing each other with "free" service, splitting the direct costs of the interconnect and not charging for traffic; the alternative is that one carrier is a customer buying service from the other. (There are also some subtle technical differences, but it's basically about the money, and normally peering only routes packets between the customers of two carriers, and not between other carriers that either side also peers with.)

        Cogent's business model is to sell large bandwidths for a low price, usually in multi-tenant office buildings. So they'd drop a fiber into the basement, and sell 100 Mbps ethernet connections to businesses in the building for about the price other carriers would charge for a T1 (that was back when a T1 was typically $1000 instead of $300; I haven't followed Cogent's prices in the last year or two.) Could you expect to get 100 Mbps consistently all the time? Not realistically, but you *could* expect to get lots more bandwidth than a T1 almost all the time, so it was a pretty competitive deal.

        But at the end of the Interent boom, every carrier's finances looked pretty unstable, and a very aggressive business model that depends on getting free peering from big carriers while stealing their business customers looked extremely volatile :-) So does it make business sense for a Tier 1 provider to peer with Cogent as opposed to charging them money for Transit? Maybe, maybe not, and it looks like Level3 used to give them free peering but has changed their mind about it. Not the first time something like that has happened to Cogent - they've been back and forth on this with one or more carriers over the last few years. L3 seems to have decided that there's not enough reason to care about Cogent customers to give them free service.

    • Re:Interesting scenario, though most likely untrue (Score:4, Interesting)

      by Anonymous Coward on Wednesday October 05, 2005 @08:03PM (#13726259)
      If large companies connect to multiple Tier 1s, then they pretty much don't have to care; their customers can still reach them.

      Personally, there are several sites I can't get to from home now. I didn't have any problem getting them from work (UT Austin). I have effectively zero power to rectify this. Annoying.

      Now, if Cogent offered me some way to connect to them for an additional $5/mo... would I?

      Think... if the government allowed an additional $5/mo. for each Tier 1 my ISP (Time Warner) is connected to... my cable modem bill would instantly double.

      That's a scenario that bothers me more than the dissolution of the Net does. Flip side, the Internet would get a whole lot more redundant really quick...
      • From Cogent's side, as linked in the summary:

        "Cogent will offer any Level 3 customer, who is single homed to the Level 3 network as of October 5, 2005,
        one year of full Internet transit free of charge at the same bandwidth currently being supplied by Level 3.
        Cogent will provide this connectivity in over 1,000 locations throughout North America and Europe."

        Not that I really know what that means, or whether their claim that Level3 cut things off really makes Level3 the bad guys. Anyone want to explain for thos

        • Re:Interesting scenario, though most likely untrue (Score:4, Insightful)

          by EndlessNameless ( 673105 ) on Wednesday October 05, 2005 @08:47PM (#13726477)
          Translation: Cogent will let any Level 3 customer who is cut off use their service for one year at no charge.

          This will eliminate any internet performance anomalies for those customers so that they are not affected in a bad way by this issue. It's also a good PR move that might let them grab a few Level 3 customers who are impressed by the goodwill gesture.
        • Anyone want to explain for those of us that don't get it?

          What they are essentially saying is: "We haven't done anything. We haven't made any changes on our side" Level 3 have terminated their connection to Cogent "Without cause". Now, that's probably legal speak on Cogent's side for we haven't got the letter in triplicate yet telling us what the reason is. Or otherwise whoever put up the notification about it doesn't know.

          Now, Cogent may have tried to change the peering arrangement, or Level 3 may have too, one side probably didn't agree, or otherwise an agreement ran out and the switch got flipped. This has happened previously with Cogent in their peering arrangement with AOL. [findlaw.com]

          What Cogent are trying to do is get business from Level 3 customers because Level 3 stopped the connection. Cogent is offering them connections to the Cogent network (And subsequently Cogent's customers) for a year with no fee on the amount of data they put through. That connection itself they will obviously have to pay for, but the customer can connect into (presumably) the closest of any of 1000 points across North America and Europe.

          Now some people are already connected to both Cogent and Level 3. These people won't have any problems as they will be able to go direct into either ISP. These people would probably have never have used the interconnect between Cogent and Level 3 either, unless one of their connections into either Cogent or Level 3 went down.

          I understand this is still rather technical, for a simpler version, take a look through the document that I linked to.

          • According to an article on NANOG, L3 gave Cogent 50 days advance warning to make other arrangements. Cogent didn't, preferring to play chicken and hope it made L3 look worse than Cogent so they'd back down. At this point, both drivers are barrelling down the road at each other, blindfolded, tossing spare steering wheels out the window, but unfortunately for Cogent, L3 is driving a bulldozer...
    • IF the backbones segmented everyone with dual links would keep their UUnet, people without UUnet would get one or pressure their provider to use the UUnet uplink, basically it would be survival of the strongest and UUnet would win.

      Yeah I am just a network guy but I bet I know more about this than the "expert" "predicting" gas prices on CNN.

    • It's happened.... (Score:4, Informative)

      by malakai ( 136531 ) * on Wednesday October 05, 2005 @10:10PM (#13726895) Journal
      What you fail to realize, is it has already happened. At this point in time, I can't access http://www.cogentco.com/ [cogentco.com] and numerous other sites. I'm coming from Time Warners NY Road Runner network. The internet, for me and 10s of thousands of others, is partioned.

      That either network corporation allowed this to occur is without pardon.

      What I'm afraid of, is when this is all over and people realize how singificant it was, the solution to mangers will be "buy service to each, so we never have to worry about being partioned". Which is exactly what both companies would like to see.
      Tracing route to cogentco.com [38.9.51.20]
      over a maximum of 30 hops:
       
        1 3 ms 3 ms 3 ms 192.168.4.1
        2 20 ms 61 ms 14 ms 10.33.8.1
        3 10 ms 14 ms 13 ms pos0-0-nycmnyb-rtr1.nyc.rr.com [24.29.97.93]
        4 11 ms 11 ms 12 ms 24.29.97.25
        5 9 ms 15 ms 17 ms pos2-0-nycmnya-rtr2.nyc.rr.com [24.29.101.253]
        6 15 ms 15 ms 15 ms pop2-nye-P13-3.atdn.net [66.185.141.37]
        7 22 ms 201 ms 222 ms bb2-nye-P1-0.atdn.net [66.185.151.66]
        8 13 ms 13 ms 14 ms pop1-nye-P1-0.atdn.net [66.185.151.51]
        9 17 ms 19 ms 20 ms Verio.atdn.net [66.185.139.150]
        10 20 ms 12 ms 13 ms p16-0-1-3.r21.nycmny01.us.bb.verio.net [129.250.3.48]
        11 25 ms 26 ms 25 ms p16-1-2-2.r21.asbnva01.us.bb.verio.net [129.250.4.27]
        12 * * * Request timed out.

    • Corporate Silent Treatment (Score:4, Interesting)

      by Jazzinjake ( 821562 ) on Wednesday October 05, 2005 @11:52PM (#13727357) Homepage
      In Israel and the Palestinian Territories, this issue has been an ever-present one with regard to the cell phone companies there. While I was spending time there, many activists who were working in Palestine would get two SIM cards, (Cellcom, and either Orange or Jawwal), because Cellcom (Israeli) and Jawwal (Palestinian) wouldn't talk to each other. Furthermore, you couldn't call a Cellcom phone from a Palestinian (Paltel) land line.

      When I asked for an explanation of this, it had to do with a corporate silent-treatment of sorts; because Paltel/Jawwal (the Palestinian telco [reference.com]) was suing Cellcom for licensing infringement and illegal operation, the Cellcom network decided to boycott the Palestinian phone carriers. This caused all sorts of problems for Palestinian society, and the effect was that everyone in Palestinian areas were ditching the local telco and getting Israeli Cellcom cell phones. Jawwal was facing dire times, after their offices were raided by Israeli military and tech imports were prevented because of blanket security concerns.

      For folks on the ground, this was just one more manifestation of the intifada/occupation, even the corporations were going at it.

      More background available here [countercurrents.org], here [66.102.7.104] and here [amin.org].

  • Really easy solution all net-ops already know (Score:5, Insightful)

    by ProfaneBaby ( 821276 ) on Wednesday October 05, 2005 @07:53PM (#13726210)
    Don't expect to peer with people bigger than you for free forever.


    "A true peer relationship is based on the supposition that either party
    can terminate the interconnection relationship and that the other party
    does not consider such an action a competitively hostile act. If one
    party has a high reliance on the interconnection arrangement and the
    other does not, then the most stable business outcome is that this
    reliance is expressed in terms of a service contract with the other
    party, and a provider/client relationship is established"


    Level3 is threatened by Cogent's bandwidth pricing model, and is using it's weight to threaten that model, forcing Cogent to buy transit if it wants to reach its network. THat's how things work: you can't get free bandwidth from everyone, you're going to have to be willing to step up and pay for your link.
  • If they don't sort it out, find a new ISP.

    • Re:Consider switching to someone less petulant (Score:5, Informative)

      by urlgrey ( 798089 ) * on Wednesday October 05, 2005 @08:05PM (#13726272) Homepage
      As a cogent customer, it's *really* not their fault, IMHO. I gather L3 pulled the same stunt with XO last week.

      As to the notion by another poster of not expecting peering with someone bigger for free forever: 38.0.0.0/8 Class A is Cogent/PSI... how much bigger than being an entire Class A (and then some?!) does one have to be to be considered [ahem] "equal"?

      It was a mutual arrangement: they both allowed transit for one another's packets... pretty fair given the size and stature of them both, I'd say.

      • Re:Consider switching to someone less petulant (Score:5, Informative)

        by BeBoxer ( 14448 ) on Wednesday October 05, 2005 @08:51PM (#13726503)
        As a cogent customer, it's *really* not their fault, IMHO. I gather L3 pulled the same stunt with XO last week.

        Yeah, Level 3 is really out of line in my opinion. It's not that they shut down the peering link. That wouldn't be that big of a deal. The traffic would just flow through other providers on less efficient routes. It's not as though every single backbone carrier peers with every other. But I just checked my BGP sessions, and Level 3 is not advertising the Cogent route at all. And you know for a fact that Level 3 is receiving the Cogent route from many of it's other peers. But it appears that they are intentionally filtering out the Cogent route. Which is pretty much not playing by the rules. It's one thing to shut down a peering agreement. It's something else entirely to refuse to accept that route from any of your other peers.

        • Partitioning occasionally happens (Score:5, Informative)

          by scoove ( 71173 ) on Wednesday October 05, 2005 @10:01PM (#13726854)
          Level 3 is not advertising the Cogent route at all.

          I'd bet L3's argument is that they will not provide transit across their AS to Cogent. It's a play that's been made several times before. The first time I know of it being done was in 1995 when Sean Doran pulled this at the CIX-W router, preferring to take commercial traffic via NSFNET or Sprint reseller service. Not only didn't it work, but it caused some immediate political backlash as Sean's action (presumably made without his boss's approval, who was the chairman of the CIX board and took some political grief for Sean's latest stunt) caused several state's to literally drop off the map.

          If my memory's right, I think this got pulled again around 1998 timeframe on Exodus by someone like Genuity (I may be wrong about the culprit), only for the higher ups at the culprit to discover they couldn't see half of the world's worthwhile websites and search engines. Much of this was in the transit battle - e.g. if you had consumers, you felt your eyeballs were the value of the Internet and all other ISPs should pay you to get to your consumers, while if you were a content provider, you had the stuff all those consumers were paying their ISP to get to and someone had better pay you for that content.

          What can you do about it? Let your ISP know you're not paying them for 80% of the Internet. When UUNET considered pulling this stunt around 1997, I worked for a small software shop that had a couple bonded UUNET T1's and we let them know we were going to drop them the moment they were only selling partial Internet. Then follow through if they do (UUNET backed off). Bilateral agreements are weird things in the world of settlement-free IP exchange, so unless you want a settlement-driven Internet (which will have unusual effects you might not want, like driving a per-packet pricing model), just expect this occasionally and drop those who don't play well with others. When L3 drops customer base, even the Denver boys will figure out their customers aren't happy.

          *scoove*

  • Reminds me of... (Score:4, Interesting)

    by mr_z_beeblebrox ( 591077 ) on Wednesday October 05, 2005 @07:56PM (#13726221) Journal
    If not, does the future hold a scenario in which the Internet is split into several separate networks, only to be connected at the whims of large corporations?

    A quote about censorship. Come on, we all know it. The internet will see that as damage and route around it. The very fact that you mention that this affects single homed computers on one or the other network means that even at the onset of this "partitioning" it is ineffective.

    • Re:Reminds me of... (Score:4, Insightful)

      by ADRA ( 37398 ) on Wednesday October 05, 2005 @08:28PM (#13726386)
      We have one large regional telco that interconnects most cities in our area and they don't pay for their peering. There's also city carrier that supplys end-point access to many city businesses. This city carrier has peer access to many different regional networks but they pay for their agreements. The small guy's peers just don't have enough financial interest in the link to give it away for free.

      A data carrier makes their money off the small guys that want to plug into the heavily funded infrastructure that the big guys have spent much time building up. If you have two equally sized carriers with equal data being sent/received to the other network, it makes perfect sense to peer them. Since they both have to bridge the gap to one another's network somehow, its cheaper to go directly to one another.

      Now, lets say the data flow rate isn't symetric. TinyISP and UberISP. TinyISP uses 100Mb/s on UberISP's network, but UberISP only uses 1.2Mb/s on Tiny's network. UberISP wouldn't feel inclined to allow a peering agreement since most of the financial benefit is happening by TinyISP.

      Now with all that said, your argument is only partially correct. Yes, "The internet will see that as damage and route around it" can happen, but it isn't the magical sugar plum fairy granting magical bandwidth to route this traffic. Its Cogent footing the bill to L3 or some other peer in order to get to their intended recipients. Thats if L3 hasn't blocked the Cogent Netblocks as well. In which case, Cogent would be forced to have a peer Source-NAT their traffic if they wanted to reach L3 resources. Thankfully, To my knowledge this crazy scenerio has never occured.

    • Re:Reminds me of... (Score:5, Insightful)

      by schickb ( 629869 ) on Wednesday October 05, 2005 @08:47PM (#13726479)
      Actually it appears that L3 has done more than just drop direct peering with Cogent. L3 seems to be filtering all traffic sourced from Cogent controlled IP blocks. So unless Cognet sets up a NAT arrangement with other peers, there is no way around this problem. L3 is actively blocking Cogent traffic.

      If your company or ISP uses only Cogent for bandwidth, it is currently impossible to reach L3 only connected services. I believe L3 to Cogent is being blocked as well.
  • The customers on each one of the company's networks needs to call them and demand resolution. This is the fastest and most effective method of getting the company to pay attention and fix the problem. If the customers open trouble tickets on this issue it will get resolved. - Dan

  • PITA but move along (Score:5, Informative)

    by Halvard ( 102061 ) on Wednesday October 05, 2005 @07:57PM (#13726230)
    Okay, it's a pain in the ass for customers and others BUT they are businesses that negotiate peering, sometimes with monopoly money changing hands, sometimes without. It's a business dispute and it's not like Germany and France closing roads and making you drive through Belgium. They'll resolve it or lose business. Their's more of a back story we don't know.

    At least it's not like UUNET more than one, some years ago, wanting to charge other Tier 1's per packet for transfer when peering while their traffic they wanted to pass for free. They were a big dog and were trying to make everyone pay. No one did and threatened to or did kill off traffic until UUNET got the sh*t together. But the did try to pull it off more than once.

    • Re:PITA but move along (Score:2, Insightful)

      by branto ( 811992 )
      Miscellaneous people calling up to bitch about not being able to reach Network B from Network A isn't going to accomplish a thing. This is a business decision made by Level3, and they are fully within their rights to do so. Any customers that are feeling the burn of this are in that position because they are not multi-homed. Maybe this "incident" will wake them up to that shortsightedness, maybe it won't... If you are building a network out there, make sure you use 2 Carriers that are either big enough

  • OK, WTF time here (Score:4, Interesting)

    by EndlessNameless ( 673105 ) on Wednesday October 05, 2005 @07:59PM (#13726237)
    For something as essential to the nation as internet service, maybe it's time to consider implementing regulations similar to what electric, water, gas, and telecommunication companies have.

    If my grandma can't check her email for a day, I don't really care that much. If my doctor is consulting with a cardiac specialist over using VoIP (V being either voice or video) concerning an acute health problem then I have a much larger problem with outages. As long as we have important economic or healthcare services running over the internet--which is the foreseeable future--this sort of thing needs to either be avoided or have a pre-planned workaround.

    I guess this explains some of the unresponsive hosts I came across today. And here I was thinking it must be Bob's Worm of the Week.
    • A doctor relying on an internet link for professional purposes better damn well have more than one provider, anyway.

      Dual-homed networks are not affected by a simple depeering.

      • Re:OK, WTF time here (Score:3, Insightful)

        by rkuris ( 541364 )
        Yes they are! They can instantly become single-peered. The whole point of being multi-peered means you have multiple connectivity. One goes down, temporarily, and the others stay up. At this point, this looks like more than just a temporary outage.
      • I'd like to see Level 3 get sued for $100M wrongful death as a result of this action.

    • Re:OK, WTF time here (Score:4, Insightful)

      by gellenburg ( 61212 ) <george@ellenburg.org> on Wednesday October 05, 2005 @08:09PM (#13726286) Homepage Journal
      Okay, first off if your physician is using any IP-based service and ISN'T using a dedicated connection then no one's to blame except the fucktard who set it up in the first place.

      If you keep the following in mind, you will be a much happier person:

      (a) The Internet is not guaranteed to be secure.
      (b) The Internet is not guaranteed to be reliable.

      Anyone making claims to the contrary is a charlatan.

      • Re:OK, WTF time here (Score:5, Insightful)

        by EtherMonkey ( 705611 ) on Wednesday October 05, 2005 @08:59PM (#13726545)

        Okay, first off if your physician is using any IP-based service and ISN'T using a dedicated connection then no one's to blame except the fucktard who set it up in the first place.

        1. Set down coffee cup;
        2. Open front door;
        3. Take deep breath;
        4. Introduce yourself to reality.

        Physicians are subject to the greater economic pressures than any other small business. Insurance companies, government regulations, litigation risks, patient scheduling, qualified and reliable staffing, emergency on-call, and obligitory hospital fundraising contributions. Given a choice between an $1,800/month point-to-point circuit PLUS provider termination and service fees, or a $59/month xDSL for probably 4x the bandwidth, which do you think most will use? If in doubt, give your physician a call and report back.

        • Re:OK, WTF time here (Score:3, Funny)

          by Anonymous Coward

          Physicians are subject to the greater economic pressures than any other small business.

          My doctor totally feels the same way about other medical equipment.

          "Put down the coffee cup and get some fresh air" she tells me, as I struggle against the restraints.

          I shouldn't complain.. she takes time to wipe the scalpels against her lab coat once or twice, and her sock *does* look like a surgical mask if you squint.

          "Physicians are subject to greater economic pressures.." she continues. Her voice travels a t

    • Re:OK, WTF time here (Score:5, Funny)

      by Frogbert ( 589961 ) <frogbert@gmail . c om> on Wednesday October 05, 2005 @08:14PM (#13726321)
      Yes if only there were some mechanisms built into the internet as a whole that would allow for the forwarding of packets through a number of alternate routes if one link were to go down... "Routing" if you will.

      I envision such a system could be seriously robust and would possibly withstand a nuclear attack.
      • Well then mister smarty pants, why can't people going from level3 get to people on the other tier 1?
        • Because they didn't set up redundancy.

          Any good data center can route around the wholesale outage of any one provider without an issue.

          Kind of like asking why you should put your financial records on the computer if a hard drive crash could mean you'd lose everything.

        • Re:OK, WTF time here (Score:4, Informative)

          by silas_moeckel ( 234313 ) <silas@@@dsminc-corp...com> on Wednesday October 05, 2005 @08:47PM (#13726481) Homepage
          OK I do this all day long so let me try and expain how teir 1's work in general yes there are exceptions. Teir 1's peer in a fully meshed network meaning all tier 1's have to have connections to all other teir 1's generaly in a multitude of locations. Teir ones only advertise the routes of themselves and there clients not routes learned from there peers. If you want a full set of routes then you need to pay for your connection. This actualy helps stability on the day to day as all teir 1's connect to all the other teir 1's thus nobody is transiting traffic from one to the other meaning L3 could go off the map but that only affects them and there single holmed clients (single holming is BAD)

          Cogent is not a bonified teir 1 as they still pay for some of there transit.

      • Re:OK, WTF time here (Score:5, Funny)

        by David Gould ( 4938 ) <david@dgould.org> on Wednesday October 05, 2005 @08:44PM (#13726466) Homepage

          I envision such a system could be seriously robust and would possibly withstand a nuclear attack.

        Wow, sounds like something that could be of great strategic military value. I wonder if the DOD would be interested in developing this idea...?

      • Re:OK, WTF time here (Score:5, Informative)

        by shadowmatter ( 734276 ) on Wednesday October 05, 2005 @09:06PM (#13726580)
        The Internet is a power law network, meaning there are some very well-connected routers out there that a lot of the end-to-end transfers through the Internet go through. These are typically the peering points, owned by Tier 1 ISPs. It's not inconceiveable that if two ISPs don't peer with each other anymore, at some level a partition is created.

        When Paul Baran [wikipedia.org] had the task of designing a network that could withstand a nuclear attack, he envisioned a "distributed network". By today's lingo, it's a mesh network where each router is connected to approximately the same number of other routers. But now that routing infrastructure is driven commercially, with tit-for-tat contracts between Tier 1 ISPs, we ended up with what he said was a "decentralized network" -- that is, power law. Not what Paul Baran had in mind. If the underlying topology were his distributed network, you wouldn't be reading this story.

        You can read his paper here [rand.org]. The Internet could withstand one nuclear attack. Several well-placed nuclear attacks? That's debatable...

        - shadowmatter

  • Been dealing with this all day. (Score:5, Informative)

    by QuantumRiff ( 120817 ) on Wednesday October 05, 2005 @08:04PM (#13726264)
    we got an email this afternoon from our provider, who let us know that cogent will be reachable by their second link, which is WilTel. However, the link is slower than the Level3 link. There will also be more traffic being routed through less points, meaning congestion. (and obvious lack of redundancy, if the WilTel connection has problems, no Level3) We have had users complaining about sites being unreachable at random times this afternoon. One of our providers very big customers is the OSU Open Source Lab, home of Drupal, mozilla download servers, master.Kernel.org servers, and many, many others. If your having problems reaching these sites, that is probably why.

  • public peering! (Score:5, Interesting)

    by po_boy ( 69692 ) on Wednesday October 05, 2005 @08:07PM (#13726280)
    Everyone knows what a success MAE-East, MAE-West and the rest of the public peering points have been. Let's build a few more of them! Or, even better, encourage the federal government to get involved. Perhaps spending some of the federal budget on this problem would be a good idea. I think I recall a peering point clause in the constitution somewhere.

    In all seriousness, these private companies will work it out when they realize that their paying customers are pissed and leaving because they're no longer selling very complete connectivity. Just like in the past, it won't take long. If TV has taught me anything, these problems are usually wrapped up pretty nicely in about 28 minutes.

    • Re:public peering! (Score:3, Interesting)

      by TeraBill ( 746791 )
      Public peering points are fine, but some of this may go back to what Johna Till Johnson wrote about in this weeks issue of Network World. She says that the major backbone providers lose money on providing the Intenet backbone links and that the pricing model for this is in need of change. That may be some of what is driving Level 3 in this scenario, since they do carry substantial backbone Internet traffic. Peering points don't really address this since they generally provide a way for more regional peop

    • Re:public peering! (Score:3, Interesting)

      by raddan ( 519638 )
      I think they're counting on the fact that it is extremely difficult for their to switch providers. We have Cogent-- it took months to get them in to light up our building. And the paperwork from our previous provider (Verizon, *shudder*) took months, too. Combine all of this with the fact that often the service agreement is written with your parent company (read: bureaucracy), and "just switching" isn't really an option. Our NY office is switching off of AT&T, but this is after, seriously, nearly tw
  • Peer Relaying should never depend on the goodwill of Tier 1 ISPs. The core backbone routing infrastructure should never have to deal with problems of ISPs. an ISP should never be a carrier, and a carrier should never be a ISP.

    Robert
  • Ironically, because of the depeering, I can't get to it!
    • Cogent's statement:

      Cogent Network Status/DNS Server Status Description:
      Date: 10/05/2005

      Level 3 has partitioned its part of the Internet from Cogent's part of the Internet by denying Level 3's
      customers access to Cogent's customers and denying Cogent's customers access to Level 3 customers. Level 3
      terminated its peering with Cogent without cause (as permitted under its peering agreement with Cogent)
      even though both Cogent and Level 3 remained in full compliance with the previou

  • As a rule... (Score:5, Insightful)

    by jafiwam ( 310805 ) on Wednesday October 05, 2005 @08:14PM (#13726318) Homepage Journal
    I have been kicking around the fringes of the high-speed data stuff for a number of years, and there's one true lesson to be learned;

    Telcos suck.

    ALL of them do in their own special way.

    • Re:As a rule... (Score:3, Insightful)

      by Vegeta99 ( 219501 )
      So what's worse? the Ma Bell behemoth that pretty much invented half of last century and wired the whole damn nation, or millions of little guys, getting into arguments about who can talk to who on the world's largest public-but-not-really network?
  • For some absurd reason I cannot connect to www.cogentco.com [cogentco.com]. I have tried going to it with a web browser and pinging it, but it doesn't connect. The ping returns:

    "Ping request could not find host www.cogentco.com. Please check the name and try again."

    but, when I ping it from nwtools.com, it works just fine [nwtools.com]. I can connect to many other websites, but not to cogent. I am on a verizon DSL, if that makes any difference. Does anyone have any ideas as to what's going on?

  • Ok, I read the wikipedia information on Tier 1 and Peering, so I present the following scenario. What's to stop Cogent or Level 3 from peering with AT&T, who is also peering with the other guy, and having traffic bridged through AT&T? Doesn't a peer of your peer give you access to both peer's networks? I'm wondering this, because I don't think there is anyway all the Tier 1 providers would have disagreements with every other Tier 1 provider at the same time to keep everyone partitioned. In the long

    • That's transit (Score:5, Informative)

      by fuzzy12345 ( 745891 ) on Wednesday October 05, 2005 @08:44PM (#13726465)
      Peering is when you agree to send traffic destined to network X directly to network X via a direct connection between you and X. If you're using X's network to send traffic to Y, that's transit, and X will naturally expect you to pay for the privilege.
    • That's called transit and you pay for it. Peering connections are intended to reach the peer's directly connected customers, which would include the peer's transit customers.

  • This could spell problems (Score:3, Interesting)

    by gcnaddict ( 841664 ) on Wednesday October 05, 2005 @08:18PM (#13726336)
    Lets assume Cogent and Level 3 split up one city (and I know they have done it to at least one place) amongst themselves. Someone happens to be using voip to call 911 while on Level 3, while Cogent is maintaining the 911 system's voip call receiver, preventing the voip 911 call from ever reaching it...

    wow they could both be sued for huge sums of money...

  • Cogent's message (via NANOG) (Score:5, Informative)

    by wayne ( 1579 ) <wayne@schlitt.net> on Wednesday October 05, 2005 @08:27PM (#13726380) Homepage Journal
    I can't get to Cogent's website, but according to a NANOG post, this is whatit says:

    Cogent Network Status/DNS Server Status Description:
    Date: 10/05/2005

    Level 3 has partitioned its part of the Internet from Cogent's part of the Internet by denying Level 3's customers access to Cogent's customers and denying Cogent's customers access to Level 3 customers. Level 3 terminated its peering with Cogent without cause (as permitted under its peering agreement with Cogent) even though both Cogent and Level 3 remained in full compliance with the previously existing interconnection agreement.

    Cogent will offer any Level 3 customer, who is single homed to the Level 3 network on the date of this notice, one year of full Internet transit free of charge at the same bandwidth currently being supplied by Level 3. Cogent will provide this connectivity in over 1,000 locations throughout North America and Europe.

    Cogent is committed to an open Internet. The existing interconnection facilities between Level 3 and Cogent remain intact. Cogent hopes that Level 3 will reactivate these connections, restoring a full level of service to their customers.

    For more information about the sales offer, please contact the numbers listed below.
    NORTH AMERICA: 1-877-875-4432
    ANYWHERE ELSE IN EUROPE: +33 (0)6 1101-7382

  • Level 3's official statement (Score:5, Informative)

    by The AtomicPunk ( 450829 ) on Wednesday October 05, 2005 @08:30PM (#13726394)
    "Recently, certain peers have been disconnected from their direct connection to the Level 3 IP network. Some disconnected peers may elect to block access to certain IP addresses as a result of the disconnection. If a peer elects not to restore connectivity to the Level 3 network through alternative means, customers seeking continued access to the Level 3 network should make alternate arrangements."

    They're saying Cogent is intentionally not advertising routes to them via other providers, presumably because they're upset about not having a peering agreement in place. Anyone affected by this presumably needs to harass Cogent.

    http://ws.arin.net/whois?queryinput=AS174 [arin.net]

    • Re:Level 3's official statement (Score:4, Interesting)

      by xsonofagunx ( 902794 ) <sonofagun.superheronumberone@com> on Thursday October 06, 2005 @12:31AM (#13727543) Homepage
      They're saying Cogent is intentionally not advertising routes to them via other providers, presumably because they're upset about not having a peering agreement in place. Anyone affected by this presumably needs to harass Cogent. I think you read that wrong, or maybe I am. What I'm seeing is We [L3] disconnected someone [Cogent]. Cogent, you might be a little pissed, and decide not to allow our traffic to go over your lines (go figure...). If a peer [Cogent, again] doesn't find another peer to route their traffic through, then their customers won't be able to access our network. If you're a customer of that peer [Hi, Cogent!] you're screwed, so hook up with us, or one of our partners. I'm reading it as L3 trying to dick Cogent around. Maybe that's just me. I don't care either way so far, haven't been denied access to anything yet. We should take this as a note that we should start the internet over, as a totally distributed network. Every country has a main link (of course, there are several of these for every decently sized country, or you'd kinda be defeating the distributed part...), and every state/province has their own peering to that link, and cities down from there etc. - all government owned, and agreed internationally (UN perhaps... tho they've been kinda missing with Bush in office...) that under no condition can they ever intentionally sever the link. I would even think that, government owned and funded, it would be almost a trifle for 10mbit internet connections to be had by all. Of course, the current providers would still be offering residential and business service, but the money would be going to the government, to help cover the costs of the network (along with federal subsidies), instead of Tier1 ISPs.
  • INANE (I'm Not A Network Engineer) but, as I understand it, peering has been a problem for awhile. Even if network providers do peer, as a customer your quality of service suffers. If your packets are sloughed off to some other provider's network and then there are problems downstream, for example, that provider has little incentive to help you since you're not one of their paying customers. Or something like that.

    In any event, that's why companies like InterNAP [internap.com] offer multi-homed services [internap.com].

  • Simple! (Score:2)

    by fm6 ( 162816 )
    This partitioning is a problem because any [single-homed] computers that are connected through Cogent Co, can not connect to [single-homed] computers connected through Level 3.
    In other words, a server shouldn't be singled-homed. Any other questions?

  • Seen it before in Australia (Score:5, Interesting)

    by Airconditioning ( 639167 ) on Wednesday October 05, 2005 @08:37PM (#13726433) Journal
    Reminds me on an incident that happened in Australia a couple of years back. Telstra and Optus were pretty much owned all the links outside of Australia, but Telstra lost their major one in a shipping incident. (Sharp anchors?) With nowhere for their data to go they rerouted everything through Optus to let them handle it.

    Optus didn't appreciate that and promptly blocked all data between themselves and Telstra. Customers with Telstra were pretty much screwed because they couldn't contact anything and with their network going nuts even sites within Telstra sucked a lot. Still, for a couple of days there, it was two halves of an internet available in here. Was amusing to watch really.

  • Easy to Fix (Score:5, Funny)

    by Waffle Iron ( 339739 ) on Wednesday October 05, 2005 @08:37PM (#13726434)
    There's an easy technical fix to this problem: Start a nuclear war at the location of this peering point. Then by design the Internet will route around that area, and communications will be reestablished.

  • Some thoughts on this mess (Score:5, Informative)

    by Anonymous Coward on Wednesday October 05, 2005 @08:38PM (#13726441)
    First, I think that Level-3 is within it's legal rights in terms of dealing with Cogent, but is probably in trouble with it's customers. I am a customer of Level-3 and of Cogent (in the same facility). When I buy IP transit from Level-3, I am not buying "part of the internet". This peering issue places 45+ Million IP addresses out of reach of the Level-3 network (and vice versa). Level-3 did not notify me that they were making this type of change. There is nothing on Level-3's website that even implies that everything is not hunky dory. If you buy a Level-3 line today, will they disclose to you that you are not connecting to the entire internet. I know I am being a little niave here, but not disclosing such a large change of policy is unconscionable.

    Second, it is dishonest for Level-3 to blame Cogent for this exclusively. Level-3 had a peering arrangement with Cogent for a long time. If you look at Level-3's interconnection policy page:

        http://www.level3.com/1511.html [level3.com]

    It still looks like Cogent and Level-3 could peer under these terms. It was Level-3 that pulled the plug, not Cogent.

    What is really annoying is that this is only traffic from Level-3 to Cogent, not to other parts of the internet. Level-3 wants money for Cogent customers to connect to Level-3's network but does not understand that this is a two-way connection and that Cogent's customers and Level-3's customer both benefit from this equally.

    Up until this point, I was very happy with Level-3. They run an excellent network and I pay top-dollar to be on it. This blatent disregard for the impact on their customers is a diservice to their customers, to their reputation, and only begs for regulation.

  • No Rules. (Score:4, Interesting)

    by Nethead ( 1563 ) <joe@nethead.com> on Wednesday October 05, 2005 @08:42PM (#13726454) Homepage Journal
    The internet has no government, no constitution, no laws, no rights, no police, no courts. Don't talk about fairness or innocence, and don't talk about what should be done. Instead, talk about what is being done and what will be done by the amorphous unreachable undefinable blob called "the internet user base." -Paul Vixie

  • This is bad. Very bad. (Score:5, Insightful)

    by hernick ( 63550 ) on Wednesday October 05, 2005 @08:47PM (#13726478)
    I have reviewed all information available at this time, including discussion threads on many sites more specialised than Slashdot. This is bad. Very bad. Right now, there are millions of Internet users with partial connectivity.

    But the action of Level3 is not merely an inconvenience to end users; it is hurting a great many small businesses, badly. There are thousands of small businesses that depend on single-homed Internet connectivity and that cannot afford dual-homing. There are dozens of low-cost datacenters that provide single-homed bandwith to tens of thousands of servers.

    As we speak, the livelyhoods of thousands of entrepreneurs are being threatened. Many people depend on being able to offer internet services to any peer on the net. But today, Level3 has changed the rules of the game, and have split the Internet into two somewhat isolated internets.

    This is happening on a very large scale. Sure, most of the affected people and businesses are going to get through it just fine. But given the sheer scale of the Internet, a small percentage of those depending on full connectivity will not escape this ordeal unscathed.

    You can be sure that a few small businesses will close because of this, the reputations of a few persons will be damaged, and there will be a few bankruptcies - all because of Level3's evil actions. You won't hear about it in the media - nobody cares about such small-scale damage. But the damage is already done, and it is getting worse with every passing hour.

    I urge you to join me in a five-minute hate against Level3 and all that their evil discriminative ways stand for. While Cogent is widely recognized for its shitty cut-rate network, they are the good guys here. In the past few years, Cogent has been a major driving force for lowering bandwith costs. Level3 is fighting back, and they long for the days where they charged 5000$/mbps. I say: down with Level3 !

    • How to Complain (Score:4, Informative)

      by randalny ( 227878 ) on Wednesday October 05, 2005 @08:58PM (#13726541)
      You can call:

      720-888-2518 (Level3 Investor Relations)

      and complain.

      Or call 877-453-8353 (Main customer service number).

    • Re:This is bad. Very bad. (Score:4, Informative)

      by pla ( 258480 ) on Thursday October 06, 2005 @10:28AM (#13729771) Journal
      There are thousands of small businesses that depend on single-homed Internet connectivity and that cannot afford dual-homing.

      Explicit dual-homing directly with tier-1s, no.

      But I do IT at a medium-small business, and we have a fairly simple solution to this.

      We get our internet service from a multi-homed tier-2.

      Problem solved.

      I agree, this seems very, very bad - Not so much the situation itself, but the fact that, at any random moment, ALL the tier-1s could arbitrarily choose to end their peering agreements, turning the internet quite literally into the Bushism "internets".

      But for any individual customer, they do have the power to prevent one such schism from limiting their connectivity simply by their choice of an ISP.


      As an aside, I have to admit I don't really understand why Level-3 would do this. Regardless of the dominant direction of traffic between the two networks, every packet sent still has two sides involved - One a paying customer of Level 3, and one a paying customer of Cogent. So which side should pay for which direction? The question doesn't even make sense - A peering agreement improves both sides.

  • Peering 101 (Score:5, Informative)

    by Anonymous Coward on Wednesday October 05, 2005 @08:50PM (#13726500)
    The short version goes something like this:

    Provider A and Provider B peer, be it public or private, normally they do this in several places and alternate who pays for the circuit, etc. Now, under normal circumstances, they both push enough traffic from one to the other to justify this mutual payment plan. However, in some cases, you find that B is either intentionally dumping traffic into A thinking A won't notice, or A discovers that its sending so little traffic to B in comparison to the amount B is sending to A that its not worth the continued cost.

    When the first sort of thing happens, it usually gets resolved -REALLY- quick, that sort of behavior is not tolerated and will result in B getting de-peered by A (and potentially others once the abusive behavior is discovered and known) exceptionally quick unless B can show that it wasn't done knowingly or intentionally.

    When the second instance happens .. well .. you get what happened today (I'm making an educated guess here based on what I know of the two carriers involved). A decides that spending 30 grand a month for what is a very lopsided bandwidth agreement is no longer economically feasible or reasonable. They go to B and say 'look, we're not doing this anymore, we're basically paying a hell of a lot of money every month for you to send a ton of traffic to us, and we don't send much of anything to you. You can either pay for all (or some larger portion of) the circuits, pony up some $$ per megabit, or we'll just cut it off at the stub and be done.'

    Based on Cogent's 'oh poor us' post from this morning, I'm leaning towards them having given L3 the finger when L3 said 'look, this isn't equitable, we're going to have to re-arrange the money'.

    YMMV of course, but I'm betting I'm not terribly far off.

  • More Regulation != Solution (Score:5, Insightful)

    by dada21 ( 163177 ) * <adam.dada@gmail.com> on Wednesday October 05, 2005 @09:25PM (#13726690) Homepage Journal
    I can't believe I'm hearing a call for more regulation, even U.N. control. The lack of rational thought and disregard of unintended consequences amazes me.

    The Internet has flourished without much control, run by Both large and small businesses for one reason: profit. Information is free yet its distribution is profitable.

    If we give government control (taxation, censorship and worse(, we'll see less freedom.

    Why did this jinx happen? Because the top tier providers aren't making a profit. But their calls for support go unheard, so they found a way to make it news.

    When businesses that rely on the infrastructure paid for by private industries, they have high expectations. But they're not paying for that infrastructure!

    Trust me, no one wants to bifurcate the Internet. Its a ploy to show a problem that needs to be solved. You will Never see it done for control, censorship or monopoly powers. You'll only see it when consumers don't pay for what they use. See California's old electric company that was forced to sell energy at a loss. They went bankrupt.

  • Comment removed (Score:4, Insightful)

    by account_deleted ( 4530225 ) on Wednesday October 05, 2005 @09:27PM (#13726702)
    Comment removed based on user account deletion

  • Not to put too fine a point on it (Score:3, Insightful)

    by drsmithy ( 35869 ) <drsmithy&gmail,com> on Wednesday October 05, 2005 @09:35PM (#13726739)
    If not, does the future hold a scenario in which the Internet is split into several separate networks, only to be connected at the whims of large corporations?

    But that's exactly what the Internet is (well, sometime's they're connected at the whim of educational institutions, but the whole point of the internet is that it's a network of networks).

  • NANOG Archives (Score:3, Informative)

    by Alcemenes ( 460409 ) on Wednesday October 05, 2005 @09:54PM (#13726825)
    Folks on the NANOG list are discussing this rather vigorously at the moment. You can follow the thread here: http://www.merit.edu/mail.archives/nanog/2005-10/ [merit.edu]

  • Level 3 = Failing Business = Retribution (Score:5, Informative)

    by Anonymous Coward on Wednesday October 05, 2005 @10:02PM (#13726864)
    I worked there and its a deathmarch shop for IT and network people; they treat their peopel like crap, and leaving there was the best thing I ever did.

    The arrogance of Jim Crowe [workign on his 7th manion and 8th large layoff at Level 3] and Kevin O'Hara (President, CEO) is only matched by the jailbird Bernie Ebbers. They only reason they have yet to decalre bankruptcy and liquidate thier debt (and clear away their bad business model with a fresh debt-free start) is that all their Omaha cronies have tied up money in the company stock, which would be flushed.

    That they would resort to stunts like this against companies that undermine their pricing is not surprising. Level 3 have amassed BILLIONS in debt that they cannot service at current pricing levels, while Cogent and other more nimble competitors can sustain operations and drain Level3 into bankruptcy. So Level 3 execs do what arrogant desperate people do: lash out.

    Level 3 is playing the "Sampson" card - if they cant make people price it their way, they will take the internet down with them.

    And they did this trying to kill XO and now Cogent. Watch for more until they finally admit their business model is a failed one, and they declare bankruptcy, wipe the debt, and then begin to price lower and rake in the profits that their debt service is now eating.

    • Re:Level 3 = Failing Business = Retribution (Score:5, Informative)

      by EQ ( 28372 ) on Wednesday October 05, 2005 @10:38PM (#13727023) Homepage Journal
      You may have a point here - remember that Level 3's main exec, Jim Crowe, was an exec under Ebbers at Worldcom. Plus I've heard from a one or two people where I work who worked there and they pretty much back up what you said: Level 3 treats the tech people liek pieces of facility, not people down there in Broomfield (I worked tech up in Boulder a while back at Adaptec, and saw them build those buildings up on the hill).

      Putting on my investor's cap, and taking a quick look at financials, its obvious that Level 3's burn-rate on cash, and billions in debt is not looking good for them if they cannot start generating both higher margins and more revenues. Neither one individually will save them at this point. The debt service is eating what EBITDA revenue they have coming in faster than they can produce it. And with companies like Cogent undercutting them Level 3 is dying; it seems the only question now is how much interconnectivity they will destroy in fits of pique like this.

      I think you may also be right on another point, after considering it and runnnign the numbers: if Level 3 were to reorganize in bankruptcy court, dump the current shareholders, turn the debt holders into stock holders to ditch the debt, then they would probably be very profitable at even lower pricing levels. After all, that is what a lot of their competition has done. If they do that, Level 3 will cut the throats of every company out there, and make a bundle doing it, free-market style. Pretty interesting scenario.

      But first they have to drop the stockholders, and from your post, it sounds like cronyism is a big factor, so its only going to happen when there has been far too much damage to Level 3 as a company. Thats a shame, because looking at their web site, they have some good ideas, but the wrong time and place for them.

      Thanks for the post AC (wow an AC that actually said something useful!)

  • Transit vs. nontransit service (Score:5, Informative)

    by IGnatius T Foobar ( 4328 ) on Wednesday October 05, 2005 @11:14PM (#13727180) Homepage Journal
    I can't believe nobody has yet bothered to explain the difference between transit and nontransit service.

    When you buy Internet bandwidth from your ISP, you are getting transit service. This means that you can use the link to send traffic to that ISP and to other ISP's upstream from it.

    Nontransit service means that the link is to be used exclusively for sending traffic to that one ISP.

    All of the Tier 1 ISP's provide nontransit service to each other, because at tier 1 there is no such thing as "upstream." This is not people playing stupid, this is how it's done at the top. It's the reason why the major peering points exist.

    Any ISP who wants to shut off a peering arrangement for stupid business-o-political purposes is creating a hole in its own connectivity, and therefore shooting itself in the foot, plain and simple.

  • Level 3/Cogent Deal (Score:5, Informative)

    by Anonymous Coward on Thursday October 06, 2005 @12:09AM (#13727430)
    I work for Time Warner Cable and we use Level 3. I can't talk about the details of what's going on but the notes on the ticket opened for this issue and the conference call/bridge have covered alot. Alot of people in high up positions are working for find us a work around until a solution can be found. Level 3 is still routing traffic TO Cogent but they are not routing it back at this time. I was working on this issue all night. My suprise to come home and find it on Slashdot. I sure hope they can come to some kind of agreement soon (contract was terminated at 5:30am 10/5/05) but from what's been said thus far it's not looking like it's going to be a quick fix.

  • Inexcusable (Score:4, Interesting)

    by OldManAndTheC++ ( 723450 ) on Thursday October 06, 2005 @12:45AM (#13727594)
    A poster on Broadband Reports [broadbandreports.com] said it best:
    If hackers or terrorists caused this kind of disruption in the internet the government and media would freak out.

Slashdot Top Deals

Two can Live as Cheaply as One for Half as Long. -- Howard Kandel

Close