Grading Telco & ISPs During the Blackout of 2003? 31
alt_cognito asks: "Our company runs natural gas generators here in Novi MI and when the power went out we didn't miss a beat. Nine hours later, our telco blinked and our T1 service went down despite the lines being run to different locations and ISPs (UUNet, LDMI). Service did not return until power had returned to the upstream offices. I was under the impression that these locations would be run by similar power generation. How did your telco/ISP perform?"
Re:K5 (Score:1, Offtopic)
Seems scoop isnt all what they thought. Guess they shold have chosen slashcode.
Dial Up and DSL (Score:1)
Re:Dial Up and DSL (Score:1)
Escape from New York (Score:1)
If the problem originated in Cleveland, I still think the problem had to do with Lewis and Oswald.
Re:Escape from New York (Score:2)
Hmmm, bet they let some of the test subjects out.
Same situation (Score:4, Informative)
Re:Same situation (Score:2)
I give WorldCom a big fat F too, right up their asses, they suck.
Re:Same situation (Score:2, Informative)
Re:Same situation (Score:2)
Back when I used to work at an ISP (like back in 1994), I was a big MCI fan because they were way more reliable than Sprintlink. My bad feelings about them mostly stem from them laying me and my whole department off (I had been there in a development-related position for 4.5 years) when they had some financial troubles due to mismanagement. Makes you think when they whack 100 or so of their brightest employees without blinking to save some cash that their executives blew.
Just fine (Score:1)
carrier lost
Re:Just fine (Score:1)
We (or at least I) pay > $0.13/kWh
If the state uses too much power we get rolling blackouts.
But, actual power failures are quite rare (and brief) at least in my experience.
Anyway, back to the topic at hand:
my ISP (Adelphia) was up the entire time. The internet in general seemed somewhat flaky, with lots of random things running very slowly or not at all.
Of course, I would have been shocked if a California provide
Sympatico fine (Score:3, Informative)
At my company the IBM eSeries servers were backed by a smart UPS that showed 17 hours remaining, 15 seconds before shutting down everything in cold blood. It was all scandisks booting back up there on Windows 2000 machines.
I maintain some small servers with no UPS in a few locations, and while one Solaris server crapped out, you had to manually do the fsck thing, all the FreeBSD servers were back up as the power came without a hitch. I have to learn to setup Solaris on sparc so it fsck itself without asking for an input.
Re:Sympatico fine (Score:1)
"What's that you say, you claim to be a maritime company supporting maritime jobs but your stuff is all in another province?"
Verizon is vucked... (Score:2, Informative)
What does that have to do with a blackout? (Score:2)
Since your telco is SBC, that should be a daily occurance.
Re:What does that have to do with a blackout? (Score:1)
I've had more than one repair guy tell me that when SBC came in they changed the rules that govern when they replace bad cabling. I think all 9 planets
Bet your local telco is the problem. (Score:5, Informative)
It is likely that you will never find out exeactly what happened but, from what you describe it sounds like; the lights went out, the local Central Office(CO) where the local loop for your T-1s went onto UPS backup or generator and after a few hours the UPS or the generator ran out of juice. Once the CO ran out of juice your T-1s went dead. So, you lost connection with the ISPs. More than likely the ISPs themselves never blinked.
The only way to avoid this problem it to use two different local loop providers which is usually going to be hard to find unless you are in a large metropolitan area. The other thing to do is get the local loop lines from different COs which will be like pulling teeth from your local telco.
Planning and preparedness, unfortunately, does not guarantee against failure.
Re:Bet your local telco is the problem. (Score:3, Insightful)
It's possible that your line doesn't come straight from the CO, rather it hops through a remote equipment location (a "hut" or "vault"), which has batteries but no generator.
Huts, vaults, and c
Quebec Side (Score:1)
Best Catastrophic Event of August 2003 (Score:1)
MCI (Score:2)
Not exactly what you wanted to know (Score:2)
But a funny story none-the-less...
Back during the great auckland power crises [wired.com] of 97, my ISP was Binary Brothers, a now extinct ISP. They were a great ISP, run by a few guys who knew their stuff.
Turns out the owners were physically located on the coremandle peninsula (about 4 hrs drive from Auckland), while their servers/modem racks, etc were located in the heart of auckland CBD.
The power in auckland blinked out, and as did my net connection (I was located outside the blackout area). I rang 'em up and
human error (Score:1)
But today at our colo facility which has battery backup and generators and can last for quite sometime without city power went down this afternoon. Why? An electrician turned off our breaker and shutdown our entire rack.
never underestimate human error.
--ajay
Verizon of Manhattan was fine (Score:1)
As a result, I could have used my laptop for some late night entertainment, but I decided to use my lanterns and read a book instead.
What was really interesting was that you could see the stars in the sky for the first time probably since 1977's blackout. Oh sure, you can see a few, but I mean see a LOT of them.
all good here in SE Ontario (Score:2)
ISP: Cogeco Cable: No interruptions
Power: NiagaraMohawk: No interruptions (I kid you not. My neighborhood hasn't lost power for a second during the outage).
Phone (both landline and cellphone): No interruptions
Work:
ISP: RoncoNet: No interruptions
Power: unsure (prolly NiMo + generators): No interruptions
Dialup: D-. Nextel: F. SBC: A (Score:2)
Later Thursday night I tried to get back online from my laptop, and my local ISP (who uses Megapop for dialin ports) would answer and connect, I'd get an IP address, but my packets went nowhere. Tracert stopped at hop 1.
So, knowing that the phone system was up, I simply dialed out of the affected area. Calling a POP