Redundant Internet Access? 78
Supp0rtLinux asks: "In order to meet uptime requirements and SLAs, we decided to get redundant T1's with BGP. We already had two Cisco 7200 routers and a T1. After the ISP turned up the additional circuit and we tested everything on our end, all seemed fine. But when the CO lost power and the generator failed, we had no access for 16+ hours. This prompted some investigations which revealed that yes, we did in fact have a redundant T1 with BGP setup and local redundant routers with separate UPS... on our side. However, on their side both our feeds were plugged into the *same* switch which was on the same PDU which happened to be in the same CO and was on the same sonet. And they were charging us for redundancy! Six month later, we have a truly redundant BGP setup. Each feed goes to separate CO's with the primary to the local one. This makes for separate physical switches, separate power, and we have confirmed we're on physically separate sonets. Now, the only true single point of failure is the physical cabling in the street, but in CA that doesn't get damaged very often. To those of you on Slashdot who know what I'm talking about: are your circuits truly redundant? What have your experiences in network redundancy been? How have you gotten past the sales guy to a tech that knows what redundancy really means? Have you been able to prove your redundancy? Have you found yourself paying for something that you weren't really getting?"
Re:Not there yet (Score:5, Interesting)
Very concerned (Score:5, Interesting)
My issues from there:
1. How do you convince an ISP to bring a feed in from another CO? Distance is a huge problem--they don't want to run it.
2. How do you know what the ISP has on their end, UPS's, generators, etc? Should that be part of the SLA? Or should you demand a tour of their facilities to see where your wire goes?
3. How can you coordinate two seperate ISP's for automatic redundancy? I suppose with a LinkProof box or something. And how do you know they aren't coming through the same telco CO?
4. Should you pay to have them manage the lines and router configurations in a 24/7 scenario? Or does it work well enough to have them do the initial install and then let it run?
5. Finally, what's a resonable cost for this redundancy?
I have some more projects that will be requiring this type of setup. Am interested to hear any opinions and recomendations from experience from fellow slashdotters......
Thanks much!
-m
One Example (Score:3, Interesting)
Part of the expense was justified by cost savings using VOIP between the stations and the operations centers.
Re:Another completely different approach (Score:3, Interesting)
If anything, my "real" motive is to keep people from putting servers in-house. If your office has the same "pipe", "power", and "security" as a good co-lo, then you spent too much money building it.
After all, there are millions of square feet of unused co-lo at rock-bottom prices just begging for more space-heaters (er, servers) to keep the resident space-heaters (er, servers) company.
The system at one of my previous place of employ (Score:4, Interesting)
We had four T1s -- two from MFS and two from Bell. Of the four T1s, two (one MFS and one Bell) went to one NSP in Santa Clara, and the other two went to a different vendor in Oakland.
We even had physical plant diversity -- the Bell loops came from cable that ran along Stevens Creek Blvd, and the MFS fiber came up from the street that ran behind us. Outside of the building burning down, we were bulletproof.
Ran three years without a single minute of downtime.
My crowning glory in network design. Never again did I work for an employer who was willing to put their money where there mouth was for reliability.
Re:Redundant should be 100% (Score:2, Interesting)
Actually you may not have to go that far. For redundancy around here we can go to the power company. Our local power company [mnpower.com] has a networking affiliate [enventistelecom.com] with their own completely separate network.
Good luck trying to get the local phone company to admit of such a situation exists in your area though.
Got you beat! (Score:3, Interesting)
It is well known that even if at any given time you are making use of different sonet rings, circuits get shifted around based on demand, and you could end up being rerouted onto the same circuits without any notice. They only way to know is to wait till a problem occurs, and see if it impacts more than one connection.
Redundancy needs verification (Score:2, Interesting)
Real redundancy costs real money.
I work in a professional Colo facility in Denver, and we are fully redundant in all systems. Once it leaves your box, there's two of everything. Dual power to the box, dual network connections (Backbone: Dual OC-12 lines from different providers, running to different boxes in the POP) Dual climate control systems, dual generator rooms with independent fire control systems...I could go on, but you get the idea. I'm on the graveyard shift, and things run so smoothly I get a lot of reading done. And with 150k square feet of building, a lot of walking as well.
It's not cheap. But if you really need redundancy, it's cheaper to rent space in professional facility than it is to try and be compliant without one.
TowerStream (Score:4, Interesting)
# Chicago, IL
# New York, NY # Greater Boston, MA
# Greater Providence, RI # Newport, RI
# Westerly, RI
TowerStream [towerstream.com] may be something to look into. I use them as our primary connection at the office - they are far cheaper than a traditional T1 ($350/mo for 512k, $500 for 1.5mbit, they can handle around 5GBit max I believe).
True line-of-site is not required, a reflected signal is usually sufficient. An external flat-panel antenna about 6 inches tall and wide is required, however. With ours setup on the roof, we get 0% packet loss, and have had no problems through heavy snow, rain or thunderstorms.
I have occasionally had connection issues, where the wireless modem has needed to be power-cycled. I suspect, however, this is simply due to it overheating
Joking and Seriously (Score:4, Interesting)
if you want to find out about "redundancy" find out what they do in the military.
Cost is another matter....
Re:Another completely different approach (Score:3, Interesting)
That's a pretty good solution for low cost. I don't know the details of how he set up the server to do the detection and failover, but he has confirmed that it works by disconnecting the DSL line from it and seeing it automatically switch the connection and keep serving on the cable line. It's a Windows Server 2003.