Ask Slashdot: How To Diagnose Traffic Throttling and Work Around It? 251
Aguazul2 writes "I live in Peru and use OpenVPN to connect to my own Linux VPS in the UK for non-live TV. Recently the VPN connection has slowed to a crawl (5% previous rate). Further investigation shows that all connections to my VPS from Peru (even HTTP) are equally slow, whilst the rest of the 'net seems fine. My VPS host says they do no traffic shaping, and connections from Germany to the VPS are fast. This leaves the NSA and Telefonica (Movistar) as suspects. Could the NSA be slowing all VPNs to/from South America because of Snowden and Greenwald? A traceroute shows traffic going through domains with NYC in their name — are my packets being indefinitely detained in transit? Or maybe it is Telefonica and their Sandvine traffic management? Either way this certainly isn't network neutrality, especially on an 'unlimited' plan. Is there a way to tell for certain who is throttling me? If Telefonica have throttled traffic to/from that one IP address, what options do I have to work around it? It seems that separate connections are throttled independently, so can I multiplex over many UDP ports without having to hack OpenVPN myself? This is really frustrating, especially with two untrustworthy parties on the route. I wonder, is this kind of mess the future of the internet?"
Passive monitoring is all that is necessary (Score:3, Informative)
You are seriously lacking basic data telecommunications experience. All government tapping is span port based. This means that it is passive, not active, so there is no latency involved.
Traffic Intercept and VPN (Score:5, Informative)
Years ago I worked on a broadband remote access server and one requirement we got was to support lawful traffic interception. Basically all law enforcement wanted was a copy of all of the packets. Packets are not slowed down or stopped by this process.
In my case the hardware was just not capable of doing what was needed but there was plenty of off the shelf hardware that could be installed in the network to provide the filtering and packet mirroring needed.
It is possible that one of the VPN's upstream providers is running into congestion. One of the best ways I have found is to use traceroute. At one time I was getting unusable Internet connectivity through AT&T after they acquired my local cable modem network from @Home. It took them many months to discover that throttling all aggregate upstream traffic to 128Kbps is a bad idea. As much as people bitch and moan about Comcast, it is lightyears better than anything I got through AT&T. In this case, traceroute clearly showed where packets were getting delayed and dropped, which was one of the routers inside AT&T.
Unfortunately, for a VPN this is much more difficult since the Internet hops are hidden via the tunnel.
There are many different ways to tunnel traffic. If the tunnel is Microsoft's PPTP protocol then it's not very secure. If on the other hand it is using IPSec then it should be a lot more secure. There are also other tunneling protocols that do not specify any encryption, i.e. MPLS.
Re:Passive monitoring is all that is necessary (Score:5, Informative)
SPAN port is the Cisco name for port mirroring.
see
http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a008015c612.shtml [cisco.com]
The Internet is a (messy) series of tubes (Score:5, Informative)
My office Internet connection recently went from about 30Mbps down to 1.5Mbps, then back to 50Mbps a month later. No explanation, and speed tests to our ISP all came through at full speeds. We only saw problems on routes going outside our city and headed west. There were also a few inaccessible sites, but those were in very specific local areas. Ultimately, the best guess anyone could come up with is that a network to the west of our city had some routing problems.
We weren't the only customers to complain about a slowdown, but our ISP couldn't really do much about it. The Internet is made up of many networks working together, and sometimes shit happens. I wouldn't jump so quickly to assume it's non-neutral throttling or the NSA, when it could just be a careless guy with a badly-aimed backhoe. Give it some time, see if it improves, and if not, it may be time to move your VPS.
As an aside, you're likely going through New York because that's how you're reaching Europe to get to your UK-based VPS. Many transatlantic cables end in New York City [cryptome.org], mostly because the stock market pays dearly for the few nanoseconds of lower latency.
Re:NSA (Score:5, Informative)
Indeed.
But, even in China where they do filter the internet, there isn't any real throttling that goes down, the main thing I saw when I was there was abysmal latency. It would have the effect of killing of websites that weren't blocked, when the website was expecting to load dozens of scripts from various other servers. Each one would have up to 2.5 seconds of latency attached. And yes, that is seconds, not often, but there were a few times when my ping was measurably with a human timer.
More likely, this is some sort of broken link somewhere along the way that's resulting in the traffic being slowed.
Some suggestions (Score:5, Informative)
Some more info would be appreciated. So, here's the basics of a few things you can do to make sure it really is the network*. First use iperf on the client and server. Test it on both the tunnel interface and the WAN interface. Second, use top via a separate ssh session. Make sure OpenVPN isn't eating all your CPU or memory. Lastly, what provider are you using? Lately the default Debian build that Edis.at gave me needs an ifconfig up/down every other day.
I've had a similar problem when using my own VPS as an HTTP proxy via OpenVPN. It turned out, the proxy application was crap. Allowing the machine to route packets and using it as a default gateway for all traffic fixed the problem, or at least worked around it.
Now. If it really is blocking, there are a couple of ways around it. The more complicated ones involve using some other VPN application. When dealing with more than one client, that rapidly becomes annoying. A simple one is using an SSH connection as a SOCKS proxy for your browser. It's not elegant, but it works. Another way is to mask your OpenVPN connection by encapsulating the UDP or TCP packets. Once again, SSH port forwarding works, but that's a TCP solution. socat was designed to do things like that, so it seems like a good choice. Finally, there's Ping Tunnel. It embeds traffic in ICMP packets.
Whoever is throttling you might detect one or more of these, but they're probably using some sort of signature based detection. Just about anything that requires a command line should get through.
Remember, since you are technically savvy enough to roll your own, you are the one percent. Good luck, and please let us know how it goes.
*I know you're probably familiar with all of these things. Just assume that I put this section here for those who aren't.
Re:is the NSA taking candy away from kids too? (Score:5, Informative)
Did you not watch the video from the Dot Com mansion raid? lol
Traceroute is to mainly fix routing problems today (Score:3, Informative)
Many ISP's perform what is known as ICMP rate limiting. Traceroute and Ping both use this ICMP protocol *i'm not going to get into semantics* where as you start traversing the internet past your internet service provider your pings and such to any point along the path have a high chance of being dropped due to this. The only way to see your actual latency is using a host-to-host ping. From your source destination to your final destination. Traceroute acts as sending a ping to each and every hop in between the source and final destination (assuming the TTL doesn't expire or somebody's carrier firewall just doesn't' start letting replies come back through, ie, multiple * * * responses but still able to reach your end destination), they are in no way obligated to reply properly and or in a timely fashion to your Ping request. During the early days of the internet we didn't have many of the problems that we have today and these tools worked flawlessly during this time and really could tell you where your latency is (these tools still function normally in a local lan if you are not doing any "crazy" firewalling tactics). This is no longer the case with ping an traceroute.
IN EXTREME CASES it may be possible to route around other carriers using private tunnels, It's not something your average joe will not likely be able to accomplish without multiple services across the country or paying for some sort of service to do so. AKA you are a business with $$$$. There are instances where it can be done, but are few and very far in between.
If your ISP only has 1 way out to reach specific destinations which are having problems. Provide them traceroutes showing them good responses AND bad responses from when and where you are seeing the problem. The only thing a carrier is going to care about is your "average" response time in milliseconds, not your "maximum" response time.
Re:NSA (Score:5, Informative)
But the NSA isn't in the business of routing data; it's in the business of mirroring data. This means that you get something like:
source
|
router A
|
router B --> NSA
|
router C
|
destination
So if router B is up to the task of sending the signal down a fixed path as well as whatever BGP indicates, there should be no slowdown. If it isn't, that's going to be a constant issue, not something that varies. It's either good enough for the volume of data it is exposed to, or it isn't. There's no analysis happening at the router, and the NSA isn't doing stateful inspection.
More likely a QoS issue by some stateful router in the hop chain, or even a corrupted BGP table.
Re:NSA (Score:5, Informative)
In finance we use them for performance monitoring and debugging. You have machines with CDMA or GPS time sources logging packets captured from passive taps on each side of your switches, routers, servers, etc. It lets you produce very accurate and detailed latency statistics. Also when things go wrong you have an exact record of everything that went in or out on the network to help you reproduce and fix it. Admittedly we don't actually get NICs with the transmit functionality removed, but the passive taps prevent anything transmitted from going anywhere, so we get a similar effect.
Re:NSA (Score:5, Informative)
You absolutely need to trap the packets in real time in order to actually break the VPN connection open so you can get at the actual payload (cleartext, post-decrypted) data within the stream. The initial cryptographic handshake has to be captured, in order for them to peel it open and get inside.
You can't do that days later, when all you have is an encrypted stream of bits.
I'm not sure I follow: how would capturing the cryptographic handshake help with "peeling open" the VPN connection? The handshake itself is secure: OpenVPN running in TLS mode (the most common mode) exchanges symmetric keys using an ephemeral Diffie-Hellman key exchange, with the key exchanged signed by the server's RSA key. Both client and server are authenticate to each other using certificates, so they can be sure that there's no man-in-the-middle. Unless one knows how to solve the Diffie-Hellman problem and one has a sensible configuration (i.e., sufficiently large DH parameters and RSA keys, good choice of symmetric cipher, etc.), capturing the cryptographic handshake doesn't really gain the attacker anything.
EFF's Switzerland Network Testing Tool (Score:4, Informative)
The OP mentions Sandvine: the EFF has a tool called Switzerland. [eff.org]
Is your ISP interfering with your BitTorrent connections? Cutting off your VOIP calls? Undermining the principles of network neutrality? In order to answer those questions, concerned Internet users need tools to test their Internet connections and gather evidence about ISP interference practices. After all, if it weren't for the testing efforts of Rob Topolski, the Associated Press, and EFF, Comcast would still be stone-walling about their now-infamous BitTorrent blocking efforts.
Developed by the Electronic Frontier Foundation, Switzerland is an open source software tool for testing the integrity of data communications over networks, ISPs and firewalls. It will spot IP packets which are forged or modified between clients, inform you, and give you copies of the modified packets.
Switzerland is designed to detect the modification or injection of packets of data traveling over IP networks, including those introduced by anti-P2P tools from Sandvine (widely believed to be used by Comcast to interfere with BitTorrent uploads) and AudibleMagic, advertising injection systems like FairEagle, censorship systems like the Great Firewall of China, and other systems that we don't know about yet.
Re:NSA (Score:5, Informative)
Anything from a higher classified system that is to deliver data to a lower classified system,
The projects I worked on called it a data diode. [wikipedia.org]
Re:The Internet is a (messy) series of tubes (Score:2, Informative)
And remember that fiber optic runs at 2/3rds the speed of light;
Wrong.
Right [wikipedia.org]. Light doesn't travel in a straight line through fiber optic. Sorry man, you and the mods are wrong on this. Physics FTW.
In what fucked-up case would you have 45deg incidence?
I suppose in the "fucked up case" where light bounces repeatedly along a very long tube at varying angles, and the sum average would, after a few dozen reflections, quickly start averaging out to... wait for it... 45 degrees. Sorry if you were asleep in science class.
When nitpicking, try to not make yourself look like a complete fucking idiot, but hey.
I got my prefix wrong. You got the whole theory wrong.