Ask Slashdot: How To Diagnose Traffic Throttling and Work Around It? 251
Aguazul2 writes "I live in Peru and use OpenVPN to connect to my own Linux VPS in the UK for non-live TV. Recently the VPN connection has slowed to a crawl (5% previous rate). Further investigation shows that all connections to my VPS from Peru (even HTTP) are equally slow, whilst the rest of the 'net seems fine. My VPS host says they do no traffic shaping, and connections from Germany to the VPS are fast. This leaves the NSA and Telefonica (Movistar) as suspects. Could the NSA be slowing all VPNs to/from South America because of Snowden and Greenwald? A traceroute shows traffic going through domains with NYC in their name — are my packets being indefinitely detained in transit? Or maybe it is Telefonica and their Sandvine traffic management? Either way this certainly isn't network neutrality, especially on an 'unlimited' plan. Is there a way to tell for certain who is throttling me? If Telefonica have throttled traffic to/from that one IP address, what options do I have to work around it? It seems that separate connections are throttled independently, so can I multiplex over many UDP ports without having to hack OpenVPN myself? This is really frustrating, especially with two untrustworthy parties on the route. I wonder, is this kind of mess the future of the internet?"
I use longer words (Score:4, Funny)
Try breaking free of the binary straightjacket. I transmit all my data in ternary and it is untraceable and unstoppable. This gives me unlimitered bandwidsh to post my brilliant world-changing essays and thoughts on Slashdort, the Facebook of the Internet!
Re: (Score:3)
Ah, using the evil bit I see... don't move, unmarked black choppers will be with you shortly.
Re:I use longer words (Score:5, Insightful)
Re:I scoffe at your "homor"! (Score:5, Funny)
Who gave a slashdot account to that computer trained to tell jokes?
Re:I scoffe at your "homor"! (Score:5, Funny)
Didn't you know? Slashdot is a large Turing Test system. Most of the participants are AIs.
Interestingly, the most promising test results are with the "First Post" trolls. Apparently nobody can imagine that an AI could be that stupid.
NSA (Score:5, Insightful)
I've had a client I provide consulting for suggest that their poor connectivity is also in some way due to the NSA. People need to understand that it is paramount to the NSA that they are covert. They do not need to do real-time processing of the data: that is only necessary for filtering. It suffices for them to simply capture raw data for later analysis or decryption as necessary. Of course capturing data does not result in any slowdown or other noticeable effects. It does not make any sense whatsoever for the NSA to be slowing or otherwise blocking connectivity, as that is counterproductive to the acquisition of intelligence data.
It's just amusing to me to see NSA as the scapegoat of the day for any quirk anyone experiences related to computers or connectivity in general.
Re:NSA (Score:5, Funny)
It's just amusing to me to see NSA as the scapegoat of the day for any quirk anyone experiences related to computers or connectivity in general.
No one ever got fired for buying... I mean blaming the NSA. :)
Re:NSA (Score:5, Funny)
Re:NSA (Score:4, Insightful)
Re: (Score:2)
an incorrect theory, because port mirroring (Score:5, Interesting)
That may have been their theory, or it may have been they wondered if US gov was intentionally slowing VPN connections from that part of the world.
If the theory was that capturing data would slow it down, the answer is "no". For that, you'd use port mirroring. Where a switch or router would normally take data in on one line and output it on another, you set it to accept data on the one line and output it on TWO others simultaneously. The data still flows at the same speed. It just flows to two locations separately - the intended recipient and the government.
Re:an incorrect theory, because port mirroring (Score:4, Insightful)
But I wouldn't think the extra few ms of latency would slow the data flow by 95%. In fact, I don't think it'd even be noticeable to the naked eye except for exceptional circumstances like gaming.
Re: (Score:2)
I think the submitter's theory was that the NSA man-in-the-middle data capturing would slow down the connection.
I know that the NSAs monitoring (as described so far) is passive. My theory is that they would quite happily throttle all 'suspicious' high-bandwidth encrypted streams if they could get away with it. And they have been getting away with quite a lot recently. If a few choke-points like that develop on the internet where encryption == slow, then what kind of an internet is that? I hope we don't get to that point.
Re: (Score:2)
Because the FBI is doing US domestic surveillance. The NSA is doing non-domestic surveillance. And I don't think Peru is part of the US.
Re:NSA (Score:5, Informative)
Indeed.
But, even in China where they do filter the internet, there isn't any real throttling that goes down, the main thing I saw when I was there was abysmal latency. It would have the effect of killing of websites that weren't blocked, when the website was expecting to load dozens of scripts from various other servers. Each one would have up to 2.5 seconds of latency attached. And yes, that is seconds, not often, but there were a few times when my ping was measurably with a human timer.
More likely, this is some sort of broken link somewhere along the way that's resulting in the traffic being slowed.
Re: (Score:3)
That's an easy assumption, but it's not correct. The sites that are blocked will just timed out because the DNS won't connect you, but most of the sites that I observed to be effected would load from time to time, they just took forever to load. And once they did load, there was nothing about China and nothing that they're usually blocking. Sites like the NYT do get blocked, but sites that just carry Chinese lessons and other innocuous content don't normally get blocked.
The main reason for the latency is th
Re:NSA (Score:5, Interesting)
Indeed. When working for a company that sold telecom and networking IP blocks, we received more than one request for the receive part ONLY of an Ethernet MAC. The companies that enquired did not make test equipment, but were known for secrecy and selling to the US government. What possible reason does such a company have for an Ethernet MAC that receives only?
Re:NSA (Score:5, Informative)
In finance we use them for performance monitoring and debugging. You have machines with CDMA or GPS time sources logging packets captured from passive taps on each side of your switches, routers, servers, etc. It lets you produce very accurate and detailed latency statistics. Also when things go wrong you have an exact record of everything that went in or out on the network to help you reproduce and fix it. Admittedly we don't actually get NICs with the transmit functionality removed, but the passive taps prevent anything transmitted from going anywhere, so we get a similar effect.
Re: (Score:2)
Hi,
I also work/worked in that space - apart from operational reasons, it's (passive data capture) is also used for various trading reasons.
Cheers,
Victor
Re:NSA (Score:4, Interesting)
What possible reason does such a company have for an Ethernet MAC that receives only?
Anything from a higher classified system that is to deliver data to a lower classified system, for example you need to get data from extremely sensitive military satellites to battle commanders in the field and it needs to happen in real time, you can't have total network separation. Then you generate a one-way feed where there is physically no possible way for anyone to connect to the feed and hack themselves backwards through routers into the satellites. And of course you put a ton of code review, surveillance and logging on the sending system to make sure it doesn't send more than it should, but that's not relevant to this discussion. So there's a lot of valid reasons for the military to buy this besides the NSA.
Re:NSA (Score:5, Informative)
Anything from a higher classified system that is to deliver data to a lower classified system,
The projects I worked on called it a data diode. [wikipedia.org]
Re:NSA (Score:5, Interesting)
It suffices for them to simply capture raw data
Lol. You have no idea what suffices for them.
And even if "capture raw data" suffices - if the bandwidth to their traffic caputring room [wikipedia.org] is at capacity, they very well may tell the upstream switches to slow down so they can "capture [all] raw data".
Until there's enough transparency; it's at least as reasonable to blame the NSA for using lots of bandwidth to cause conjestion as it is to blame all those movie-pirates for using all the bandwidth.
Re:NSA (Score:5, Interesting)
That may be true for passive surveillance (http traffic, emails, IMs), but most-definitely not for VPNs, as in this specific case.
You absolutely need to trap the packets in real time in order to actually break the VPN connection open so you can get at the actual payload (cleartext, post-decrypted) data within the stream. The initial cryptographic handshake has to be captured, in order for them to peel it open and get inside.
You can't do that days later, when all you have is an encrypted stream of bits.
Re: (Score:2, Redundant)
That may be true for passive surveillance (http traffic, emails, IMs), but most-definitely not for VPNs, as in this specific case.
You absolutely need to trap the packets in real time in order to actually break the VPN connection open so you can get at the actual payload (cleartext, post-decrypted) data within the stream. The initial cryptographic handshake has to be captured, in order for them to peel it open and get inside.
You can't do that days later, when all you have is an encrypted stream of bits.
They only need to know that the citizen is using an encrypted VPN. This implies that they have something to hide and are therefore a suspect, and actual evidence no longer matters.
Re: (Score:2)
Re: (Score:2)
what is an encrypted VPN? I thought all VPNs were encrypted?
You can use GRE or IPIP tunnels to make a VPN which will be completely unencrypted. I normally use IPSEC over the top of that where encryption is required.
Re:NSA (Score:4, Insightful)
Re: (Score:3)
He's using a new form of encryption. I bet even the NSA won't be able to crack that one.
Re: (Score:2)
At least he didn't use "ur", which can only reasonably be pronounced as the first syllable of the word "urgency" (and preferably with an elongated "r" sound as in "arrrrrrr matey").
I chooce to pronounce "u" as just the first sonant of "urgency". I know it's only slightly better than writing "u" instead of "you", but it makes it sound more like the mouthbreathing way it was probably intended.
Re:NSA (Score:5, Insightful)
Yeah, NSA tech guy, we really don't think you should be listening in on our business plan and buying up stock before we announce the acquisition...
Lotta non-poilitical reasons why a person might want to encrypt communications. I do have something to hide AND I'm not doing anything wrong.
Re:NSA (Score:5, Informative)
You absolutely need to trap the packets in real time in order to actually break the VPN connection open so you can get at the actual payload (cleartext, post-decrypted) data within the stream. The initial cryptographic handshake has to be captured, in order for them to peel it open and get inside.
You can't do that days later, when all you have is an encrypted stream of bits.
I'm not sure I follow: how would capturing the cryptographic handshake help with "peeling open" the VPN connection? The handshake itself is secure: OpenVPN running in TLS mode (the most common mode) exchanges symmetric keys using an ephemeral Diffie-Hellman key exchange, with the key exchanged signed by the server's RSA key. Both client and server are authenticate to each other using certificates, so they can be sure that there's no man-in-the-middle. Unless one knows how to solve the Diffie-Hellman problem and one has a sensible configuration (i.e., sufficiently large DH parameters and RSA keys, good choice of symmetric cipher, etc.), capturing the cryptographic handshake doesn't really gain the attacker anything.
Re:NSA (Score:5, Insightful)
It does not make any sense whatsoever for the NSA to be slowing or otherwise blocking connectivity, as that is counterproductive to the acquisition of intelligence data.
That's generally true. The NSA is competent. But not all government agencies are... and not all of those agencies work for the United States either. So I can't conclusively tell you (nor can anyone else) that it isn't the result of some law enforcement action that's causing your internet connection to behave strangely. What I can tell you, is that it's pretty unlikely.
The more likely explanation is QoS being implimented that targets either based on IP, subnet, port, or content. Content-aware QoS is pretty rare, but it is out there. Alternatively, it could be a misconfigured router, or an oversaturated link. Traceroute and measuring the latency during TCP handshakes to various ports both to the destination of interest and elsewhere would help identify this. Lastly, it may not even be network-related; it could be the server itself that is slow, or the application it is running on. In today's 'cloud all the things!' service model, there are all kinds of weird performance glitches due to complex interactions within the cluster. For example... several data centers bought the (server) farm during the last addition of a leap second, as circuit breakers tripped out due to sudden load spikes.
The fact is, without a lot more information from the OP, this question simply can't be answered. It could be one of dozens of different things... all we can do is give odds on the likelihood of what it might be... and I'd put the NSA pretty far down the list. The 'NSA Effect' is the same thing happening now in the media that caused people to beat the crap out of random muslims out of 9/11, or jerkwads in Florida to shoot black kids -- perception and media attention creates a new social reality. Social reality is not based in actual reality, however... but it's stuff like this that gives rise to all kinds of prejudices -- racism, sexism, religious persecution... it's ironic that the NSA's surveillance policies are based on such faulty logic ... and now they are the victim of it as well. Ah, but I digress... short answer: Your router doesn't need a tin foil hat.
Re:NSA (Score:5, Interesting)
I seem to recall that Kim Dotcom realized he was being spied on long before the raids due to seeing his latency spike and seeing that traffic was being routed an odd way.
I think you overestimate the NSA's competence. Snowden was a leak waiting to happen. Read Bruce Schneier's analysis.
Re: (Score:3)
The 'NSA Effect' is the same thing happening now in the media that caused people to beat the crap out of random muslims out of 9/11, or jerkwads in Florida to shoot black kids -- perception and media attention creates a new social reality. Social reality is not based in actual reality, however... but it's stuff like this that gives rise to all kinds of prejudices -- racism, sexism, religious persecution... it's ironic that the NSA's surveillance policies are based on such faulty logic ... and now they are the victim of it as well. Ah, but I digress... short answer: Your router doesn't need a tin foil hat.
The "NSA effect" introduces doubt. There is someone watching my traffic, and they would probably meddle with it if they could get away with it and had the resources. What if all 'suspicious' encrypted streams were slowed at various choke-points on the internet in the name of national security -- i.e. "if we can't see what you're sending then your traffic will be penalised"? Wouldn't they love to do that? What kind of internet would that be? I don't think that is entirely paranoia.
Re: (Score:2)
Re:NSA (Score:5, Informative)
But the NSA isn't in the business of routing data; it's in the business of mirroring data. This means that you get something like:
source
|
router A
|
router B --> NSA
|
router C
|
destination
So if router B is up to the task of sending the signal down a fixed path as well as whatever BGP indicates, there should be no slowdown. If it isn't, that's going to be a constant issue, not something that varies. It's either good enough for the volume of data it is exposed to, or it isn't. There's no analysis happening at the router, and the NSA isn't doing stateful inspection.
More likely a QoS issue by some stateful router in the hop chain, or even a corrupted BGP table.
Re: (Score:3)
In order to monitor effectively, they need to make sure the is no alternative route, or technology, for the data which they cannot also effectively monitor. This was precisely why they tapped the fiber at the AT&T facilyt in "Room 641" in San Francisco. It's also why telecom companies are forbidden, by law, from using technologies that do not have law enforcement monitoring capacity built in.
So, in your diagram, that "router B" needs to be a core router which cannot evaded by alternative routing or load
Re: (Score:3)
It does not make any sense whatsoever for the NSA to be slowing or otherwise blocking connectivity, as that is counterproductive to the acquisition of intelligence data.
Normally I would agree with you, but since "THEY" (the generic they) are forcing Presidential planes to land, detaining boyfriends, seizing electronics, what makes you so sure some arm of the US government isn't deliberately slowing or blocking binary transfer streams in an attempt to stop Snowdens 400gigabyte cache of information from spreading ?
(I suspect his Peru ISP is lying to him, but still I consider the possibility of intentional interference).
Re: (Score:2)
deliberately slowing or blocking binary transfer streams in an attempt to stop Snowdens 400gigabyte cache of information from spreading ?
For some reason, my torrents on Comcast (CentOS, Fedora, Mint) are running at full speed, except for those three. transmission-daemon FWIW.
Re:NSA (Score:5, Funny)
Re:NSA (Score:5, Funny)
Yes.
Better stop using Facebook--in fact, the entire internet--now. Discuss this feeling of yours with your doctor, and then use all the free time you'll have to learn scrimshaw and grow tomatoes.
Re: (Score:2)
Do you think this could be the NSA snooping on my facebook and pushing me to buy audiobooks that will contain subliminal messages to hate Snowden and freedom?
You're starting to sound angry again. Maybe you need another treatment.
Back to re-education camp; to write lines; repeat after me. "Terrorists want to kill us. The NSA protects our freedom. Without NSA snooping, Snowden, the 9/11 hijackers, and other terrorists will destroy America."
Re:NSA (Score:5, Insightful)
Given that they did, in fact, cause poor connectivity for critical west coast trunk connections at AT&T with the "bent fiber optic" taps installed in Room 641A, it seems that interfering with a typical customer's bandwidth is not their highest priority. While there are ways in many environments to tap data surreptitiously and at full bandwidth, such setups are often quite expensive and instead done with less sophisticated, possibly slower devices and bandwidth throttled to allow full data capture.
I've certainly seen this in industry when monitoring a network problem, where we throttled the bandwidth so our monitors could keep up and analyze who was abusing our systems.
You are all major assholes (Score:2, Interesting)
Ok, so the same people that say it can't be piracy because no one was deprived of their DVD give a free pass to "The NSA is capturing the data"??
They didn't capture the data, because if they did then when did they release it? It wasn't like they were tagging an antelope and then let it go at some later time. Why do you give a stamp of approval that the "NSA captures data" as if they held it hostage at Gitmo and wouldn't let the datas go unimpeded.
I
Re: (Score:3)
Not any more.
Re: (Score:2)
Not any more.
Yes, exactly. How long before passive monitoring becomes active manipulation of streams. "Wouldn't it be great", they say, "if we could stop the terrorist communications from arriving". "Wouldn't it be great if we could stop the Guardian sending all our secrets to/from South America". I know the difference between passive monitoring and messing with packets, but I don't think I'm being too paranoid to think that some part of US cyber defence might think it a good idea to slow down VPNs as an 'emergency mea
Re: (Score:3)
It's just amusing to me to see NSA as the scapegoat of the day for any quirk anyone experiences related to computers or connectivity in general.
It's just amusing to me to see people like you indicate it's impossible, when it's provably h
Re: (Score:2)
I don't think anyone's saying it's impossible the NSA is responsible for slowing down his VPN (and only his VPN). I think what they're saying is it doesn't make the Top 10 list of possible suspects because it would be fairly trivial for the NSA to get his VPN data without doing so (and thus tipping him off).
If he was Wikileaks/Snowden's guy in South America I'd believe the NSA could convince somebody to send his data through their servers, and that they might screw with him on purpose. But that's not what h
Re: (Score:3)
Re: (Score:2)
But this guy is in Peru. He's talking to the UK, not Boise, and the transatlantic cables to the UK run through places like NYC all the time. So the NYC servers don't exactly scream sub-optimal data path.
Moreover his idea isn't that the NSA is screwing with him personally, it's that the NSA is screwing with all VPNs on an entire continent.
I don't think it's technically possible for the NSA to intercept all VPNs from South America, and only VPNs from South America, in such a way that all VPNs from South Ameri
Re: (Score:2)
Re: (Score:2)
They do not need to do real-time processing of the data: that is only necessary for filtering.
They do real-time processing, though.
However, real-time processing does not need to introduce any more delays than mere capturing would do, namely almost zero in both cases if the traffic is unencrypted. To be fair, a MitM attack on a VPN probably would introduce a delay.
It's just amusing to me to see NSA as the scapegoat of the day
It's not amusing.
Re: (Score:2)
"It's just amusing to me to see NSA as the scapegoat of the day for any quirk anyone experiences related to computers or connectivity in general."
Well, you know how it is in IT. Anyone who has computer-related skills, last seen in the vicinity of the machine, when it stopped working, is suddenly suspect. They're just experiencing what everyone else in IT has experienced for decades...and getting a dose of their own medicine. The paranoia they've created, plus the problems those backdoors / other tricks have
Re: (Score:3)
Traffic was slow on the drive home yesterday.
Damn NSA!!
Passive monitoring is all that is necessary (Score:3, Informative)
You are seriously lacking basic data telecommunications experience. All government tapping is span port based. This means that it is passive, not active, so there is no latency involved.
Re: (Score:2)
It's hypothetically possible that ISPs might be influenced to route traffic to physically pass through a NOC where taps are in place, the extra hops causing latency.
Though I do think OP is jumping the gun just a bit.
Re: (Score:2)
I do think OP is jumping the gun just a bit.
+1, Understatement.
Re: (Score:2)
It's hypothetically possible that ISPs might be influenced to route traffic to physically pass through a NOC where taps are in place, the extra hops causing latency.
In that same vein, it may be that the NSA is equipped to record/decrypt certain types of data, but not others.
For example, suppose they could decrypt normal traffic just fine, but not VPN traffic. So to discourage VPN use, they make it unpractical for normal use by slowing it to 5% of the speed it should work at. They could break it entirely, but they want to remain covert, so they just slow it to a crawl.
The user knows that its still working -- so he doesn't set to "fix" it (either by using different por
Re:Passive monitoring is all that is necessary (Score:5, Informative)
SPAN port is the Cisco name for port mirroring.
see
http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a008015c612.shtml [cisco.com]
Traffic Intercept and VPN (Score:5, Informative)
Years ago I worked on a broadband remote access server and one requirement we got was to support lawful traffic interception. Basically all law enforcement wanted was a copy of all of the packets. Packets are not slowed down or stopped by this process.
In my case the hardware was just not capable of doing what was needed but there was plenty of off the shelf hardware that could be installed in the network to provide the filtering and packet mirroring needed.
It is possible that one of the VPN's upstream providers is running into congestion. One of the best ways I have found is to use traceroute. At one time I was getting unusable Internet connectivity through AT&T after they acquired my local cable modem network from @Home. It took them many months to discover that throttling all aggregate upstream traffic to 128Kbps is a bad idea. As much as people bitch and moan about Comcast, it is lightyears better than anything I got through AT&T. In this case, traceroute clearly showed where packets were getting delayed and dropped, which was one of the routers inside AT&T.
Unfortunately, for a VPN this is much more difficult since the Internet hops are hidden via the tunnel.
There are many different ways to tunnel traffic. If the tunnel is Microsoft's PPTP protocol then it's not very secure. If on the other hand it is using IPSec then it should be a lot more secure. There are also other tunneling protocols that do not specify any encryption, i.e. MPLS.
Re:Traffic Intercept and VPN (Score:4, Interesting)
When AT&T was providing cable Internet to me, there was a time when my IPSEC VPN did not work. The VPN apparently connected, but data traffic never made it though. Other people complained, but AT&T claimed they were doing nothing to VPNs. Using tcpdump at both ends, I could see that the media (udp/500) was not getting though while the AH and ESP packets (required to set up the connection) were getting though. Clearly AT&T was blocking VPNs, but in such a way that it would not be obvious to the average user what was wrong. Pure evil.
Re: (Score:3)
In the end what OP wants to be answered, is the question whether his provider throttles traffic. The odds are, provider does this.
To test, you don't need traceroute necessarily.
Are all connections to the VPS slow? Only VPN or also http, smtp, ssh, etc? Then there certainly is an issue on that specific connection.
Try to find another server within the same data centre to connect to (same route for the packets to get there), see what happens.
Find a server in a different location, same protocols, and see what h
Re: (Score:3)
Except for the very last step, OP did this all already according to description. Conclusion should be quite clear, and a call to ISP complaining about this issue would be appropriate.
Calling Telefonica is not a solution to anything, unfortunately. They can't even get billing right. They obviously do have some technical people somewhere, and mostly they do a pretty good job, because uptime is good and we haven't seen many problems otherwise. The customer-facing people though ... what can I say ... Until you learn how to make an official complaint and involve the regulator, you can't even get basic billing and contract problems solved. The chance of making progress with some obscure techn
Re: (Score:3)
When I was living in Colombia telefonica bought up much of the government run landline/internet business.
.768 down, 128 up for like 70 US a month. Then the government mandated that min speed for anyone was 2 meg, so we got a bump. But they throttled youtube and my vpn traffic.
I had telephone and DSL through them
I did not mind to much because my office had a ten megabit fiber connection, so any thing that needed a heavy payload I just did at work. Though it did suck
Re: (Score:3)
Paratrace (or whatever its descendents might be called these days) might yield a bit more accurate information. Both rely on interim hops playing by ICMP rules. Many of the highly utilized hops have at least throttled ICMP responses to conserve CPU, so you need to be careful to not just firehose test packets.
OP might probably calm down and remember not to attribute to malice what can be explained by stupidity. A simple change in fragmentation, buffering depth, or the ever misguided per-flow fairness AQM
I find... (Score:2)
The Internet is a (messy) series of tubes (Score:5, Informative)
My office Internet connection recently went from about 30Mbps down to 1.5Mbps, then back to 50Mbps a month later. No explanation, and speed tests to our ISP all came through at full speeds. We only saw problems on routes going outside our city and headed west. There were also a few inaccessible sites, but those were in very specific local areas. Ultimately, the best guess anyone could come up with is that a network to the west of our city had some routing problems.
We weren't the only customers to complain about a slowdown, but our ISP couldn't really do much about it. The Internet is made up of many networks working together, and sometimes shit happens. I wouldn't jump so quickly to assume it's non-neutral throttling or the NSA, when it could just be a careless guy with a badly-aimed backhoe. Give it some time, see if it improves, and if not, it may be time to move your VPS.
As an aside, you're likely going through New York because that's how you're reaching Europe to get to your UK-based VPS. Many transatlantic cables end in New York City [cryptome.org], mostly because the stock market pays dearly for the few nanoseconds of lower latency.
Don't believe your provider... (Score:2)
is the NSA taking candy away from kids too? (Score:2, Interesting)
why would they care about your pirated or whatever TV?
a super secret US intelligence agency that employs some of the smartest mathmatecians in the world is going to care about people's pirated movies instead of tracking down our enemies so the military can kill them
Re:is the NSA taking candy away from kids too? (Score:5, Informative)
Did you not watch the video from the Dot Com mansion raid? lol
Re: (Score:2)
They have to track every byte of every peer to peer transaction, in case someone is using modified clients to communicate. $
Also, they are ordered to retain every single phone sex conversation between non US persons, in case blackmail material is required some decades later for commercial or diplomatic purposes.
Re: (Score:2)
They care about what you send over that connection. They do want to know. As long as you're watching the BBC, they won't care much.
But as soon as you switch to jihad-TV, they will care, and to know whether you do so, they'll have to keep on monitoring your BBC broadcast stream, to make sure you're not secretly switching networks. Or as soon as you switch to some encryption method resulting in them only seeing random bits, they also start to care about your connection.
And with the suspect j-word twice in thi
Re: (Score:2)
They care about what you send over that connection. They do want to know. As long as you're watching the BBC, they won't care much.
Well my VPN is encrypted so they don't know what I'm transferring, although I don't use it for anything sensitive. I guess if I turned off all the encryption and it was still throttled then that would eliminate the NSA as the culprits.
Re: (Score:2)
It won't eliminate the NSA. It only suggests that there is no man in the middle doing decryption/encryption. NSA won't work as MiM; that'd be too easy to detect; and that's also not necessary for listening to a signal (regardless of whether they can decrypt it).
Re: (Score:2)
I assume you mean't "*isn't* going to care". And you have some starry eyes, my friend... you seem to think that the NSA must be like a James Bond movie. But once corruption becomes the operating mindset (and it has), it all ends up being ab
From an ISP network engineer (Score:5, Insightful)
If you are a US ISP, it is required that you have monitoring in place. If you don't want to hamper your entire infrastructure while doing so, you get a bunch of taps and install them all over your network. One very good provider for this is Gigamon. Taps do not add any latency in your traffic. They are completely invisible to all other network devices. Traffic shaping (throttling) is done by the source typically but can be done at the destination ISP. Basically, your connection is assigned a Package in the Shaper. The packages determine how fast each classification groups of traffic are allowed to go. Classifications are determined by whoever manages the shaper for that ISP. Shapers can also dynamically change the speed you are allowed to have for a classification group based on bandwidth used, time used, and volume of traffic.
If you are not throttled from Germany to your home but are from Peru to your home, chances are you are throttled from your ISP in Peru. It is typical for transits to cross borders, so your traffic going through NYC is normal. BGP (the routing protocol of the internet) determined that to be the best path. This is mostly managed, but is still fairly dynamically determined by the routing protocol.
Course of action: Switch ISPs, get a new IP address (if they are not very good at configuring a shaper this will work, otherwise not), try a proxy, stop using it for a day or more and it will go away (temporarily most likely). This is done dynamically in the shaper. There is not some dude with his finger on a 'throttle' button. Everything is automatic. Just figure out the how their throttling deterministic state diagram works and you can avoid throttling. Most likely they are throttling you because of your volume of use. It costs a lot for transit access and you are using more than most others by streaming through a vpn.
Re: (Score:2)
Switching ISPs is one option.
SSRR (Source Routing) will also work.
If you think it's because of the encryption, switch to using PPOE and see if the problem resolves itself.
Also, you can do TCP active probing to see which intermediate hop(s) actually have the slowdown; this is the same techniques used to detect black hole routes for when an ISP blocks ICMP packets, and you can use PMTU discovery.
Re: (Score:2)
Course of action: Switch ISPs, get a new IP address (if they are not very good at configuring a shaper this will work, otherwise not), try a proxy, stop using it for a day or more and it will go away (temporarily most likely). This is done dynamically in the shaper. There is not some dude with his finger on a 'throttle' button. Everything is automatic. Just figure out the how their throttling deterministic state diagram works and you can avoid throttling. Most likely they are throttling you because of your volume of use. It costs a lot for transit access and you are using more than most others by streaming through a vpn.
Thanks for the explanation and suggestions. The volume of use is not excessive, typically 20GB a month, 40GB max. But maybe the shaper is very sensitive, because the bandwidth peaks are quite high probably. So perhaps I could try and limit the peak bandwidth used to avoid triggering it as another option.
Probably not sinister, but you never know... (Score:5, Interesting)
I work in the ISP industry, and here's my $0.02...
The NSA (or other spies), not likely. Everything I have ever seen about what they do is passive monitoring. What that means is that somewhere there is a pretty dumb device (like an optical splitter) that takes one signal and makes two copies, one goes to the NSA, one on to its destination. In this arrangement there is no way for the NSA to inject data at all, including slowing it down. I am highly skeptical any government spying is the direct cause. It may be indirect, I'll come back to that in a minute.
Rate shaping is entirely possible, and would be most likely in your immediate provider. It's entirely common for residential consumer ISP's to employ products like Sandvine, or even more crude QOS controls to rate limit particular types of traffic (e.g. VPN or VOIP). Most won't admit to what they are doing as well.
Rate shaping is less likely, but possible at the country level. This is seen mostly in countries with strong government controls on technology (think Iran, China, North Korea). Egypt was doing it at one point in time. I'm not an expert on Peru, but I would not expect this problem in Peru.
Lastly, is plain old congestion. Likely your ISP has multiple paths to reach Europe, riding undersea cables. These are the most expensive assets an ISP owns, and often get congested before they get upgraded. It's entirely possible for instance there is one cable they use from South American to Western Europe that is congested, while another goes from South America to the US and is fine. You can probably map these routes out by traceroute, and may find that particular routes always show poor performance. This also happens, but to a lesser degree, where two ISP's meet. There can be peering disputes, or one customer may not order enough capacity from their vendor. Either way the result is full ports that degrade service for everyone passing through them.
Now, here's where the spies come back in. If a particular spy agency decrees "all new connections must have our spy apparatus on them" they can in fact be the delay to a new connection getting set up. It's not that they are delaying any packet traffic once it is up, but rather they are delaying the installation by not having their equipment ready on time for a new connection. I don't think this happens often, but I'm sure it does happen in some places.
So sadly, this is probably some plain old incompetence/bad luck. Someone either could not afford a timely upgrade, or didn't correctly order an upgrade early enough to get it installed before there was a problem, and there's now congestion somewhere. If it's not bad luck it's probably your provider deciding your particular type of traffic is "bad", and should be rate limited down.
Some suggestions (Score:5, Informative)
Some more info would be appreciated. So, here's the basics of a few things you can do to make sure it really is the network*. First use iperf on the client and server. Test it on both the tunnel interface and the WAN interface. Second, use top via a separate ssh session. Make sure OpenVPN isn't eating all your CPU or memory. Lastly, what provider are you using? Lately the default Debian build that Edis.at gave me needs an ifconfig up/down every other day.
I've had a similar problem when using my own VPS as an HTTP proxy via OpenVPN. It turned out, the proxy application was crap. Allowing the machine to route packets and using it as a default gateway for all traffic fixed the problem, or at least worked around it.
Now. If it really is blocking, there are a couple of ways around it. The more complicated ones involve using some other VPN application. When dealing with more than one client, that rapidly becomes annoying. A simple one is using an SSH connection as a SOCKS proxy for your browser. It's not elegant, but it works. Another way is to mask your OpenVPN connection by encapsulating the UDP or TCP packets. Once again, SSH port forwarding works, but that's a TCP solution. socat was designed to do things like that, so it seems like a good choice. Finally, there's Ping Tunnel. It embeds traffic in ICMP packets.
Whoever is throttling you might detect one or more of these, but they're probably using some sort of signature based detection. Just about anything that requires a command line should get through.
Remember, since you are technically savvy enough to roll your own, you are the one percent. Good luck, and please let us know how it goes.
*I know you're probably familiar with all of these things. Just assume that I put this section here for those who aren't.
Re: (Score:2)
Thanks for suggesting iperf -- I'd not tried it. I ran through their tests. Both TCP and UDP show about 400kbps on the WAN interface. Running 4 parallel connections for TCP also adds up to around 400kbps more or less, so more connections doesn't actually help, it seems. Over the tunnel I also get about 400kbps. I seem to get much less than 400kbps in practice but the order of magnitude from iperf is right. 'top' doesn't go below 99% idle. I'm running Debian stable. The only thing I have from the host is t
Re: (Score:2)
Glad to help.
The reason why I think may of the wrappers will work is just because they aren't commonly used. Right now people can go pay for an OpenVPN service and download an installer that will do all the work for them. Like tor, OpenVPN is a big target.
The only other thing I can think of is ping times.* It might not look like it, but HTTP is horribly latency sensitive. After every web page is loaded, all the images and javascript are downloaded. Repeat for about a dozen times because javascript is h
Re: (Score:2)
I set up my own OpenVPN with an obscure port number, but using common recommended settings otherwise. Ping times are ~220ms. In my HTTP tests I was downloading one large file with 'wget', so JS/etc weren't an issue. I notice that other people mention that iperf tends to give theoretical rather than practical figures, so that agrees with my experience. Someone below suggested 'pchar' which looks promising but I haven't managed to get results out of it yet.
pchar? (Score:2)
Re: (Score:2)
'pchar' looks interesting. I left it running all night piped to tee but if it generated any output, it never flushed it. I'm trying again now.
I don't see how it isn't (Score:2)
Traceroute is to mainly fix routing problems today (Score:3, Informative)
Many ISP's perform what is known as ICMP rate limiting. Traceroute and Ping both use this ICMP protocol *i'm not going to get into semantics* where as you start traversing the internet past your internet service provider your pings and such to any point along the path have a high chance of being dropped due to this. The only way to see your actual latency is using a host-to-host ping. From your source destination to your final destination. Traceroute acts as sending a ping to each and every hop in between the source and final destination (assuming the TTL doesn't expire or somebody's carrier firewall just doesn't' start letting replies come back through, ie, multiple * * * responses but still able to reach your end destination), they are in no way obligated to reply properly and or in a timely fashion to your Ping request. During the early days of the internet we didn't have many of the problems that we have today and these tools worked flawlessly during this time and really could tell you where your latency is (these tools still function normally in a local lan if you are not doing any "crazy" firewalling tactics). This is no longer the case with ping an traceroute.
IN EXTREME CASES it may be possible to route around other carriers using private tunnels, It's not something your average joe will not likely be able to accomplish without multiple services across the country or paying for some sort of service to do so. AKA you are a business with $$$$. There are instances where it can be done, but are few and very far in between.
If your ISP only has 1 way out to reach specific destinations which are having problems. Provide them traceroutes showing them good responses AND bad responses from when and where you are seeing the problem. The only thing a carrier is going to care about is your "average" response time in milliseconds, not your "maximum" response time.
Blame the NSA (Score:2)
Paranoid much? They only make copies of the data to process off-line, they don't insert themselves into the data stream to do it in real time.
Bounce through SSH or use Tor (Score:2)
Use OpenVPN in TCP mode (rather than it's default UDP mode).
Then set up local ssh port forwards through a bounce host you know works well.
Instead of going from Peru --> UK instead go from Peru --> Localhost --> SSH bounce host in Germany --> UK.
Or try an onion network like Tor.
Obligatory Guide to Knowing Who Is Listening (Score:2)
Martin Bishop: Sorry to waste your time, gentlemen. I don't work for the government.
Agent Wallace: We know. (flashes a badge) National Security Agency.
Martin Bishop: Oh. You're the guys I hear breathing on the other end of my phone.
Agent Wallace: No, that's the FBI. We're not chartered for domestic surveillance.
Martin Bishop: Oh I see. You just overthrow governments. Set up friendly dictators.
Agent Wallace: No, that's the CIA. We protect our government's communications. We try to break the other fella's codes. We're the good guys, Marty.
Martin Bishop: Gee, I can't tell you what a relief that is, Dick.
Courtesy of Sneakers [wikipedia.org] (1992) (video clip of the above here [youtube.com])
Misunderstanding PRISM (Score:2)
You're misunderstanding what PRISM supposedly does. (And you're not the only one.) PRISM does not cause any delays whatsoever - it's not a man-in-the-middle attack. It's simply a copy of all traffic on a fiber. Also an old fashioned "tap" on your Internet connection (usually port mirror at the ISP or Internet exchange) does not cause any delays.
Switch to a different VPS provider.
Ridiculous, but ... (Score:2)
Such a thing would be ridiculous and childish - however things like the diversion of an aircraft that didn't even have Snowdon on it show that the NSA is being ridiculous and childish. Instead of toy soldiers and a way to funnel money out to friends in the private sector the task should be either handed over to military professionals with a focus on things that matter or abandoned entirely. Collecting more data than c
I have a OpenVPN you could try (Score:2)
Try this service and see how it compares to yours:
https://www.vortexvpn.com/ [vortexvpn.com]
See if you get the same behaviour. You get 1GB of free data, if you email support I can give you more. I could also open port 443 if they seem to be shaping non-Http(s) traffic. I have had it running for a few days. There is a server in Dublin you could use.
Meanwhile in Britain (Score:2)
Ministry of Defence (UK) employees spend £40000 on illicit use of the speaking clock.
Down the hall, GCHQ is listening for free.
httptunnel (Score:2)
I had a similar problem with O2 Telefonica, over 3G, in Czech Republic. Their FUP is quite bad. After you reach the imposed limit, they will throttle *all* connections individually to something like 4-5KB/s. Using OpenVPN, or even just HTTPS was impossible.
However, I noticed that HTTP connections were allowed a throughput 4-5 times higher. It's still very low, but usable. My guess is that they separate HTTP connections from everything else. Note that using OpenVPN over TCP port 80 did not help. So, I've sta
EFF's Switzerland Network Testing Tool (Score:4, Informative)
The OP mentions Sandvine: the EFF has a tool called Switzerland. [eff.org]
Is your ISP interfering with your BitTorrent connections? Cutting off your VOIP calls? Undermining the principles of network neutrality? In order to answer those questions, concerned Internet users need tools to test their Internet connections and gather evidence about ISP interference practices. After all, if it weren't for the testing efforts of Rob Topolski, the Associated Press, and EFF, Comcast would still be stone-walling about their now-infamous BitTorrent blocking efforts.
Developed by the Electronic Frontier Foundation, Switzerland is an open source software tool for testing the integrity of data communications over networks, ISPs and firewalls. It will spot IP packets which are forged or modified between clients, inform you, and give you copies of the modified packets.
Switzerland is designed to detect the modification or injection of packets of data traveling over IP networks, including those introduced by anti-P2P tools from Sandvine (widely believed to be used by Comcast to interfere with BitTorrent uploads) and AudibleMagic, advertising injection systems like FairEagle, censorship systems like the Great Firewall of China, and other systems that we don't know about yet.
Re: (Score:2)
But yes, now you can watch BBC online - it will not let you with non UK IP. And to watch hulu you need US IP. And if you live in one of the countries in growing "Wanna Great
Re:ask slashdot: (Score:5, Funny)
No. That's the KGB. Since the alleged fall of the Soviet Union, they've had to run their operations under far more secrecy than ever before. Sometimes, this means they have to leave a job before they have a chance to clean up entirely.
In your case, you've become a test subject for the Soviet loyalists' conspiracy to sap and impurify all of our precious bodily fluids. They are attempting to steal your very essence, and it is your patriotic duty to resist them. Place loaded mousetraps around your bed to damage the stealth robots that are invading your sanctuary of slumber. To prevent their essence-extractor from invading your body, apply a liberal coating of cyanoacrylate to your penis before sleep. It may cause an unusual sensation, but that's far better than the empty fatigue the Communists will inflict.
The NSA is actually fully aware of this conspiracy, and you should assist their efforts to protect our precious bodily fluids. As it is clear that the Red Menace is most interested in corrupting your penis, you must aid the resistance research that is underway. As the NSA must also keep their research secret, no scientists will contact you directly, but you can still contribute to the noble cause by announcing publicly every time your penis functions normally, and especially whenever it does not. This is best accomplished by loudly shouting your results from an open second-story window, followed by displaying your penis for remote optical inspection. Be sure to announce that you are a subject of General Jack Ripper's studies.
The Soviet collapse was a sham, designed to lull the Americans into a false sense of security. The KGB have not given up, and neither can we. God willing, we will prevail, in peace and freedom from fear, and in true health, through the purity and essence of our natural fluids.
Re: (Score:2)
Re: (Score:2)
Thanks -- I'll try port 443 if/when they unthrottle me. Yes, I also get packet loss when there are storms. I guess they interfere with the microwave links over the mountains. Anyway, that is irrelevant as I get good bandwidth on all connections apart from that one IP so the weather doesn't explain it.