Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
Software The Internet

Making BitTorrent Clients Prioritize By Geography? 227

Posted by timothy
from the hey-you're-blocking-my-bandwidth dept.
Daengbo writes "While I live in S.Korea and have virtually unlimited bandwidth in and out of the country, not all my Asian friends are so lucky. Many of the SE Asian and African countries have small international pipes. Even when a user has a high-speed local connection, downloads from abroad will trickle in. Bittorrent clients apparently don't prioritize other users on the same ISP or at least in the same country. Why is that? Is it difficult to manage? If I were to write a plug-in for, say, Deluge, what hurdles would I be likely to come across? If this functionality is available in other clients or through plug-ins, please chime in."
This discussion has been archived. No new comments can be posted.

Making BitTorrent Clients Prioritize By Geography?

Comments Filter:
  • by dave562 (969951) on Tuesday December 02, 2008 @12:47PM (#25962271) Journal
    There is already a plugin for Azereus that does this. I downloaded it about a year ago. I'm at work right now or otherwise I would look at my installation and tell you the exact name of it.
    • by LSD-OBS (183415) on Tuesday December 02, 2008 @12:52PM (#25962369)

      Is that the same plugin that constantly runs a barrage of pings in a hidden shell? Can't remember the name, but I ran a similar sounding plugin and didn't see much speed improvement but it sure did chew up my CPU.

      • Re: (Score:3, Informative)

        by dave562 (969951)

        It probably is the same one. I seem to remember that it was developed as part of a university project. I probably even read about it right here on /. I never checked the background activity so I can't comment on the barrage of pings. I do recall that it didn't make much of a difference in my torrent speeds. I'm on DSL at home and always get consistently good torrent performance.

        • Re: (Score:3, Insightful)

          by Guspaz (556486)

          The "barrage" of pings may not have been necessary. A good first step is simply running the IPs through a geolocation database. There are various free ones available, and it's a pretty good first step for narrowing things down. They're very effective at getting the country right, and do a decent job at getting you to at least a nearby city.

          From there, if you need farther precision, a single ICMP packet is required to determine the number of hops to a host by checking the TTL. Combine these two things and it

    • by markov_chain (202465) on Tuesday December 02, 2008 @01:06PM (#25962583) Homepage

      You can tell me, I work in the same place.

    • by mark_hill97 (897586) <masterofshadows@ g m a i l .com> on Tuesday December 02, 2008 @01:18PM (#25962805)
      It is called Ono and it can be found here [northwestern.edu]
    • Re: (Score:3, Interesting)

      You could make a whole lot of ISP's happy by prioritizing the "closest" hosts. To do this would require computing the number of hops between the two devices. Ip packets have a ttl header that is (or should be) decremented by each router in the path. A little "magic" would be required since hosts don't universally have a consistent beginning ttl, and some firewalls play with it to obscure the host information. Discovering it shouldn't be too hard though. Another factor could be latency between hosts. T
      • by Shotgun (30919)

        It just requires a traceroute to the handful of connections each client makes. Throw is a little routing protocol, and your in business against Cisco.

    • I wish there was a way to setup a p2p app with something
      like internet radios multi cast.

      basically 100 or 100,000 ppl could download because they
      would "tune in" to a download that was just like a song
      that played, and replayed and eventually they would have
      all the song once the CRC lines up.

      It would save enormous amounts of bandwidth, and if you could
      get multi-seeds setup, you could tune into other parts
      of the data stream and get it faster.

      Ie. if 5 ppl were seeding, they'd xmit just 20% of the total
      stream and

      • Won't work (Score:3, Insightful)

        by Andy Dodd (701)

        There is little to no support for multicast by last-mile ISPs.

        It would be nice - ISPs keep bitching about how P2P is eating their bandwidth, but they don't bother implementing multicast which would make P2P use a fraction of the bandwidth it currently does.

        Admittedly, in addition to lack of support, IPv4 multicast is pretty "meh" - there aren't many multicast addresses available and I have yet to see a good way of choosing/assigning them on a global network.

        • Re: (Score:3, Insightful)

          by bluefoxlucid (723572)

          It would be nice - ISPs keep bitching about how P2P is eating their bandwidth, but they don't bother implementing multicast which would make P2P use a fraction of the bandwidth it currently does.

          I believe you mean P2P uses a fraction of the bandwidth it would if we had multicast. I have a constant upstream of 200k/s, to 5 clients, each getting 40k/s down from me. With multicast, I could have a constant upstream of 200k/s, to 5 clients, each getting 200k/s from me. This means I would send 200k/s up; my router would route for about 6 hops, then send duplicate packets out to 5 different routers from that hop, which would then send 200k/s down to various points on the Internet. Instead of consumin

          • by Sancho (17056) *

            Seems like for the bandwidth that ISPs care about (that which they have to send across the backbone), it'd be about the same. But they'd be providing better service, assuming they can fix the DoS issue.

            But it's obviously never going to happen because the implementation details are a pain in the butt.

          • Re:Won't work (Score:4, Informative)

            by Andy Dodd (701) <atd7&cornell,edu> on Tuesday December 02, 2008 @05:19PM (#25967111) Homepage

            Look at it from a different perspective - 100 people want one file of size A megabytes, and you start with one person seeding.

            With classic unicast distribution, 100*A needs to be sent by that one person.

            With P2P, much less needs to be sent, but still 100*A needs to traverse the network. It may or may not traverse fewer hops - it may in fact traverse more.

            With multicast, A megabytes leave the single origin, and that gets multipled by routers as needed in the most efficient manner. In the end, the least amount of data needs to be sent in order for everyone to achieve completion. Yes, in the short term the network load will probably not be reduced much, but the time that high load occurs for will be far less before everyone has the file and there is little need for lots of bandwidth.

    • ISP's in America are against locals serving content. This is very obvious by the fact that your upload allowances are a significant digit smaller than your download allowances. For this reason, bit torrents are far better prioritizing a larger pipe than a shorter hop.

      Remember, ISP's are NOT your friends. They are a contract partner. Their interest is not to make your experience better; it's to only make your experience slightly better than the competition.

      For torrents to coexist with ISP's would require

      • Re: (Score:3, Insightful)

        by nabsltd (1313397)

        Extending a business partnership with them, and convincing them that they CAN allow users to serve content without choking their already oversold bandwidth

        This is fairly easy to do right now eith US ISPs...it's called a "business account".

        They do cost more, but in my case I get 5 static IPs, guaranteed bandwidth, and no interference of any kind with my data (no port blocking, no throttling, and no caps) for $140. Compared to the non-business version with one dynamic IP, port blocking and some throttling (but no caps yet) at $70, it's not a bad deal.

        • They do cost more, but in my case I get 5 static IPs, guaranteed bandwidth, and no interference of any kind with my data (no port blocking, no throttling, and no caps) for $140.

          Is that for a symmetric link, or do you just get a standard asymmetric account with a couple IPs and a different TOS?

          • by nabsltd (1313397)

            That's 20Mbps symmetric for both the regular and business accounts.

            The only real differences are the TOS, the support (24/7 for business), and the static IPs.

            • That's 20Mbps symmetric for both the regular and business accounts.

              If consumer accounts are symmetric, then you don't have the general problem that was being complained about in this thread to begin with. Many people are stuck with drastically asymmetric connections as the only consumer-level internet accounts available (think 10Mbps down / 1Mbps up).

      • Not all ISPs hate their customers ;) Mine gives me 15Mbps up and down, for $41.90/month. I asked if I can host a server, they said yes. I asked if there's a bandwidth limit; the answer was "yes but we don't have a way to enforce it right now. We'll let you know when that changes."

        Contrast that with what Comcast told me when I called. 7Mbps down, 3Mbps up (or something along those lines) for approximately the same price. Can't host a server. "No bandwidth limit, but if you use too much we'll cut you o

    • by Nick Ives (317)

      Ono [northwestern.edu] uses statistical data from CDNs to be a little bit smarter about picking peers in certain cases. In most cases the random solution is fine; your client can just randomly pick peers then stick with fast ones and drop slow ones. Ono aims to improve performance in certain cases where that strategy isn't very good.

      Just in case anyone reading doesn't notice, Ono aims to find peers that are close to you on the network. That doesn't necessarily mean close to you geographically and so doesn't answer this ask /.

  • "Prioritizing" (Score:5, Insightful)

    by zombietangelo (1394031) on Tuesday December 02, 2008 @12:48PM (#25962303)
    IPs could, theoretically, be prioritized based on a database of known general geographies associated with certain digits. Just remember - prioritizing is one thing, but it's a slippery slope to peer exclusion.
    • Re:"Prioritizing" (Score:5, Insightful)

      by evanbd (210358) on Tuesday December 02, 2008 @01:27PM (#25962947)

      Just remember - prioritizing is one thing, but it's a slippery slope to peer exclusion.

      Not really. Prioritize who you request data from, but not who you send it to. If all incoming requests are treated equally, but you only request from the optimal peers, you get all the benefits without any of the omgcensorship fud.

    • by cs02rm0 (654673)
      I believe IPv6 addresses give you quite a lot of location information (lots more than IPv4).
    • Theoretically == libgeoip
  • uTorrent (Score:5, Informative)

    by SpitfireSMS (1388089) on Tuesday December 02, 2008 @12:49PM (#25962317)
    uTorrent has a feature called local peer discovery that does that exactly. It was even able to discover other people at my university sharing the files.
    • by gstoddart (321705)

      uTorrent has a feature called local peer discovery that does that exactly. It was even able to discover other people at my university sharing the files.

      Don't tell the RIAA -- they'll get a law passed insisting the universities implement this so they can keep their funding if they turn over the list. :-P

      Cheers

    • by msormune (808119)
      Is the plugin called "To-be busted by the RIAA" plugin?
      • Re: (Score:3, Interesting)

        by GuldKalle (1065310)

        It's not a plugin, and it doesn't make you more or less likely to get busted. It simply searched the peer list from the tracker and prioritizes peers on the same subnet as you, and optionally removes your throughput limits.
        A good feature, but it doesn't completely solve the problem, since it only works within your local subnet, and is therefore inherently incompatible with NAT.

    • Re: (Score:2, Interesting)

      by lc_overlord (563906)

      Well not exactly, it does try to find local users, but that is more or less useless on anything larger than any single localized network, like a university, which is all good and well.
      But i seriously doubt that it optimizes this in a way that takes the network infrastructure in consideration.
      Even though the local peer your downloading from may be your neighbor you might just end up being routed trough Zimbabwe for all utorrent cares at this moment, especially if net neutrality is destroyed.
      And that is not g

    • Re:uTorrent (Score:5, Informative)

      by Fëanáro (130986) on Tuesday December 02, 2008 @02:03PM (#25963603)

      Local peer Discovery works only within a lan, the corresponding broadcast messages are not routed over the internet

  • Ono (Score:5, Informative)

    by pythonax (769925) on Tuesday December 02, 2008 @12:50PM (#25962319)
    There is a plugin for Vuze (formerly Azureus) called Ono which does exactly that. Not sure what the problems they ran into, but as it is a college project I am sure they would be willing to discuss some of it with you. http://www.aqualab.cs.northwestern.edu/projects/Ono.html [northwestern.edu]
  • Non-geo-ip (Score:4, Interesting)

    by rbanffy (584143) on Tuesday December 02, 2008 @12:50PM (#25962335) Homepage Journal

    One would not even need to prioritize by geographic location: the client could easily give extra priority points by network class: C first, then B, then A, then the rest. The odds of having a very fat pipe to another machine in the same class C are far better than having a fat pipe to a random machine across the planet.

    And that would also alleviate the load on backbone links.

    • Re: (Score:3, Informative)

      by VGPowerlord (621254)

      That doesn't work well with networks split with CIDR. For example, the 24.x.y.z block is in the Class A address range, but it's not a class A block.

      • by jeffmeden (135043)

        He forgot to say: Check ARIN, RIPE, APNIC, etc for the assignment of that block, and analyze accordingly. The information is there, it just has to be asked for.

      • by nabsltd (1313397)

        Although it's not perfect, what the GP suggested ("C first, then B, then A, then the rest") would mean that if you are match on /24, then it's likely to be very close. After that, check /16 then /8.

        You could generalize this to just XOR your public IP with the IP address for the proposed peer and count the leading set bits. More bits means the peer is likely closer to you.

        Again, this isn't perfect, but it would help some and doesn't require any extra information about the network other than what is already

    • you have the "windows guide to networking" definition of Class. Class != subnet mask. Classes are quite a bit different, and millions of times larger than you are thinking.

      For example, the Class A network is:
      1.0.0.1 - 126.255.255.254
      Thats around 2Billion hosts, if i'm not mistaken

      Class C is: 192.0.1.1 - 223.255.255.254

      For reference:
      http://www.webopedia.com/DidYouKnow/Internet/2002/IPaddressing.asp [webopedia.com]

      • You misread.

        The "class A" networks (plural) each have a unique first octet between 0 and 127. i.e. each class A is 2^24 (~16 million) addresses, and there are 128 class A networks. IIRC 0... and 127... are both reserved, leaving 126.
        "Class B" networks have a unique first two octets, between 128.0 and 191.255. Each contain 2^16 (~65 thousand) addresses, and there are 16,384 class B networks.
        "Class C" networks are identified by the first 3 octets, between 192.0.0 and 223.255.255. Each has only 2^8 (256) addre

    • by imbaczek (690596)
      outdated knowledge warning in 3... 2... 1...

      google CIDR.
  • by Anonymous Coward on Tuesday December 02, 2008 @12:50PM (#25962337)

    That way they can take advantage of the tendency of IP packets to flow downhill.

  • Already Made "Ono" (Score:5, Informative)

    by StarWreck (695075) on Tuesday December 02, 2008 @12:51PM (#25962347) Homepage Journal
    What you're looking for is an Azureus plugin called "Ono". It prioritized based on router hops. Theoretically, this would make those connected to the same ISP preffered. After that it would make ISP's with direct connections to your ISP preferred. After that, resonably close geographically, ie same country.
  • Latency? Hops? (Score:4, Interesting)

    by Midnight Thunder (17205) on Tuesday December 02, 2008 @12:52PM (#25962357) Homepage Journal

    How good is latency or hops as indicator of distance from peer? The idea is that if it takes 5 hops, as opposed to 10, then the peer taking the least hops to get to is the closest.

  • by chrysrobyn (106763) on Tuesday December 02, 2008 @12:52PM (#25962359)

    For Vuze [vuze.com], formerly Azureus [sourceforge.net], there are Ono [northwestern.edu] and P4P [yale.edu], which should do what you're looking for, although for different reasons. Unfortunately, they both rely on people in your region being interested in the same torrents you are, while P4P additionally benefits from an iTracker, an ISP provided tracker that's topology aware (they did some work to prioritize based on ping latency, using that as a distance estimate, but I don't know if it's a fallback mechanism). Due to the iTracker infrastructure and possibly conflicting supporters, there are some privacy concerns [torrentfreak.com].

    • "they both rely on people in your region being interested in the same torrents you are"

      No Problem. I mean, come on, who doesn't like midget llama spanking pr0n?
  • by VGPowerlord (621254) on Tuesday December 02, 2008 @12:57PM (#25962445)

    "Bittorrent clients apparently don't prioritize other users on the same ISP or at least in the same country. Why is that? Is it difficult to manage?"
    The reason BitTorrent doesn't prioritize other users on the same ISP or the same country is that it doesn't know which ones are part of the same ISP or the same country. For ISPs, since the introduction of CIDR addresses, ISPs can have multiple blocks of IPs. Can you honestly tell me what all of, say, Comcast's IP blocks are with any degree of certainty?

    For countries, you either need to know which IP blocks IANA has allocated to which IP registry or use a geolocation library.

    MaxMind's GeoIP [maxmind.com] seems to be the de facto geolocation library, but they charge money for the "good" version. There is a free version [maxmind.com] now, but it has some annoying requirements, such as having to include "This product includes GeoLite data created by MaxMind, available from http://maxmind.com/ [maxmind.com]" in all advertising materials and documentation. It also only has a 99.5% accuracy as claimed by its creators, which means the the accuracy is probably considerably lower than they claim. Even if it were 99.5%, that means it's wrong for 1 out of every 200 people.

    • Re: (Score:3, Insightful)

      by yachius (1348219)
      The 99.5% accuracy number is considerably greater than 1/200. The accuracy figure is not a percentage of lookups that are incorrect, but of how close to the correct physical location the query result will be. Even the .5% of wrong locations won't be on the wrong side of the planet so it still makes for a very useful resource for figuring out regions.
    • by vux984 (928602)

      Even if it were 99.5%, that means it's wrong for 1 out of every 200 people.

      How wrong is wrong, and is its wrongness regional?

      If it gets the country/state/province right... but wrong town/city, that's not all that bad for this purpose.

      Or If the wrong results / sketchy results are heavily concentrated in Africa and Eastern Europe that's not really going to impact me much, again, that's just fine... at least for me.

      • by Lennie (16154)

        It probably just says something is in europe, instead of country in europe. The free database only has country-information.

    • by Bieeanda (961632)
      Another problem is that ISPs will sometimes acquire blocks that the GeoIP services still think are in other countries. One of the big Stateside ones just did (Comcast?) and Google spent some time shunting users in that block to an 'appropriate' Google International start page. I don't want to think about the trouble they must have had with sites that specifically block access from outside of the States, like Showtime.

      And geography is one thing, when you've got trans-oceanic cables to contend with... but r

  • by PhrostyMcByte (589271) <phrosty@gmail.com> on Tuesday December 02, 2008 @12:57PM (#25962457) Homepage
    I've always wondered why ISPs can't give higher speeds if you stay within their network. You'd get your download faster. You'd use less peering bandwidth, costing the ISP less money. Everybody wins.
  • by Nick Ives (317) on Tuesday December 02, 2008 @01:08PM (#25962617)

    If your ISPs international pipe is flooded then bittorrent will automatically prefer local peers as they'll be the only people who can send you data at a fast enough rate. If you notice local peers who you're not connected to then it's most likely just because they've already reached their connection limit.

    Most bittorent clients will connect to many peers and try to saturate your downstream bandwidth. They don't care where in the world those peers are as long as they're fast. If, in your part of the world, local peers are faster then that means you should just automatically connect to more local peers.

  • by Aloisius (1294796) on Tuesday December 02, 2008 @01:14PM (#25962717) Homepage
    At Napster I wrote a system to weight peers that were closer to the person searching by using network distance.

    It was mostly because universities were complaining and so we weighted everyone on Internet 2 towards each other, but it also worked quite well for service providers like @Home and AOL. Since ISPs don't seem to care as much when their own bandwidth is used, a lot of complaints about our bandwidth consumption disappeared overnight. Indiana state university and someone else helped out if I remember correctly.

    It was a rather simple system that used BGP routing tables from a number of routers to build a graph of network connectivity. It wasn't perfect, but it didn't have to be.

    That said, with IPv6 weighting is *much* easier because of how the IP space is divided up. You can do a super naive implementation just by prefix.

    An Azureus plugin Ono does something similar, though I believe they just look up the IP address for a CDN and weight people that look up the same IP towards each other. It is a decent solution, but it only works for between people who are running the plugin.
  • by WinDev (1422301) on Tuesday December 02, 2008 @01:35PM (#25963079)

    The biggest speed issue facing Asia/Australia is the latency of traffic to the rest of the planet. The (Windows) TCP Receive Window is tuned too small for the distances required. If you change the receive window to the maximum, you can get 4x more data in the same period using any client (P2P, browsers, etc...).

    Refer to:
    http://cable-dsl.navasgroup.com/index.htm#IncreasingWindow [navasgroup.com]

  • by PJ The Womble (963477) on Tuesday December 02, 2008 @02:17PM (#25963829)
    When I was in my first year at college, we were asked to produce a questionnaire about using ATMs, including the question: "If you could change one thing about your bank's ATMs, what would it be?"

    The most popular answer I managed to get was "if the machine's running out of money, they should restrict the cash withdrawal function to customers of this branch".

    Does anyone see a parallel here?
    • by PingSpike (947548)

      Odd. I would have thought "Make it give me free money" would have been the most popular answer.

      • Re: (Score:2, Funny)

        by PJ The Womble (963477)
        I went to University of Teesside for 3 years. Have you ever been to Middlesbrough? It's grey, cold and wet in July, and the rest of the year the weather is *really* depressing. They're not the most optimistic people in the world. Even free money wouldn't improve their stoical disposition, IMHO.
    • by TheLink (130905)
      No. If an ATM is running out of money, a notification should be sent, and someone/something should go reload it with money.

      If lots of machines are running out of money, a notification should be sent so that the Bank can ask the Government for a bailout and to make hopefully confidence inspiring statements :).
  • I'd prefer one that deprioritizes (or outright bans) my ISP and geographical region.. but then again, I'm a Comcast subscriber in the US. As a Comcast subscriber, I know that I'm not likely to get decent speed from other Comcast subscribers (unless they happen to be on my node), and as a US citizen, I'm well within the reach of anyone who decides they want to sue me because my kids torrented some MP3s when I wasn't looking.

  • by WittyName (615844) on Tuesday December 02, 2008 @07:17PM (#25968833)

    And I proposed it for two popular BitTorrent clients, only to be told, "we don't need that.."

    Simple enough to do. Start with Class C address.. less then 2^24 of them, as some
    are reserved, and not routable. Make a bitmap that big, so divide by 8.
    Only a two meg file. Then just watch your connections, total by bytes received, then divide by 8. If the result is greater than say 1 kb/sec sustained, then
    set the appropriate bit to true. Allow the bitmap to be zeroed if you move or whatever.

    After getting the list of peers, prioritize connection attempts towards those
    that have a useful sustainable rate. Nothing worse than seeing 80% of my
    connections saying connected 30 minutes, 100 KB transferred. Sigh.

    While in china, I had a 2 mb connection. But too many Chinese hammering it, so
    I could sustain, 2 mbit up, and 1 mbit down. Asymmetric the wrong way..

    Seems blindingly obvious to me, yet I still see no clients with this feature.

The tree of research must from time to time be refreshed with the blood of bean counters. -- Alan Kay

Working...