Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
The Matrix Media Movies

Solutions for Serving Lots of .torrents? 53

Posted by Cliff from the legal-suprnova dept.
torpor asks: "For 10 years now my friends and I have managed to form a loose musicians collective with the purpose of putting our totally free music/art online for free access. Lately, we've been discussing the approach we can take to utilize BitTorrent to do our file distribution, with the idea being that we can use it to cut our bandwidth costs from that of a single server, and 'spread the bandwidth' among us for hosting files. However, we've found that serving a large number of torrent feeds from one server is not quite so easy as it sounds, as it appears we have to have a single instance -per file- of the torrent server. What solutions are available for Bittorrent-like distribution of media files? We're a small group of budding stars, and we want our tunes out there in the free world. Are we limited to continuing to serve things by HTTP, or are there easy/simple ways of starting a large-quantity of torrent feeds which, perhaps in our fervour, we're overlooking?"
This discussion has been archived. No new comments can be posted.

Solutions for Serving Lots of .torrents? More

Solutions for Serving Lots of .torrents?

Comments Filter:

  • What I would do (Score:3, Funny)

    by Daxster ( 854610 ) on Thursday July 14, 2005 @01:24PM (#13064607) Homepage
    Rename all the files to something like "[porn star] hot secks bath tub", and list all the filename conversions on your website with HTML. Then distribute these torrents on popular trackers and let them handle the torrent :)

  • Torrents suck, but.. (Score:5, Informative)

    by irc.goatse.cx troll ( 593289 ) on Thursday July 14, 2005 @01:25PM (#13064623) Journal
    btlaunchmany can be set to read torrents from a directory. Just throw the .torrent in and it will automaticly start downloading it/seed it when its done. You could easily crontab it and then just ftp upload to the dir. Removing a file is as easy as deleting the torrent.

    Im sure theres a pretty web based solution for the cpanel 'admins' out there too, just google around.

    • Azureus (Score:3, Informative)

      by Guspaz ( 556486 )
      I don't think that solves his problem though, since he would still need once instance per torrent file, unless I'm mistaken.

      The answer is an application like Azureus. It handles everything itself. It serves up the torrent files with a built in HTTP server, it acts as the tracker for all torrents, and it acts as the client to seed each torrent. It allows very powerful rules about when to seed which file, how fast/how much, and so on.

      So the net result is you can handle the entire hosting procedure with just

      • Re:Azureus (Score:3, Interesting)

        by Dahan ( 130247 )
        No, the whole point of btlaunchmany and btlaunchmanycurses is that you have one process that downloads/seeds a bunch of torrents.
        • Ah, not so bad then. However you still need the tracker and web server, and you don't have the option of seeding a torrent trackerless. So you need at least 3 processes, which is admittedly better than one per torrent plus 2 extra.
        • But how it btlaunchman/btlaunchmanycurses currently works is to have a process per torrent and a process to manage the display of the download stats.

          It sounds like the original poster should ask someone at a site like dimeadozen, etree, or tradersden what they are currently using to manage the numerous torrents they are hosting. As far as I know, they still need a seeder process for each torrent. Maybe there is a server version of azureus.

          • No, that's not correct either. It's a single process, but it's multithreaded. Perhaps you're confused by how LinuxThreads [inria.fr] (the old Linux 2.0 thread implementation) created a process per thread. There were some issues with that approach, since processes really weren't meant to be threads. The current Linux thread implementation, NPTL [wikipedia.org], is much more efficient, and is supported in Linux kernel version 2.6 and later (and I guess some 2.5.x development versions).
      • The answer is an application like Azureus

        Has azureus fixed its memory leaks? It has some *severe* problems, so much so that it locks up my X server on a daily basis.

        • It fixed the memory leaks from 2.3.x.x, yes. It still takes a lot of memory though (Your definition of a lot may vary), though it makes up for it in power/flexibility.

  • How about having no server at all? (Score:4, Interesting)

    by AtariAmarok ( 451306 ) on Thursday July 14, 2005 @01:28PM (#13064673)
    Check into www.ourmedia.org. You might be able to just uploaded your stuff there.

    It is just starting out and has problems, but if it shapes up the way they intend, it sounds like it would be perfect for you to host any of your art elsewhere with whatever licensing you want. If this works out, it would be a lot simpler, too. Certainly it would be easy to test. The worst thing I have seen is that it often takes 30+ hours for something you uploaded to be listed.

    • We want to host ourselves, 100%. The whole point of doing the free art thing is so that we can get involved in the delivery of it as well .. if we put our stuff on someone else's server, we have zero control over it.

      But if we start the distribution from a system we own and control and monitor ourselves, it puts us in the position we want to be in, as artists, to keep track of things. Putting it on some other host just gives that host the right to manage our free art accordingly .. thats not the point of
      • "The whole point of doing the free art thing is so that we can get involved in the delivery of it as well .. if we put our stuff on someone else's server, we have zero control over it."

        Have you looked into ourmedia.org? Their fine print looks to leave you in control of the stuff. You can even delete it. The problems I see with it are technical "does not work as well as it is supposed to" rather than "I give up control of my stuff".

        • Physically, we have our own server. Totally Managed, 100%. No license agreements, no business relationship: the machine itself belongs to the community.

          Thats the point. We don't want unknown-entity/3rd-party Jones to have anything to do with it: we're fully DIY...

      • You said:

        For 10 years now my friends and I have managed to form a loose musicians collective with the purpose of putting our totally free music/art online for free access.

        Then you said:

        if we put our stuff on someone else's server, we have zero control over it.

        ...and:

        Putting it on some other host just gives that host the right to manage our free art accordingly .. thats not the point of our project.

        I think you are using a different definition of "totally free" to the popular one here. Coul

        • "I think you are using a different definition of "totally free" to the popular one here. Could you define "totally free" for us so we know what is and isn't acceptable? "

          I think it is pretty clear that they want to be the access point for their free stuff. They don't want others to be the access point, due to concerns that others will charge for it, alter it, place restrictions, add odd download requirements, or other odd things that strangers could do with it.

        • This is one of the problems of being a collective - we might not all have the same definition of "totally free" (I use the word 'we' because I'm on a couple of Ampfea.org mailing lists, and I share the occasional bit of my own music there).

          I think the intent is to keep everything under Ampfea's direct control. If this needs hardware or bandwidth, we'll have a whip round (and not stick ads on anything) to fund it, if it breaks, Ampfea's tech guys will fix it. In short, if it's in-house, everything will be d

        • the point is: we want to be the source of our free material, not someone else. it is free, but we want it to be free from us .. not an archive or website we have no control over.

          its a collective with the purpose of providing ourselves with the means to deliver free content easily .. and manage it.
      • Of course, we're assuming that you've read http://www.bittorrent.com/guide.html [bittorrent.com], which shows you how to do it. Recent versions are trackerless [bittorrent.com], so setting up the tracker is no longer valid, though if you want to service the 3.x versions, you may still want to set one up.

        There's always freshmeat [freshmeat.net] to search [freshmeat.net] for options. EZTorrent [freshmeat.net] appears to be what you want. mod_bt [freshmeat.net] also looks promising if you're using Apache 2.x.

  • BitComet (Score:4, Informative)

    by Fry-kun ( 619632 ) on Thursday July 14, 2005 @01:28PM (#13064676)
    BitComet can serve multiple torrents with different priorities, all within the same instance of the program

  • The Matrix? (Score:2, Interesting)

    by mr_rattles ( 303158 )
    Am I missing something here? Why is the topic "The Matrix?"

  • One tip... (Score:3, Insightful)

    by Burianski11 ( 554142 ) on Thursday July 14, 2005 @01:30PM (#13064708)
    If you're trying to save on bandwith costs, don't put a link to your site on /.
    • Actually, that hits the nail on the head, I'd LOVE to have a link to my latest album in my slashdot sig, and for Ampfea (the collective we are dscussing) to be the place that link goes to. Seamless, easy to use torrent system would be the ideal way to do that without breaking the bank.

  • Is there an actual problem? (Score:3, Informative)

    by Otter ( 3800 ) on Thursday July 14, 2005 @01:41PM (#13064830) Journal
    As with a lot of P2P-related Ask Slashdots, it sounds like you have a solution in search of a problem.

    The value of P2P comes on several fronts: obviously, it's ideal for illegal sharing (which doesn't concern you), BitTorrent in particular is good for huge legal files with huge demand peaks (e.g. new Linux distribution ISOs and it's good for large distributed bodies of files (like Furthur.net).

    In your case, a website and HTTP distribution seems the best way to go, despite its unsexiness. You control the process, so you can track downloads and referrers and you can make sure that things work properly, so your new fans don't suffer. Just bite the bullet and pay a good hosting provider.

    • We have our own server, and we have an excellent hosting provider.

      We want to use Torrent because it 'spreads the load' easily, without needing a big management back-end to watch our bandwidth usage...

      We've used HTTP distribution for 10 years now .. what we want is to make it even easier for members to contribute to the bandwidth bills by simply hosting their own torrent client peers themselves .. either you can make a donation to ampfea, or leave a torrent server running for all to access, on your own loc
    • BitTorrent in particular is good for huge legal files with huge demand peaks (e.g. new Linux distribution ISOs and it's good for large distributed bodies of files (like Furthur.net). [...] In your case, a website and HTTP distribution seems the best way to go, despite its unsexiness.

      Another advantage of BitTorrent is that other people can easily join in to reduce your bandwidth costs.

      When somebody recently a book [accelerando.org] via BitTorrent, I thought that was pretty cool, and wanted to help out. I joined all the t
  • Funnily enough, I wrote up my experiences with BitTorrent only the other day, and it might be of some help to you: Using BitTorrent on Linux [dhs.org].
  • Sign up for a webhosting account over at Hurricane Electric [he.net]. They include free torrent hosting. According to their torrent FAQ [he.net], they'll eventually limit you to 25 torrents per account, but their servers sit on fat pipes and they won't charge you for the bandwidth used. :)

    FYI: I don't work for them or anything; I'm just a very pleased customer.
  • This may be totally off-topic, but since the Matrix as a cultural focus was destroyed with the release of "The Matrix Revolutions" I wondered how it would be before Taco & the gang decided to recast the Blue & Red Pill icon away from the movies and onto something else.

    That day arrived today.

  • No problems with that here.. (Score:4, Interesting)

    by hacker ( 14635 ) <hacker@gnu-designs.com> on Thursday July 14, 2005 @02:07PM (#13065234)

    We're not having any trouble serving lots of torrent files [plkr.org] from the same instance of the server. To the tune of over 109GiB of Plucker [plkr.org] torrent downloads this year.

    That doesn't count the downloads over http, rsync, and through our mirrors. Probably another 200GiB there, rough estimate.

    I use bttrack and point it to the torrent directory (locked with --allowed_dir of course), and it works great.

    I run a separate server on a separate port for various projects, to separate the torrents per-project. No issues at all.

    • Replying to myself here... I just did a quick check and we've served up 25,502,515,260 bytes over http for one of those files (plucker_desktop, Windows build) THIS MONTH on the primary server.

      Since we round-robin across 3 mirrors, that could potentially be 75GiB for this month alone for that one single download (10,241,974 byte file).

      Its massive, and that's the primary reason we started moving to BitTorrent to help distribute those downloads.

  • It may be worth a look.

    http://dijjer.org/ [dijjer.org]
  • torrentcasting!

    Customize a podcasting feed and augment Azureus or something to handle it. Neat!
  • Use the btlaunchmany.py (part of the official distribution) to seed several torrents with just one instance of BitTorrent running. It recognizes most of the same command line options as the vanilla bittorrent client.
  • I can't believe anyone has said this but you're asking the wrong crowd!

    Ask ThePirateBay!!

  • eztorrent (Score:3, Informative)

    by mrsam ( 12205 ) on Thursday July 14, 2005 @06:46PM (#13067922) Homepage
    When I was in a similar situation, I ended up writing eztorrent. You can find it on Freshmeat.

    Publishing BitTorrent content is as simple as copying the files into a directory, and running a single command. Eztorrent automatically creates the matching .torrent files, start the tracker, and start a seeder for each .torrent file.

    Files can be added or removed from the torrent directory at any time. A single command adds/removes any .torrent files, and starts or stop the seeders, without any downtime for any other, active torrents.
  • Check out the people sharing live music shows at http://www.dimeadozen.org/ [dimeadozen.org] - their tracker is even open source. Could be able to help you, if I understood the question correctly.

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close