Managing Bandwidth and Bandwidth Costs? 202
"I'd like to illustrate the second concept. When you have your (for example) T1 and you're not really using it, you are still paying for all that bandwidth. It's like the car that sits in your garage, you're still paying insurance and car payments on it even though you're not using it. But then you put up a new game, serve new media or suddenly become the 'Site of the Day' and your bandwidth is flooded and maxed out. For that case, it's like you've bought a car that only goes 40 miles an hour but while the demand exists and only while that demand exists, you need a car that goes 150 miles an hour. You don't want to pay the money for a car that goes 150 because you only need it occasionally. Later, you know you'll need that car to go 220 but you're not there yet.
So if this makes sense with regards to bandwidth, it is like you'd want burst-bandwidth depending on need. Do any of you face this problem? If you do and have solved it, I'd love to hear about your strategy. Once this is solved, we get back to the first question, how do you manage that cost, put a number on it and either fit it in to your business model or pass it on to your customers?"
Doh!, (Score:3, Funny)
Or we can do it the debian way - let someone else tar it up and then do it.
But we like the Slashdot way most. Post it up and watch the traffic floooooooooooooooow....
Re:Doh!, (Score:2)
Many Options (Score:3, Informative)
You figure our how much you will estimate you will use as a total per month, and you pay for that. You cruise at 2mbit/s mostly, then you explode to 145mbit/s and at the end of the month you average out to 6mbit/s.
The idea is you BUY your peak speed, but pay for a low average..
EVERYONE does this now, call around... It was kind of silly for you to ask Slashdot how to do this. UGH.
- Voxel
Re:Many Options (Score:2)
If it's a game, get it on fileplanet, ign and such. Places like ign even have a built-in p2p client for people to use if downloads are heavy for a particular game.
Are you kidding? (Score:2)
Re:Are you kidding? (Score:3, Insightful)
largest ftp site in the world.
Re:Are you kidding? (Score:2)
Colocation! (Score:5, Informative)
Be sure to tell your colo or file hosting provider what your projected usage is, and how many megabits you may want access to, to assure that they can handle it. You may also want to make a courtesy call a day or so prior to each launch to let them know what to expect.
Remember when Eddy Van Halen got tounge cancer a couple years ago? THAT was a busy weekend for their website, which we host. Of course, they didn't have any warning, but boy-o, that was bigger than any slashdot effect that I've ever seen. We also host O'Reilly (the computer book folks), so we certainly see plenty of slashdotting.
We're at: http://www.sonic.net/sales/colo/ [sonic.net]
Shop around - but keep in mind that buying from someone near your intended downloader may help you with both latency and costs. The SF Bay Area has the best pricing for bandwidth, and the lowest latency connections to the highest number of users - that said, if your target market is on the east coast, you should be in Hearndon, VA or NY or Boston.
-Dane Jasper (Sonic.net)
Re:Colocation! (Score:2)
Dane, what does the cost per megabyte work out to with your model? I think we are using the 1 cent per megabyte model now just in case this may need to be covered in the costs we have to pass on to the customer.
Re:Colocation! (Score:2)
The earlier posting about 95percentile peak rate is pretty important to consider.. That means you only get grace on the top day and a half per month. If you release somethi
Re:Many Options (Score:2, Insightful)
Attach it to a Britany Spears tune... (Score:1, Funny)
Managing bandwidth (Score:4, Informative)
Re:Managing bandwidth (Score:4, Informative)
Re:Managing bandwidth (Score:3, Interesting)
The current UI looks like puked out in 2 days
Also it might make sense to provide a BitTorrent version that installs through ActiveX. The download version should be offered for Linux, Mozilla and Opera users.
Christian
So go hire Bram.... (Score:4, Insightful)
Re:Managing bandwidth (Score:2)
The thing is, clients don't want to be leaving windows open to be helping the reseller, or whatever it is the poster's company does. It's simply not done - the client should be able to log on, get w
Re:Managing bandwidth (Score:2)
Didn't somebody do a Java BitTorrent client already? That would be better than an ActiveX version, especially if you're delivering something that's platform-neutral.
Re:Managing bandwidth (Score:5, Informative)
Daniel
Re:Managing bandwidth (Score:3, Insightful)
The combo of the two would be great for moving distributions and other mirroring of publicily available data.
Re:Managing bandwidth (Score:4, Interesting)
In other words, if I'm uploading at my maximum speed, I can't reply quickly enough to tell the nodes I am downloading from that the last packet got through, so they resend it instead of the next packet until I can squeak out an ACK or two to let them know to move on.
BT is a nice concept, but in practice it needs work.
Re:Managing bandwidth (Score:3, Informative)
I don't have the problem very often on my cable connection, but the one time I experienced a major BT-related slowdown, I was quite annoyed - so I can empathize with your hatred.
Here's the link to the experimental client:
http://ei.kefro.st/projects/btclient/ [kefro.st]
Re:Managing bandwidth (Score:2)
I love the idea of BT, but it's far from the greatest solution. I think the Kazaa method of assigning everyone a contribution rating is a MUCH better idea. That way,
emule (Score:2)
Re:Managing bandwidth (Score:2)
Re:Managing bandwidth (Score:3, Interesting)
Re:Managing bandwidth (Score:2)
You can set it to just under your max upload... There's stll enough bandwidth for your ACKs, and then you can DL as fast as other people upload to you(which can be a problem due to these other clients...)
Re:Managing bandwidth (Score:2)
The point here isn't a handfull of geek users who configure their own Linux routers, its thousands and thousands of people of varying degrees of intelligence and ability connecting to a system that just works.
Re:Managing bandwidth (Score:4, Insightful)
What you're missing is that if you end up using your client's bandwidth to distribute your software without their knowledge or permission, you're screwed.
In serving the general public, something like this would be subject to a large negative reaction not to mention the problem of "have you ever gotten a file from bit torrent that was invalid?" I have.
Though technically neat, practically, it's unfeasable for a mass market product.
1) You can't impact/rely on the user base to help you deliver your product.
2) Added chance for error introduces risk and jeapordizes your distribution model and therefore business model.
Re:Managing bandwidth (Score:5, Insightful)
If you had to pay for your bandwidth, would you give it for free to some company from which you are currently downloading a product update? I wouldn't...
Re:Managing bandwidth (Score:3, Interesting)
Actually I was working on a P2P software that solves this problem,
by introducing "neighbourhood". That means, when several machines
in the same LAN receive the update, they will prefer to connect to
each other rather than to a machine outside on internet. This
SAVES money (on ingres), provided that exgres bandwidth isn't
excessively donated back to the P2P community.
Re:Managing bandwidth (Score:2)
BitTorrent is a P2P-style app, even though it can have a specific aim in mind that's not illegal filesharing, and the vast majority of people actually are under an AUP (acceptable use policy) or EULA or whatever that prevents
Re:Managing bandwidth (Score:2)
Violating ISP AUPs result in the contract being terminated and possibly a bill for early cancellation, I don't know if you would count that as a "fine".
No university would expell someone for sharing legitimate content with BT. There would be a process and some other punishment (loss of computing privileges prolly) before resorting to expelling.
Fired, is a possibility as private firms hav
Re:Managing bandwidth (Score:2)
You're not following the news to make the statements you're making.
If you use BT and you wind up with illegal content on your machine, even in parts, you're certainly subject to fines and arrest, just as with Kazaa, et al.
What you're saying isn't unreasonable, it's just not how companies and institutions and government is coping with file sharing at the
Re:Managing bandwidth (Score:2)
If you use BT and you wind up with illegal content on your machine
Of course if you violate copyright law you risk serious consequences. That was not the point you were making earlier. You maintained that one risked serious consequences, including arrest, whether using BT with legitimate material or not.
Universities may well ban BT as a bandwidth hog. But they will not expell someone for
here's what he's asking (Score:3, Interesting)
maybe if he got some comments from the peeps at osnews or cnet...
BitTorrent? (Score:1, Redundant)
Maybe?
Check it out: http://bitconjurer.org/BitTorrent/
Get what you want... (Score:5, Informative)
Get Burstable Fibre (Score:5, Informative)
*blatant sales pitch*
If your buisness is near Southern Ontario, check out our website at www.sentex.net [sentex.net]. We rock
Free Tip (Score:3, Funny)
Congratulations - you passed this one.
Anyway the answer to all your prayers is obviously BitTorrent [bitconjurer.org].
Savvis has huge sales right now. (Score:2, Interesting)
Right now they're about 700/month for a managed T1 with local loop included.
Re:Savvis has huge sales right now. (Score:2)
We see 180 Kbps max downstream on our T1.
If we are serving all modem users at a max dl speed of 5 Kbps, then a T1 can only serve 36 users at once and allow 5 Kbps per user.
That ain't much.
This is easy... charge the customer! (Score:5, Interesting)
2) Use RTG [sourceforge.net] to monitor traffic in and out, making sure that you know what switch/ports/etc. that client is using.
3) Charge the client (this is usally done based on 95th percentile).
4) Profit!!!
P2P it (Score:2)
All your company needs to do is to seed the files on the networks. Much lower bandwidth than serving each one.
Akamai or other offsite hosting (Score:5, Informative)
It's almost certainly going to be cheaper than just buying bandwidth.
Or you could go for the approach of colocating your own box somewhere central for the heavily hit stuff.
Even this will be a whole lot cheaper and won't impact on your normal traffic to your organisation.
Re:Akamai or other offsite hosting (Score:3, Interesting)
Re:Akamai or other offsite hosting (Score:2)
Akamai isn't cheap, but having done both, it is cheaper than setting up your own infrastructure.
One word (Score:3, Informative)
Testing Solution (Score:3, Funny)
Big Pipe (Score:2)
Then this whole "unused bandwidth" issue would be kinda moot. Everyone ones (well, except the person paying the bill).
Bit Torrent? No, Kast! (Score:3, Interesting)
If you need proof that Kast is better than BT in this situation look at http://konspire.sourceforge.net/BitTorrent.shtml [sourceforge.net] .
Hope it helps.
Re:Bit Torrent? No, Kast! (Score:3, Insightful)
P2P and partial files is the answer (Score:5, Interesting)
With 100 gig hds, and reliable high speed connectivity 24/7. It's pretty easy to see what could be built from that.
Someone wants to download the new 150mb CS or BF1942 upgrade? Just enter the name of the file, select it...and the P2P software does the rest. Initiating multiple downloads from about 20 or so nodes in parallel. Then you just glue the program together once you have downloaded all the pieces.
Mirroring is such a hack, and dynamic bandwidth is the last gasp of the client-server paradigm. Let's move on.
Re:P2P and partial files is the answer (Score:3, Insightful)
P2P is severely overrated. It's not a solution for anything that doesn't involve sharing illegal files. If everyone on P2P is on a standard 128k/768k ADSL connection, it will not work -- there will be more demand than supply. It only works when there are nodes with a fat upload line that never download anything. But those
I disagree (Score:3, Insightful)
1) Not everyone uses their Internet connection 24 hours per da
2) Most people don't need hard drive space these days (i.e. storage is cheap)
Combine these two developments, and you have a lot of upload bandwidth sitting idle. I would argue that P2P becomes MORE effective, not less, as you move to legitimate files, because people are more likely to leave it running when they aren't afraid of the RIAA/MPAA tracing their connection down. Since ISPs have been r
Re:I disagree (Score:2)
Re:P2P and partial files is the answer (Score:2)
Oh? So then I guess "illegal p2p" sites like GameTab [gametab.com] (bittorrent) and FileFront [filefront.com] (redswoosh) aren't a solution for getting game demos faster than CrapPlanet's [fileplanet.com] fantastic client-server lines eh?
P2P is a hack
Why do I get the impression that your job depends on the centralization of power that client-server allows?
If I need to download a BF1942 patch, I'll get it and delete it
Speak for yourself. Not everyone is as selfish as
Re:P2P and partial files is the answer (Score:2)
P2P allows much more centralization of power, actually. Just cap the upload or block some ports and it's over.
Speak for yourself. Not everyone is as selfish as you apparently are, and p2p will eventually have reputation systems for weeding rogues and assholes out of our webs of trust.
Webs of trust? Nice buzzword, not likely to ever happen. Unless you massively centralize the whole P2P network
Re:P2P and partial files is the answer (Score:2)
Been implemented before -- and without a central trusted server, and in a very difficult-to-circumvent fashon. See PGP's method for determining keys' trustworthiness.
Re:P2P and partial files is the answer (Score:2)
Why anonymous? (Score:2)
Find a new provider... (Score:5, Informative)
Since we have more then one link as well, it gives us redundancy and the upper hand to negotiate the best price per GB, so we can send 90% of our traffic out that link. If the next month a different provider comes back with a cheaper price, we switch it around and send the 90% out their link. Within days of cutting the traffic off for a link, we can usually expect a phone call from the sales rep with lower price offer.
Any idle links we have don't cost us anything extra, since we _will not_ deal with any provider that doesn't offer pay by the GB. Paying for the raw link speed, regardless of how much traffic you push through, or paying 95th percentile prices are all mostly a rip off.
Re:Find a new provider... (Score:2)
Re:Find a new provider... (Score:2)
CDN, colocation (Score:5, Informative)
Another option is colocation. In particular if you have short traffic spikes. Many colocation places charge your for at a '95 Percentile'. This will cut out about 3 days worth of 'peek traffic' and you only pay for the maximum bandwidth you use after removing the top 5%. Just make sure the colocation place has enough bandwidth to handle the spikes.
Some ISPs (e.g. Yipes) offer flexible contracts that allow fast (daily?) bandwidth changes. So if you announce a new version of your product, you can increase your bandwidth until the rush is over.
One hint: Try to move the large file/content away from your 'importants' networks, so other things like e-mail keep flowing even if the content site is running into issues due to load.
Content Delivery Network (Score:3, Informative)
Re:Content Delivery Network (Score:3, Informative)
No, I didn't site through the presentation; however, our admin guys seemed impressed. Just another option.
Option pricing methods may well be the solution... (Score:5, Interesting)
Based on recent research at the University of Waterloo, you may well be able to treat the bandwidth usage as a risk factor and treat the option to buy more bandwidth as exactly that: an option on a real commodity. You would likely be able, then, to price the value of waiting to invest versus the value of investing now with a given expected return. Basically the cost of holding off on investing would then be quantifiable and you could choose the best time for investment.
There has been some good research [uwaterloo.ca] done on this lately which you can read up on at the U. Waterloo Scientific Computation Group [uwaterloo.ca] which did the work in co-operation with telecoms and the Finance department. The math is perhaps a little heavy going, but the results may put you on a firmer footing than doing the same computation with NPV or similar methods.
Disclaimer: I'm currently doing research with this group, though not exactly on this topic.
Re:Option pricing methods may well be the solution (Score:2, Insightful)
"Bandwidth trading" escapades have bancrupted Williams and contributed to Enron having to go ever more aggressive with their "creative accounting." It reeks of the same foul smell as the "real options" nonsense. (Note: the article referenced even has "real options" in the keywords).
Basically you've got a whole bunch of not-so-smart people who can fake expertise in nearly anything as long as they don't get their hands dirty (aka academics) colluding with s
Don't host it yourself. (Score:5, Interesting)
The easiest solution is not to host it yourself, but to use specialized file hosting ISPs. There are lots of these around, and it's a trivial task on Google to find one at the price you want. These are ISPs that entirely focus on hosting large files for download, with servers optimized for that job.
There's no point in lagging out your regular servers which are probably optimized for something else.. and a dedicated file host can scale as you go.. which would usually cost you a packet.
Re:Don't host it yourself. (Score:3, Insightful)
Re:Don't host it yourself. (Score:2)
Sorry, this is half-pure bollocks. Companies can easily distribute via P2P ... Nobody said you'd have to use the "official" client. Many bigger software installers download the bits and pieces they need from the net. There's absolutely no reason why you wouldn't be able to put a P2P technology in there instead of simple http gets.
Even if you don't go the P2P route, look at the many dedicated server providers out there. If your tech staff is half-way decent, they can set up a "simple" fileserver in a very
Re:Don't host it yourself. (Score:3, Interesting)
> distribute stuff via Kazaa or BitTorrent? Sorry,
> but when Microsoft says 'To download our latest
> Service Pack, use Kazaa' then pigs will be flying.
> It's so unprofessional.
Replace all the statements in here about kazaa with statements about linux.
And you get the same thing people were saying 4/5 years ago. Don't underestimate the ability for the tech industry to evolve.
Re:Don't host it yourself. (Score:3, Insightful)
They will do it if saves them money or if the alternative is simply not to host large files. Many sites already allow or even require you to have special download managers (Fileplanet etc.). What if they provide special rebranded versions
Re:Don't host it yourself. (Score:2)
Alternative methods. (Score:2)
How much will be web-content? Files? Ftp? Small or large files? What about your customers? Are the "tech savy"? US-only? Old? Young?
Maybe there are other distribution-modells that are better for you than the tradidional client-server approach?
Some suggestions:
1.I heard about a game company that saved *thousands* of dollars each month when they actively started to supply game magazines with demos a
Managing your network is an ongoing process (Score:2, Informative)
Some things to consider checking into:
Use SNMP and RMON to manage and monitor your wide area connections. You will be able to do trending on your traffic to see what percentage of used is used during any predetermined interval. Free tools like MRTG are a great pl
Move your servers off-site (Score:2)
It's just not economical for a single company with occasional high-bandwidth requirements to bring in a pipe that sits idle 90% of the time. A co-location facility will serve many differen
Re:Move your servers off-site (Score:2)
It's mentioned elsewhere, but worth mentioning here: be very, very careful if you're going the burstable route. Most providers charge based on 95% utilization, not actual bandwidth usage. What this means to you is that a single short burst of extremely high usage (of the sort described by the poster) causes you to pay as though you were
Re:Move your servers off-site (Score:2)
Huge Burstable Pipes (Score:2)
Spread out the downloads (Score:2)
Your Options (Score:3, Informative)
If you have your servers hosted with co-location provider like AT&T, Globix, Cable and Wireless (ha ha), Verio ot the like then you'll almost always have the option to "burst" above your monthly allotment of transfer per month - problem solved. Just do some simple math and figure out what is the max of the LAN connection of the provider, how much your server(s) can handle with respect to transfer and you'll be able to figure out if you need to add hardware or NIC's to handle the load. Generally speaking the "bursting" is usually calculated fairly. Also, in months that you use less bandwidth you won't have to pay for a higher class of service above the already agreed upon monthly transfer rates.
However, It doesn't sound like this is the case. A t-1 has a physical capacity to transfer 1.544 Mb/sec. You will never be able to "burst" above this rate. As long as your servers are at the end of this pipe they will never be able to transfer more than ~ 180K/sec to the internet at large. Your options at this point are:
* Add more capacity to your hosting site (expensive).
* "Partner" with a company like Akamai for content delivery. It'll be a few thousand dollars plus the cost of bandwith to set up the content redistribution. Not really a bad deal if you are serious about delivering your content to your users. They also have great reporting about who and where your users are coming from.
* Your last option is to set up a shared/dedicated hosting account with a provider that charges you by the GB. That way you only pay for what you use + the monthly cost of the server. Try interland they have some good deals.
The bottom line is that if your site is hosted at the end of a physical connection you own - it's not going to be enough. You'll need a datacenter environment that has a PhatPipe to the internet and a machine or machine(s) that can handle the throughput. If you have the cash go with Akamai - they are good at what they do.
Characterize your traffic, then apply rules. (Score:3, Informative)
http://www.linuxgeek.org/netflow-howto.php
http://wwwstats.net.wisc.edu/
http://www.arbornetworks.com/products_sp.php
Then you need to make use of technologies which allow you to provision accordingly:
http://www.cisco.com/en/US/tech/tk543/tk757/tec
http://www.cisco.com/pcgi-bin/Support/browse/in
Juniper and other vendors have equivalent capabilities, I'm just not very familiar with them. But the concepts are the same.
Buy an idle server at an ISP (Score:2)
Management Tools (Score:2)
since you are talking about FUTURE, and not now, I would first and foremost use Linux to manage the bandwidth. I assume you haven't made any buying decisions yet?
You can buy propritary heavy lifting equipment such as Cisco gear, and collapse the entire management tier into one switch, but I wouldn't.
It will work fine, BUT you can't reuse Cisco gear as desktop PC's, or even donate them to a school when they get too old.
My way of building things, you can reuse the equipment when its all done.
Wit
Re:Management Tools (Score:2)
Why would you use fibre and not copper for the "interconnects". Surely the distance can't be *that* far ?
Akamai ? (Score:2)
I think their basic product is "x amount of space and bandwidth very close to the end user".
For example, Ximian used akamai at somepoint when their first gnome 1 desktop came out. I downloaded it and installer was *blazing* fast. I was wondering what was going on as i never get +500kb/s speeds from us and i was pretty su
Re:Akamai ? (Score:2)
One option, and it's probably the easiest. (Score:3, Interesting)
Have the timeout on a file reasonable, though. FTP files generally don't change that frequently, so hold those for 48 hours. HTTP pages can change a lot more often. Make sure the caching server is configured to respect the HTTP flag to not cache, and don't cache unmarked web pages for more than a few minutes.
This should be sufficient to spread out the load some. I prefer to have a few "extras" in there, though. If I know there's going to be a popular file released, it makes sense to have a CRON job pre-load that file into the cache during off-peak hours. That way, it's in the cache and ready for users by the time anyone gets round to requesting it.
If there are a few web sites that are VERY popular but largely static, set up a "neighbor" cache, which ONLY caches those web sites, with a very long time-out. Use a program like Harvest to grab the entire site, via the cache, and you'll then have everything ready for the users. (It'll also be searchable, via Harvest, which'll be a bonus.)
The second option is at the network layer, and should be used only if the above is not sufficient. Enable "diff-services" and "Quality of Service" in the kernel. How to do this depends on the OS you use. Linux and all the *BSDs support these options, but how you set it up varies with each.
Once you've done that, enable "HQF" or "CBQ" queueing discipline, and attach those to the FTP and HTTP services. Configure them such that each user is guaranteed a certain level of bandwidth for that service, if they request that or more. They get more only of nobody else needs it. (This is usually described as a "soft ceiling".)
You also want to enable the "RED" networking option.
This isn't as hard to do as it sounds, and it can massively ease network congestion.
(CBQ = Class Based Queueing. HQF = Heirarchical Queueing Function. RED = Random Early Detection.)
Once you've applied both the "high level" and "low level" solutions, your network congestion should be massively eased. Again, though, use pre-loading for the cache as extensively as you can to ease those peak-time burdens.
Re: One option, and it's probably the easiest. (Score:2)
Re:One option, and it's probably the easiest. (Score:2)
Here's an easy solution... (Score:2)
Put your server in a colocation center. You can normally get deals where you pay for a low bandwidth regularly, but have the ability to burst quite high. As an example, you can be connected to a 100 megabit port on their router, but pay a base rate for, say, 1 or 2 megabits.
If that's not an option, you can always look into per-IP bandwidth limitting and QOS.
steve
DOH! (Score:2)
Here's the easiest solution yet: Distribute your file via BitTorrent. Problem solved!
steve
Outsource (Score:2)
Disclaimer: This particular site is run by friends of mine, but they're not the only ones
burstable bandwith with a control shaping. (Score:5, Insightful)
Take a burstable bandwidth, let's say that can burst to 100mbs, but to control your bandwidth in most time to ensure you do not go over the cost, you configure your router to not allow more than let's say 1mb of bandwitdh or whatever you want as a maximum and willing to pay for in normal time.
You should then monitor your bandwidth usage in real time, as well as the logs on the machines, and adjust the traffic shaping to the amount of traffic you want to allow.
For example, you know what on that day, you will do a marketing operation, and you are willing to spend $xxx more for the bandwidth, you then change your setting right before your marketing plan to the maximum of bandwidth you are willing to pay.
my 2c...
Manage, Manage, Manage - Plan, Plan, Plan (Score:2)
As the first local court system in the US that streamed their own court proceedings via the Internet, we were inundated with hits to our media server. By using the bandwidth throttling in Real Server we were able to max
plan ahead... (Score:2)
You should buy a ram jet powered locomotive... never a problem with speed or overloading your hauling capacity... be sure to lease it.
Bandwidth Arbitration (Score:2)
Getting a burstable line to the net is not hard at all; most ISPs work that way (you pay for 6 mbps, for instance, but have a 100 mbps ethernet line, so you just pay when you burst above 6).
One thing you guys might want to think about is using a bandwidth arbitrator for when you do have a busy day. There's one good project I know of: the Linux Bandwidth Arbitrator [bandwidtharbitrator.com]. It's easy and free, and it'll keep individual users from hogging bandwidth -- and meter all users to whatever rate you choose. It's based on
Re:Multicast - yes, mbone is dead (Score:2)
Several carriers will move multicast data - Sprint comes to mind, I think UUNet does, and I hear there are some other tier 1 providers out there, but I don't use them
Only works as long as all the clients are on multicast aware networks