Microsoft Windows Update and Network Bandwidth? 144
Brett Glass asks: "As we reviewed the cache statistics for our small ISP today, we noted that the traffic generated by Microsoft's Windows Update feature constituted 45% -- no, that's not a misprint -- of our total throughput. Because so many computers on the Internet run Windows, this massive resource drain occurs whenever Microsoft announces major security holes (as it did this week). The traffic could be greatly reduced, and service to users much improved, if the updates were cacheable at the ISP. But Microsoft has set up the service in such a way that the data can't be cached. (It's digitally signed, so inserting Trojans into the cache is virtually impossible; in any event, no more of an issue than intercepting the data stream.) Are others out there seeing the same pattern? How might Microsoft be convinced to make its updates cacheable, so as not to waste unthinkable amounts of bandwidth?"
MS wants you to host one internally (Score:3, Funny)
Valid (Score:4, Interesting)
It seems that it could be an easy implementation. The proxy requests the file verification in, an XML-RPC request is returned from the server to perform the checksum, the resulting data is sent via SOAP, and approval is given or denied, causing the cache to be used or flushed.
I don't know enough about it to say how difficult it would be to have the proxy determine if the service is available, though. It needs an acronym if it's going anywhere. How about Verify Cache Request (VCR)?
Re:Valid (Score:3, Interesting)
Why invent a new protocol if you already have one with the required functionality.
Re:Valid (Score:5, Insightful)
It seems that it could be an easy implementation. The proxy requests the file verification in, an XML-RPC request is returned from the server to perform the checksum, the resulting data is sent via SOAP, and approval is given or denied, causing the cache to be used or flushed.
Ahh, but then that would involve Reverse Engineering, which, as we know [slashdot.org] is now illegal.
Not to mention that this is approaching a P2P network, which as we know [slashdot.org] can only be used for piracy.
Sorry, we're all just going to have to live with this new "innovation" in bandwidth utilization.
Re:Valid (Score:2)
Should work just fine. The product currently using "VCR" has been declared obsolete anyways.
Re:MS wants you to host one internally (Score:3, Informative)
Re:MS wants you to host one internally (Score:3, Informative)
Let me just say, SUS sucks ass.
Microsoft's systems of GPO's makes it pretty useless -- you need to set GPO's for hosts to use your SUS servers, so if your domain has any divergence from the stock GPO's there is a good chance it isn't going to work and it will be impossible to debug in less than a month.
I believe there was a giant thread about it on focus-ms.
Re:MS wants you to host one internally (Score:2)
Hey now... whose side are you on?? You're supposed to suggest to him "Why don't you ditch MS and install Linux on all your PC's - it is ready for the corporate desktop afterall."
can't be cached? (Score:5, Informative)
Re:can't be cached? (Score:5, Informative)
Our store Squid server caches the likes of IE 6.1, Media Player and DirectX, but the vast majority of the Critical/Security updates are not cached. Our connection is quick enough to handle it, but a PITA nonetheless due to the dozens of machines requiring updates every week.
Re:can't be cached? (Score:2, Informative)
Re:can't be cached? (Score:5, Informative)
# Objects larger than this size will NOT be saved on disk.
# maximum_object_size 4096 KB
maximum_object_size 32767 KB
Re:can't be cached? (Score:2)
Re:can't be cached? (Score:1, Informative)
use a redirector in squid and point customers to the local version on your website a la http://www.squid-cache.org/Doc/FAQ/FAQ-15.html [squid-cache.org] Seems basic to me
Re:can't be cached? (Score:5, Interesting)
Looking at my squid logs, it appeared that there was a problem with WindowsUpdate issuing a 0 byte sized reply to the GET request (must be somthing to do with the activex control I guess, but never really bothered to look further into it). Squid seemed to choke on the 0 byte reply and obviously didnt cache the rest of the download.
Interestingly enough, MS's caching offering ISA appears to deal with it, but I suppose that they specifally coded it with a knowledge of how their activex control works and hence it knows whats following that 0 byte reply.
Disclaimer - I checked this all out when the new WindowsUpdate first came out and havent been arsed to look at it since then. I ended up just setting up a shitty old box as a SUS server and going that route. (The only benefit to being an MSDN partner being $0 cost for licensing as I justify it as being for testing purposes
Re:can't be cached? (Score:5, Informative)
Re:can't be cached? (Score:3, Interesting)
Re:can't be cached? (Score:4, Informative)
You might want to analyze exactly what is occuring in your site(s).
Cheers,
Rob
(Squid core developer)
Reload_into_IMS on a per-site basis? (Score:2)
Perhaps allowing Squid to be configured to ignore proxy:nocache and to convert reload into IMS based on an ACL would allow a site admin to tweak around this without breaking other sites>
Re:Reload_into_IMS on a per-site basis? (Score:2)
Still, I've not seen a windows update client set no-cache on their requests to date....
What comes to mind though, is that windows update clients (both the web interface, and automatic updates) use the MS http support libraries, that are configured via the internet options control panel. And in there, the 'check every time' option results in no-cache bei
How big are these things? (Score:3, Interesting)
Apple's own software updates are pretty big, although with a much smaller percentage of machines as macs they're not going to cause the same volume of problems. The last few I've seen have been around 40MB, with one topping out at 80, and most security updates (every 2 months perhaps) being 5-15MB
Re:How big are these things? (Score:2)
Re:How big are these things? (Score:3, Informative)
This makes it easy to share with others who might not have the bandwidth to download these freakin' things.
Re:How big are these things? (Score:2, Informative)
either you can build your own windows update server or at the very least download the individual updates and store them just as files
and you can even build them into an iso image, my win2k cd has sp3 built into it so whenever I build a new machine it's already there, and you can do that with most of the updates
Re:How big are these things? (Score:2)
I mean, I can see that an ISP couldn't really make any headway by hosting the file on web site or something, but there are several replies here from network managers complaining about bandwidth.
Why don't they host it for their users and save the bandwidth?
Am I missing something?
Re:How big are these things? (Score:3, Informative)
Re:How big are these things? (Score:2, Insightful)
Re:How big are these things? (Score:4, Informative)
Since Microsoft release patches via Windows Update so frequently, they are usually fairly small. 1MB-5MB downloads are frequent, with the occasional 10MB+ one every now and then. There are updates practically every few days, so having a Windows Update Server running will negate the expense of everyone having to download redundant files.
Some help about storing Windows Update files for later can be found here [microsoft.com].
Re:How big are these things? (Score:2)
Individual patches are usually well under 5MB. When MS rolls all those little IE updates up into a service pack, they're usually around 10MB. Media Player updates weigh in around that size, too. Above that, you're looking at things like major DirectX revs and the .NET Framework. And I weep for those who try to download an OS service pack over a dial-up.
Re:How big are these things? (Score:2)
Standard anti-MS rant (Score:4, Funny)
this nothing (Score:3, Funny)
The rest 5% is Netbios traffic.
Re:this nothing (Score:3, Funny)
95% code red
50% Kazaa
40% HTTP
20% Spam
45% windows update
Gee, has anyone heard of a new science called Mathematics ?
Re:this nothing (Score:1)
Willy Wonka Says... (Score:2)
Invention, my dear friends, is 93% perspiration, 6% electricity, 4% evaporation, and 2% butterscotch ripple.
That's 105%!
there is the way that large corperations do it (Score:4, Informative)
You can download the updates individually, and there is probably a way to have them downloaded to the server automatically. All you have to do is convince the users to download them from you and install them manually. Can you block traffic from the autoupdate applet? I bet that would significantly reduce traffic, at the cost of insecure customers.
What about running an internal WU server and changing the DNS entry at the local level to a local server? You'd have to keep the catalog of updates stocked and refreshed constantly, for multiple OS's, so I don't know how cost effective it might be.
Re:there is the way that large corperations do it (Score:2)
Re:there is the way that large corperations do it (Score:2)
Or, in other words, are the users paying for their bandwitdh? Yes? Then who the hell cares how they're using it?
Re:there is the way that large corperations do it (Score:2)
Re:there is the way that large corperations do it (Score:3, Informative)
Yeah, except the fact that SUS is a free download [microsoft.com]. Maybe you're talking about Systems Management Server [microsoft.com] which does cost a bit, but does a lot more than just security updates.
Re:there is the way that large corperations do it (Score:2)
This means that for smallish organisations, you need your AD master, a backup master (at least 1) for redundancy and then your additional SUS server, which means minimum 3 server licenses, plus all your cals. It also means you have to run IIS (http and ftp) which noone in their right minds wants to run. I havent been able t
Re:there is the way that large corperations do it (Score:2, Informative)
Q. Can I run SUS on an Active Directory domain controller?
A. Yes, SUS 1.0 SP1 allows for this.
Re:there is the way that large corperations do it (Score:1)
As I said though, there wasnt any valid reason that it couldn't be run on an AD DC. I imagine that enough people complained and MS was forced to remove whatever restrictions were in place, rather than added the functionality. (But then I'm always quick to abuse MS. I have had to support their crap software across 20 odd client sites, hundreds of servers and destops in their thousands over the past 8 years, so I reserve this right.
Re:there is the way that large corperations do it (Score:3, Insightful)
I run it at home for my network (1 Email/Web/DNS, 1 DNS secondary, 2 AD, 1 SQL, 2 XP pro workstations for devel), just so I don't have to abuse my DSL too much. Instead of each machine hitting WU, only the SUS Server does. Each machine gets the update off the SUS server.
I do think that MS should wise up and m
Several options available (Score:5, Informative)
Another option is to use a systems management package (LANDesk, ZENworks, SMS, etc.) to build the packages and deploy them while only using your internal network bandwidth (once you've downloaded the hotfixes anyway).
Of course, the two options above are really meant for company networks, but even those can help reduce the bandwidth used for more important things.
Re:Several options available (Score:1)
Out of my experience (Score:5, Informative)
First step is to download the patches/update manually and save them elsewhere accessable to all users:
- Windows 2000 users, please visit the
Windows 2000 Downloads [microsoft.com] site.
- Windows NT 4.0 users, please visit the
Microsoft Download Center [microsoft.com].
- Windows 98 users, please visit the
Windows 98 Downloads [microsoft.com] site.
- Windows 95 users, please visit the
Windows 95 Downloads [microsoft.com] site.
Second, we found that users would rather use windowsupdate.microsoft.com then to go to our patches/update repository, that make sense. You could forbid your users from accessing windowsupdate.microsoft.com, but it might have a problem, as some update might actually request windowsupdate.microsoft.com during installation.Therefore, we limit the priority of traffic in/out of windowsupdate.microsoft.com. Eventually we lower the prior of entire microsoft.com because that's really necessary. Users could access to windowsupdate.microsoft.com on their own as usual - if they don't mind holding up their machines for a couple of days.
This works great. Larger and bigger patches are stored locally for users, while they could still access to windowsupdate for smaller patches/fixes. Our bandwidth load lessen(to a certain degree, we still can't solve that 5-15% Netbios traffic jam
Hope this help.
Re:Out of my experience (Score:2, Redundant)
Re:Out of my experience (Score:2)
Re:Out of my experience (Score:1)
I particularly like the idaa of limiting the priority of traffic for windowsupdate.microsoft.com as it still lets the user run
Re:Out of my experience (Score:2)
Or may be, regardless of all the effort, we aren't very good at tuning squid. We'd be much appreciate if experts out there could give us some
Hope they don't notice . . . (Score:3, Funny)
~~~
The other 55% (Score:5, Funny)
You could... (Score:3, Informative)
If it's a big problem, just block off windowsupdate and redirect them to your own page. You could implement a simple scan using something like HFNetChk [microsoft.com]. It's command line and works well.
Hey, look at it this way.. at least your users are updating! That puts them above 90% of the users out there.
'convincing' Microsoft? (Score:1, Troll)
The only way to convince Microsoft of anything would be to _buy_ Microsoft.
Just watch... (Score:2)
Software Update Services (Score:5, Informative)
In the meantime, you should be aware that all the major service packs [microsoft.com] for Microsoft products can be downloaded as stand-alone executables. Also, the IE download page [microsoft.com] includes some critical updates. Make your own "cache" on the network, and let everybody get their updates from there.
Re:Software Update Services (Score:2)
Re:Software Update Services (Score:5, Informative)
So, since ISPs can't administer their users' systems, this really isn't an answer. Caching is a much better solution.
Re:Software Update Services (Score:2)
They still do, kind off, as a function of the standard Windows Update site. For instance, Microsoft wants me to get my Windows XP Updates from the URL: http://v4.windowsupdate.microsoft.com/en/default. a sp [microsoft.com]. However, since I have a few machines running XP with no Internet access I get my patches instead from the URL http://v4.windowsupdate.microsoft.com/catalog/en/d efault.asp [microsoft.com]. Note the addition of the "catalog" in the URL!
It's a pain, because y
Re:Software Update Services (Score:2)
The other 55%? (Score:3, Funny)
So dare I ask what the other 55% is? Here's my guess:
No, don't check. You don't want to know.
Re:The other 55%? (Score:2)
Microsoft offers Software Update Services (Score:3, Insightful)
Re:Microsoft offers Software Update Services (Score:2)
No, this is Microsoft. They want your MONEY, so it isn't that simple. The main reason is it doesn't work with XP Home, only XP Professional. [Note: This is according the the
Re:Microsoft offers Software Update Services (Score:2)
2. Insert foot
3. Correct the mistake
Digging further finds that it DOES work with XP Home. Still, 98 and ME users are left out, but it is a step in the right direction.
Re:Microsoft offers Software Update Services (Score:1)
They do. No patch. (Score:1)
The question he actually asked - (Score:1)
> cacheable, so as not to waste unthinkable amounts of bandwidth?"
Well, you could try threatening them with legal action - that usually works...
What about Debian Linux (Score:2)
Stats for the past 24 hours are even worse.... (Score:4, Informative)
Re:Stats for the past 24 hours are even worse.... (Score:1)
Re:Stats for the past 24 hours are even worse.... (Score:5, Funny)
That's so typical of Microsoft. They don't care about the little ISPs, they just want their customer base to have free, simple, access to frequent updates and fixes, without giving a damn about the impact that has on Internet traffic.
I mean, at least when slashdot directs huge amounts of traffic to some dumb site about making a spaceship out of a floppy disc or whatever, they have the courtesy to always cache the site so that it doesn't take down the whole ISP that hosts that page.
Why can't MS be more like
Re:Stats for the past 24 hours are even worse.... (Score:2)
[ponders, wonders, decides...yes! WACK WACK WACK WACK goes the Clue(tm)-brand clue bat against Jon Peterson's head.]
The issue isn't that it is easy, but that they have such an ineffecient and mostly uncachable mechanism for distribution.
The frequency of updates in relation to the bugs fixed isn't too much or too often.
Re:Stats for the past 24 hours are even worse.... (Score:2)
What's even more terrible is that no one really knows how many times the EULAs have been changed as a result.
I installed Windows XP the first time recently and was disturbed at the default settings for Windows itself and the Media Player. MS should not be trusted. They're practically as bad as Real Player.
Re:Stats for the past 24 hours are even worse.... (Score:2)
Invoice them for your bandwidth.
Homepage (Score:2)
ISP Caching (Score:4, Informative)
Software Update Service (Score:4, Informative)
Bandwidth and AOL (Score:2)
Hmmm...
On a related note: I haven't looked recently, but it used to be that Windows clients were TERRIBLE about DNS lookups - they would not cache anything, and were always making DNS requests on every little thing. I was helping a FOF set up his DSL, and his DNS lookups were taking 3-5 seconds, because his ISP's name servers (swbell) were overloaded. We finally set up his own internal name server, and
Re:Bandwidth and AOL (Score:2)
None. AOL users don't do "updates" or "patches".
Seems obvious in retrospect, doesn't it?
Total Cost of Ownership (Score:1)
In a world of open systems, everyone who felt like doing it could cache software updates, freeing money and bandwith for more sensible uses than trying to cure a dead horse.
The crux of the problem ... (Score:1)
This posting [squid-cache.org] from the squid-users mailing list sheds some more light on the issue.
If you were wanting to break the RFCs and were using squid, then you could probably modify src/http.c to return 1 for the relevant parts of the httpCachableReply function instead of 0, but that would be a "Bad Thing"(tm) when it came to RFC compliance.
Re:The crux of the problem ... (Score:2)
Wouldn't one reason for doing this be so that if they needed to recall a patch that turned out faulty, and put out a new one, there's no risk of the old ones being cached somewhere?
Must be a small ISP (Score:3, Insightful)
Why don't you post some hard data instead of percentages? Saying windows update is 50% of your traffic is meaningless unless you provide background. What is your normal traffic? How close are you to capacity?
Talk to them (Score:2)
Please don't (Score:2, Insightful)
If you think it wastes too much bandwidth, think about the bandwidth which could be wasted by a network full of machines which were compromised due to not fetching the latest securty updates.
Re:Please don't (Score:3, Insightful)
Doesn't happen. If there's an update to the update, it's done as a separate update.
Interesting. (Score:2, Interesting)
I suspect that someone at Microsoft has been reading this discussion, which is good.
Most of the stuff that became cacheable, though, was for Windows XP. Windows 98 and Me updates (and we have a lot of users ru
Measurement a little high? (Score:2)
All the traffic I see to/from microsoft - including msn and hotmail, accounts for perhaps 30% of my traffic on a typical day. On a day when somethign like DX9 comes out, that figure goes up a bit - but still not to the 50% level.
Do you maybe have a customer who builds systems and mass-updates them? that would almost make the number reasonable....
Re:GPL is to blame (Score:1)
Re:Would you all bitch if it was another vendor? (Score:3, Insightful)
Just settle down, really. Maybe you should go to bed.
Re:Would you all bitch if it was another vendor? (Score:1, Informative)
the red hat updates are cacheable yet individually gpg-signed. they are also freely distributable by anyone. you can set up a red hat satellite proxy server for your organization. you can download once straight from red hat's FTP server (the URLs are conveniently listed in the emails) and push them to each machine. there are probably 50 different ways you can write perl scripts to fix the problem.
seriously, this is a difference between FREE SOFTWARE and VENDOR L
Re:Would you all bitch if it was another vendor? (Score:1)
Microsoft probably knows EXACTLY how much of a pain this is and will happily SELL you some overpriced "Windows Update Proxy Server Professional 2000
Don't be a moron. Software Update Services [microsoft.com] is free. All you need is a machine running IIS.
Yes, but it's not (Score:1, Troll)
Re:Yes, but it's not (Score:1)
Re:Would you all bitch if it was another vendor? (Score:2)
Actually, this is easy to combat. Just make
Re:Would you all bitch if it was another vendor? (Score:3, Informative)
Re:Would you all bitch if it was another vendor? (Score:2)
Re:BITS (Score:2)