How Much Are Ad Servers Slowing the Web?

vipermac writes "Most of the times I have a problem with a Web page loading slow or freezing temporarily, I look down at the status bar and see that it's waiting on an ad server, Google Analytics, or the like. It seems to me that on popular Web sites the bottleneck is overwhelmingly on the ad servers now and not on the servers of the site itself. In my opinion we need a better model for serving ads — or else these services need to add more servers/bandwidth. Are there any studies on the delay that 3rd-party ad servers are introducing, or any new models that are being introduced to serve ads?"

use firefox and adblocker!

problem solved.

Re:use firefox and adblocker!

Well then, feel free to call me a rampant kleptomaniac.

Re:use firefox and adblocker!

Kleptomaniac? Is the the KDE p2p app?

Re:use firefox and adblocker! (mod up parent)

At least you get mod points. I haven't had the ability for easily five years now despite consistently having excellent karma.

Interesting, how does an AC get karma?

Re:use firefox and adblocker!

But isn't that like saying if you don't look at the bilboard as you drive down the highway, or at the ad poster on the subway, or walk into the kitchen for a snack during the commercial break, that you're a theif? The television example is probably the best and easiest to associate with.

Advertisements are an opportunity to make an impression on a customer. While there will always be technologies that make it difficult to ignore the ad, in most cases you are not obligated to be impressioned. Just because the advertisers get upset that you are stealing the cheese from their trap, it's your prerogative.

I'm a little surprised that we don't see more "banner" ads on TV. Imagine all these people with the widescreen sets that are viewing content with black sides because it's 4x3 formatted instead of widescreen. Imagine banners on both of those dead zones on the sides, that change every 20-30 seconds and adjust their product to something related to the main feature. Y know I think I would prefer that to the "four miniutes of commercials every 10 minutes" we get now. Also, even though a lot of shows are timed for like 49 minutes for the hour to accomodate commercials, a lot of stations trim out scenes or cut them short to insert more ads, so we would be getting more content. I wouldn't mind them doing this so long as they were not animated. Sort of like how I can't stand the animated shockwave banners here and at other sites. I don't understand why no one is doing this already. Though I have seen a few isolated examples of banner ads being shot across the bottom of the screen briefly during some shows - those are overlays though and degrade the content so maybe that's why those have been unsuccessful.

Re:use firefox and adblocker!

They are worth it, though. The mouth-breathing morons who can't block ads like we do are the types to click on the ads or punch the monkey. I consider annoying ads a "tax" for people who can't be bothered to educate themselves marginally about the exceptionally complex system they want information from.

Re:use firefox and adblocker!

You mean some browsers serialize that? My browser appears to fetch multiple hosts concurrently.... Unfortunately, it still won't render the page with certain missing content (e.g. slow-loading javascript crap from ad servers).

Well, the RFCs only allow 2 connections per client to one webserver; any more is considered abusive. And to support some older webservers, most browsers only get one page element per connection. So they open a connection, send one GET request, let the server respond, close the connection ... rinse, repeat.

This was OK on narrowband/dialup connections (in fact, most browsers used to render the page between elements by default, so that it would show you the whole page, then re-render as various images or other elements were downloaded and ready to display -- although as connections got faster relative to rendering time, most browsers switched to only rendering the page once when it was complete), but it sucks on broadband. As the amount of time each data transfer takes drops relative to the time required to establish the connection, the establishment and resetting of the connections for each page element becomes more "expensive."

So in HTTP/1.1 they introduced a way of making multiple requests in one connection. (It may have predated HTTP/1.1 but I think that was when it was first formalized). Basically the web browser opens a connection to the server and make multiple requests at once. Then the server will respond with all the requested elements. Then the connection will close. This is considered kosher and non-abusive because it doesn't require spawning a whole lot of connections at the server; everything is done in one.

However this isn't enabled in default in Firefox; you have to go into the about:config page and turn it on, and set the number of requests per connection to something reasonable (I think 8 is the max).

Also, it requires a certain amount of intelligence on the part of the browser to do this correctly. There are certain kinds of requests that shouldn't be pipelined (PUT requests, for instance), and some older servers may not like it. However, I think we're moving pretty quickly towards a time where it can be made the default.

Re:use firefox and adblocker!

So in HTTP/1.1 they introduced a way of making multiple requests in one connection. (It may have predated HTTP/1.1 but I think that was when it was first formalized).

Close. It was possible to make multiple requests per connection in even the HTTP 1.0 spec. HTTP 1.1 simply made that behavior the default. In other words, you had to specify an additional header to support multiple requests in 1.0, but in 1.1 you had to specify "Connection: close" to prevent the behavior.

Good introductory overview in general. Kudos.

Re:use firefox and adblocker!

Pipelining? So Ted Stevens was right!

By the way, I set pipelining.tube.maxrequests to 128. The googles, they do nothing...

Re:use firefox and adblocker!

Quicker way:

$ sudo echo "127.0.0.1 www.block.this.site" >> /etc/hosts

No need to muck about with vi. (If you do want to advise people to use vi, it's good form then to tell them the exact keystrokes they will need. Some people aren't as smart or brave as the secretaries who used to work for AT&T, and do actually get put off by a screenful of tildes, a beep every time they press a key and no obvious way out. So: shift+G to Go to the end of the file; a to enter append mode; type the extra line; ESC to get out of {or escape from} append mode; shift+Z, shift+Z to save and exit {go to sleep ZZ}.)

Sheesh

"Nothing for you to see here. Move along."

Must be 'cause I'm using Firefox...

Like slashdot?

I probably would have had first post if slashdot did not serve up so many ads!!

Jokes aside, I do notice waiting for ads on slashdot quite often but it is one of the few sites that I allow more to get through.

Browser's fault?

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

I realize this means performing some speculative page layout that may need to be re-done when the dimensions of the ads are served. But it sure would beat waiting tens of seconds to see the page's real content.

Re:Browser's fault?

Another option is to use the "DEFER" option in the script tag. Any script within the tags will wait until the page loads before executing. I wish ad companies would start using that *sigh*

Re:Browser's fault?

They don't want to, obviously, because you may end up going away from the page (cuz you realised it wasnt the right one) before the ad loads, unfortunately.

Re:Browser's fault?

Good point. When I posted that, I had in mind the typical mainstream sites, like news web sites, this very one, etc. Web sites made by professionals. Even blog web sites tend to have a few software engineers behind em, the bloggers don't make the engine.

Don't see THAT many ad driven web site made in MS Word these days...

Re:Browser's fault?

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

It depends how the ad is served. If it's served as an external piece of JavaScript (using a script element), then most browsers will reach the script tag and won't render anything else until the script has been downloaded [blogspot.com]. This can cause a delay if the ad server is slow or down.

If the ad is served using an img, iframe or object element, you generally don't have this problem, as the browser can leave a space for the advert and carry on rendering the rest of the page.

I work for an ad serving company and most of the ads we serve are in iframe elements. The growing popularity of script elements (they seem to be used for most third-party ads now) confounds me. Generally, I'm continually surprised at how much control over the user experience most websites are willing to give to ad serving companies.

Re:Browser's fault?

I work for an ad serving company and

release the hounds.

Re:Browser's fault?

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

Most ad systems seem to work by placing a <script> tag where you want the ad to appear which loads a script from the ad server that does a document.write() to insert the actual code. This is very bad practice (and explicitly disallowed for XHTML) but even Google do it (which sucks since I have to jump through all sorts of hoops to get AdSense to work on my XHTML site).

document.write() works by actually writing out HTML and feeding it into the parser and thus parsing the page must be suspended at that point until it's finished executing, so you can't render the page until the advert has loaded.

The _correct_ way to do this is for the ad-serving Javascript to actually modify the DOM tree. But that requires the ad server developers to not be lazy and have clue, which seems to be asking too much. (or alternatively, don't use Javascript at all).

0 slowdown for me

For the reasons mentioned in the op I have several notorious slow adservers in my /etc/hosts. I don't know if they're still a problem, but doubleclick used to be horrible about taking 10 or 15 seconds to get their ad bits back to you. I'm not even particularly zealous about killing ads, but if you're stalling out my webpage then it's in /etc/hosts for you.

Abusers aren't satisfied with one kind of abuse.

The people who make ads are a self-destructive bunch. Numerous times I've waited for a Slashdot page to load while some ad server took its time. Abusing me with abusive, dishonest ads wasn't enough, they wanted to abuse me by wasting my time, too. Mentioning the problem to Slashdot editors brought only temporary fixes, or no change.

So now I don't see the ads at all, thanks to Firefox's AdBlock Plus [mozilla.org] and NoScript [mozilla.org] add-ons. (I recommend NoScript only for people who don't mind fiddling with permissions for each new web site.)

I guess abusers aren't satisfied with only one kind of abuse. I can dimly remember some of the Slashdot ads. When they weren't misleading, they were generally stupidly written. People with no technical knowledge shouldn't work for technical companies.

Re:0 slowdown for me

If you're interested in populating your hosts file, check out http://www.mvps.org/winhelp2002/hosts.htm [mvps.org]. There's a downloadable hosts file that's 406k, and was updated on July 31st, 2007. If you're running 2000/XP/Vista, be sure to read the Editor's Note about steps you must take to use a large hosts file.

Re:0 slowdown for me

Additional to my note, looking through the specific host file you reference I see it blocks a load of porn sites.

These are wanted and are not crappy adverts

ahem..

Re:0 slowdown for me

I'm not even particularly zealous about killing ads, but if you're stalling out my webpage then it's in /etc/hosts for you.

Agreed. For the most part, I'm happy to let ads load, but there are four things that will get you added to my killfile real quick:

* Boobies (or anything else that might get me fired)
* Shaking, flashing, screen grabbing, "look at me!" type stuff
* Fake dialogs and error messages
* Slowdown

I just started filtering for the last one recently. For about a week straight I kept seeing "waiting for ads.doubleclick.so.goram.slow" in my status bar, and eventually I just adblocked their entire domain.

Also, slashcode is apparently unable to figure out what to do with ul and li tags.

no slowdown at all...

...for me at least. Blocking Google Analytics, Doubleclick, etc, with noscript has made my browsing experience much smoother. Not only is it nice to not have the random pauses while it hits the ad-server, not running the javascript has helped the render time on some pages as well (even if you still run the javascript for the page itself!)

Display the page before the data's all loaded

Yes, I had noticed it recently too, where the page isn't displaying because of waiting for a response from an ad server.

So why don't all web browsers start displaying the data they do have, rather than waiting for the ad server to submit it's data first? If there's a delay in downloading an image on the site or a style sheet it still starts displaying and when the image/stylesheet is downloaded the page is re-rendered to reflect that. So what is it about the page design that forces web browsers to not display anything if the delay is due to an ad server?

Re:Display the page before the data's all loaded

Javascript is usually involved. Because Javascript is single-threaded and does in-order execution, if an ad uses Javascript, then waiting on that javascript to finish will hold up the rest of the page.

High-CPU Flash Ads

What pisses me off are badly designed Flash ads. They use plenty of CPU power just to animate something completely useless. Last year Dell was running this ad on my local newspaper's site that took 80% of my CPU just to animate FALLING SNOWFLAKES. I complained to the website, and they took it down.

Some Flash ads barely take any CPU at all, and those are honestly fine by me, but some just hog my resources. The problem is that the people who DESIGN these ads typically have cutting-edge machines, so they don't know what it's like to run them on a shitty office machine. So, please, TEST your ads on a shitbox average computer before you force them on us!

Agreed

I run a few web sites and on some I have a geo-IP targeted ad that loads in an iframe. This particular ad is often a bottleneck so I wanted to solve it. My first idea was to run a wget on the server and cache the output to the hard disk so I can load the ad from the server instead of a 3rd party. This would also require one less DNS look-up.

Then I realized that it would completely fail because the ad is geo-IP. So the cache will always display the location of my server, and not the user.

The obvious solution is for ad companies to offer scripts to their affiliates that could be run on the servers hosting the sites. Of course that opens up new problems, like security issues. But if the code were open we could spot such issues.

In fact, that seems to me like such a simple and obvious solution. The only reason that ad companies don't do that (that I can think of) is that they want to appeal to people running on free hosts where they can't run server-side scripts. But there's no reason not to offer both IMO. I also thought that they wanted to keep things as absolutely simple as possible, and there's nothing simpler than saying "just copy/paste this into your html document". But any web master who rents hosting (shared or dedicated) knows how to upload a php script.

what ads?

What ads are you guys talking about, I see barely any at all. *turns off ad block plus, refreshes* Holy crap! How do you even go online like this? You might as well just watch TV.

Ads? or Webmasters?

The big problem is that most webmasters design their sites in such a way that they are dependent on a third party product being available prior to their pages being rendered.

Google isn't always up. Plenty of times, I see issues because my comcast connection can't see the google servers even though everyone else can get to them just fine.

It's entirely feasible to write your page in such a way that it can display data before any other files are loaded. Serve up ads in an iframe, include tracking images in an iframe or as the last element of a page, etc.

But ads aren't the only thing causing page load problems. Third party widgets, crazy fat CSS and JS files, and pages with way too many images are still a problem.

That is the reason I use a filter

Back in ages long gone, when firefox did not exist you had (still have perhaps) a company called doubleclick whose adservers would sometimes choke freezing the loading of the rest of the page. Why and how this happens? Do I look like someone who gives a shit?

I wanted it gone, and finally I bit the bullet and read up on squid and available plugins and setup my linux router to just filter all http traffic. Haven't looked back since.

Browsing without a blocker is like... well it just sucks. At times I am offcourse forced to browse the web without such blocking software and my god, the internet has become as bad as tv. Do they really think that if you saturate people with advertising to the point the original content becomes unusable people are really going to be more inclined to buy?

Apparently so. However not to me. This story offcourse neatly links to the story below about a site block firefox because of adblocker.

Well, who gives a shit. You went to far, now you gotta pay the price. If you don't get revenue from me, blame doubleclick and all those others who just pushed me over the edge.

At the moment I recommend bfilter to people who are fed up as well, it is browser neutral, works out of the box and does a lot more then just ad-blocking. Granted some flash bits require you to click them before they actually load but that is okay, because 99% of flash stuff I don't want to load.

So yes, ad-servers are slowing the net, by adding stuff to webpages I do not want. Can this be solved? It has been solved, not to the liking of those who depend on those ads being seen, but hey, fuck them. Do they care when I have to reload a page over and over again because some server borked?

Blocked because I'm paying for the pipe

My internet connection ain't free. If the ad folks want to use MY bandwidth they should pay me for the privilege.

Re:Blocked because I'm paying for the pipe

"My internet connection ain't free. If the folks want to use MY bandwidth they should pay me for the privilege."

It's funny that both the user and the website owner share the same argument. As usual in life, it's the few who make it rough for the many. Most people don't mind reasonable ads, and they don't mind contributing financially to a site they enjoy. Unfortunately all it takes is a few greedy jackass types taking the ads to extremes and ruin the concept entirely.

It's too bad there isn't a advertising standard that sites can be certified with and filters can be aware of. An ad whitelisting service, that legitimate companies would value as much as a BBB or google page ranking.

Re:Blocked because I'm paying for the pipe

If the ad folks want to use MY bandwidth they should pay me for the privilege.

How about they give you free content instead of paying in cash?

How about just fewer ads?

In my opinion we need a better model for serving ads -- or else these services need to add more servers/bandwidth.

I know that this doesn't speak specifically to the rest of your question, but IMHO, we need a better model than having ads. Just because we can have 'em doesn't mean we should all the time. It seems to me that the click-throughs, browser-tracking, etc., benefit the ad companies themselves far more than the individual content providers.

I realize I'm tilting at windmills here, but the current web ad-model has even city and local community web pages (like libraries) littering their pages with 'ads' for other parts of the same site, etc. It is really quite annoying.

/...and stay off my lawn!

Re:Use Adblock with my subscription...

I took one look at your website and immediately clicked away. No offense intended, but it didn't look like a site I would trust downloading anything from.