Truth in Advertising? 393
PerformanceEng wonders: "I work as an engineer for a large technology company in the U.S., and have been privy to what I find a interesting practice. It's well known that marketing data sheets often paint the best picture of a product while leaving the devil in the details. I've come to expect this, and when I am evaluating technology, I always have a skeptic's eye for claims made by the sales and marketing folks.
However, I've also witnessed our product go into test labs (usually for the purposes of running a series of tests for a 'bake off' in a trade publication). Not uncommon is the attempt to 'tune' the configuration of the device under test to perform in the best light (not unlike tuning your car to pass emissions tests). I have seen it go as far as exploiting weaknesses in the test that, if the test operator discovered, would be considered bad faith. To the other engineers: Are you aware of this kind of practice at your company? To the IT professionals: How much faith do you put in these sorts of publications and their 'bake offs'? To everyone: When does spin doctoring cross the line and become false advertising?"
Peer review (Score:4, Insightful)
Oh, you work for Intel then.
To answer your question of false advertising, I would say keep to the standard that most of us scientists do: Specifically, peer review and ensure that your results can be duplicated by said peers. If results cannot be duplicated, then it is false advertising.
Well known truth (Score:2, Insightful)
The product brochure may lie or hide facts, but the product's technical details book (like the manual for Kyocera VMSE47 Phone) HAS to tell details and truth.
I always make it a practice to read the technical manual of any product i buy over the web. if the company can't provide the manual, then it isn't worth buying.
Political Spin. (Score:1, Insightful)
Been reviewing the previous election, have you?
Re:Peer review (Score:4, Insightful)
To answer your question of false advertising, I would say keep to the standard that most of us scientists do: Specifically, peer review and ensure that your results can be duplicated by said peers. If results cannot be duplicated, then it is false advertising.
Even science has a problem of touting the best data and "leaving the devil in the details." Research is driven by money just as much as industry. If you're not producing good results, you won't get funding.
Truth - Advertising? (Score:5, Insightful)
Analyze, analyze, analyze (Score:5, Insightful)
I think it's just a fact of life: everybody wants their product to be seen in the best light, and to sell well (in the case of commodities or services).
That's why Amazon.com has reader reviews, sites like epinions.com exist, and Slashdot has moderator points. It's also why there are hardware review sites -- we can't just trust the manufacturer's PR now, can we?
So, people may be inherentely biased and often untruthful, but with proper monitoring (read: community involvement), the truth will out.
Re:Consumer Reports pays cash (Score:1, Insightful)
Re:Analyze, analyze, analyze (Score:1, Insightful)
Astroturfing is even more common on epinions (where it might not just be the first review you can't trust).
Oh, and nobody has submitted a story to Slashdot pointing to a poorly written essay they wrote themselves. No, that would never fucking happen.
When? Instantly. There is no gray area. (Score:2, Insightful)
Instantly. There is no gray area between honesty and dishonesty. You either tell the truth, or you tell a lie. Your company either attempts to subvert tests [i.e., lies], or it doesn't [i.e., does not attempt to lie]. No ambiguity exists in this case.
Your question reminds me of a question posed on the cover of a national "news" magazine in the wake of revelations that the New York Times had published falsified news reports. Their question was, to paraphrase: "Does this signal a new standard in journalism?". Of course it signaled no such thing; it only signaled that some publications, or at least reporters writing for them, were willing to be dishonest and to print lies. I wonder if the author of that question was perhaps a bit hopeful that, yes, this event did signal a relaxing of ethics?
Not in IT, but... (Score:2, Insightful)
... when I worked for a German owned plumbing fixture manufaturer's US subsidary, we had to have all faucets certified for lead contanimation (leaching from the solder and brass compounds). As it turned out, a lot of what we were already selling in the US market would not come close to passing. The Fatherland offered to send faucets that were garanteed to pass. All we had to do was tell them what levels that they needed to meet for a particular model (has a lot to do with the length of the flow chamber).
They seemed quite upset that the units had to be pulled at random from stock. Maybe they were just to use to cheating.
Re:Truth in advertising? (Score:1, Insightful)
Read carefully (Score:4, Insightful)
Marketing materials do not set out the faults of the product. This is not the role of marketing. Marketing aims to connect buyers to sellers. Providing information about faults does not help to make that connection. Also, many of the "tests" cited by marketers are labeled with titles such as, "Customer Success Story". This should be a clue that the material will not detail unsuccessful characteristics of the product.
Finally, marketers in most companies are not technical experts. They have to rely on the information provided by engineers and programmers. Many companies avoid ever telling the marketing department anything negative. As a result, in many cases, marketers aren't lying when they make claims -- they're explaining what they were told. Many of these marketers, especially the ones writing up collateral, are junior, new to the company, or even working on contract, so they don't have the depth of knowledge to tell that they've been given misleading information. Other people in the company sometimes lie to the marketers. It's not always black and white. (Not that all marketers tell the truth, of course.)
Re:Analyze, analyze, analyze (Score:3, Insightful)
Re:Consumer Reports pays cash (Score:2, Insightful)
You can't look at the top rated model and decide that it is the best one long term. The ratings in a CR review represent how the products performed during the test. The ratings do not necessarily represent the best products.
Nearly every CR review has another section that details the reliability of the brands represented in the test over a period of time that they've been testing the products. Also, poorly performing brands will generally be noted as such in the performance ratings if a product scores well.
Whats new? (Score:3, Insightful)
Lies sell, since most people are stupid and believe whatever they are told.
"Yes, None, Ten years ago" (Score:5, Insightful)
No, I trust none of these "bakeoffs". Or any other IT advertising for that matter. There isn't a single mainstream IT rag which is even marginally trustworthy. Go ahead and, instead of reading just the bakeoff that you're looking for, read an article about something you already know about (through hands-on experience with all the primary alternatives, including a FOSS alternative if it's software and there is a FOSS alternative). Note how much stuff they get wrong, how shallow the article is, and how it almost reads like an advertisement. The same is true for cars too, largely, at least from what I've read. I can't comment on other industries since I'm not particularly familiar with their trade press. Note, however, that I still don't trust them at all - I expect they're just as bad. It's just that I don't make enough decisions relating to those industries' products to warrant reading the trade press - instead I go to the store and carefully examine the alternatives.
This sort of thing crossed the line into fake advertising at least a decade ago. Companies routinely make absurd claims and get away with it. There's just no political interest in enforcing it. At best they'll include fine print in their ad. If it's a print ad, maybe you'll be able to read it. It's been a while since I've seen an ad with fine print whose fine print didn't take up at least 10 lines of extremely small type. Television ads are a joke, it's impossible to read the fine print at broadcast resolution, regardless of the size of your TV, and it typically takes up a whole screen.
What can we do about it? Elect governments with some spine. These sorts of advertisements will continue to be successful so long as people are poorly-educated, and people will continue to be poorly-educated unless there is a strong collective agreement in place that says "yes, everybody needs some minimum level of education, otherwise they're prone to manipulation and our society is controlled by those who control the media or the other forms of information dissemination." It's funny, isn't it, how political campaigns in the US almost exclusively take the form of commercials? (Except for the "debates", which are a joke to everybody outside the country.)
Note that when the US was founded, everybody who advocated democracy made sure to point out that the requirements for democracy included an educated public, free speech, and free press. People have totally forgotten the education bit and the press bit. (A government-controlled press is no more effective at disseminating important information than a press controlled by an aristocracy - corporate or otherwise.)
Re:Truth - Advertising? (Score:3, Insightful)
You mean Code Red isn't a sports drink for advanced athletes? That I shouldn't be on a dozen prescription drugs? That my children aren't better taught by a talking book? That school loans aren't the source of happiness for all successful students? That cross-over SUVs aren't station wagons? That my computer doesn't make the Internet go faster?
Re:Video drivers (Score:4, Insightful)
In the mid-1990s my company did quite a bit of graphics card testing (still does, but it was much higher profile back then.) It was pretty routine for us to get baked drivers (and there were some very impressive cheats) less routine, but still common was to get a board with a BIOS cheat, which would do anything from altering its own board timings to be out of spec (sort of "overclocked out of the box") to running code that would adjust the PC's heartbeat interrupt to slow the clock ticks to make the board appear faster if benchmarked using the PC's own clock.
In the end the best solution we came up with -- because we worked with a lot of alpha/beta silicon since we tracked chips more than boards -- was to more or less formalize the cheats and what was/wasn't permitted, and also to give the companies that submitted alpha/beta hardware to pull the results before publication, so that if one company pulled a fast one, the others that would be look bad in comparison simply wouldn't be compared; this resulted in a sort of a stalemate of cheating.
The most extreme (but permitted) cheat I ever encountered had the company involved paying over $100,000 to have a custom graphics driver written overnight that incorporated an optimized version of parts of the DirectX rendering engine (this was ~ DX5 era). When they found out their primary competitors pulled their boards from testing, you can imagine they were less than pleased.
The point of all this: a competent testing lab, particularly part of a magazine "shootout," should be well aware that cheating is taking place, and prepared to identify major cheats. Back in the heyday of PC Magazine in the mid-90s, their benchmark people were top notch and the benchmark ran a considerable number of cheating tests to clear out the more bogus attempts.
Oh, and you can be pretty assurred your competitors are doing the same things you are.
Re:Truth - Advertising? (Score:5, Insightful)
No, you sound practical. Advertising usually affects the reptilian part of brains, preying on our patriotism (truck ads), vanity (gyms, makeup), greed (everything), etc. Its shameful there aren't controls on corporate "free speech" as McDonalds and others hire child psychologists to craft effective ads for their unhealthy products.
This is the golden age for ads. They're everything. Every webpage, above the urinal, people aren't very skeptical and have disposable incomes, the art of creating a working fad/meme is getting perfected, celebrities are manufactured from scratch, etc. And this is what people want.
The problem is two-fold. People, in general, need to take a good look at their consumerism and corps need controls on what they can and can't say. I'd like to see informative ads telling me cost, MPG, etc but a typical car ad is all mom, america, and apple pie stuff.
Similiar post over at nerdfilter today. [nerdfilter.com] The video is hilarious and worth watching.
Re:Consumer audio (Score:3, Insightful)
It may be true that there is a capacitive charge on the cable (due to the inefficiencies of the dielectric, but that's beyond the scope of my ability to explain), and the degree of the charge may affect the sound quality. I'll agree that this is possible. BUT -- and this is a big but, at least as big as Roseanne Barr's -- this charge will vary with the signal that is present on the cable. You might be able to capacitively charge the cable, but that charge will change as soon as a signal is applied. It may increase or decrease. It will do this very rapidly, and the result a few seconds after listening to a particular musical selection will be no different than if the cable had not been precharged.
If there is such a phenomenon as "burn-in" on cables, it would represent a breakdown of the dielectric strength of the insulation, which in turn would cause signal strength degradation and increase coupling. This degradation might not be equal in all frequencies, and therefore represents a discoloration of the audio signal from its source to its destination. I can think no case where "burn-in" would improve the integrity of the signal. It might improve the listener's perception of the signal, based on the listener's personal preferences, but it certainly wouldn't improve the integrity.
bake off tuning (Score:2, Insightful)
There's no equivalence between marketing numbers and YOUR reality because the variables change. So in a bake off you have to try tuning for the customer's variables. And when you do, you find that the performance falls off because, duh, marketing published the optimized numbers. That's not deception, but a common denominator with the competition.
Think about it. When was the last time the "how fast" question was answered with worst case numbers? You get best case, always.
Then you bake off and it sucks.
Then if the vendor has a good Sales Engineer and Support team, they'll tune it for your variables and get the performance where you'll make a buy.
Re:Consumer Reports pays cash (Score:3, Insightful)
I have worked for a number of years in various roles in the mainframe IT industry, and have repeatedly observed (from both sides of the customer/vendor fence) that the best-prepared consumers take the vendor's claims with a grain of salt and ALWAYS do their own independent benchmarking to see how the product works in their own application environment.
This certainly isn't constrained to big-ticket hardware products. A responsible consumer always tries out a small pilot operation on ANY product -- hardware or software or even services -- to see how it stacks up for them.
The biggest cost is the effort involved in evaluating and maintaining a competent staff with which to do the evaluation, something that has gone by the wayside as companies get more streamlined and lightweight in their quest for the perfect business enterprise (i.e., one with a richly compensated top management presiding over a single layer of "operational" management who outsource everything else to the lowest bidder, with cost as the only metric).
The fashionable trend today is to make one decision and put all the chips behind it, eyes closed the entire time.
How many companies can justify their "standardization" on any given product (I'm thinking Windows here, but it applies everywhere) by any sort of intelligent data acquisition method (sorting a spreadsheet by price is *NOT* intelligent in any real-world decision, as real-world issues are too complex to fit into a single column)? How many conduct honest evaluations of their decisions a year or two or ten after they are made? How many even bother to break down costs and look for escalating costs (like the cost of defending against worms and viruses)?
"Benchmarking" (Score:2, Insightful)
This is the reason I completely ignore benchmarks and spec-tests. I go straight for the heart of the matter: reviews by users. Go to Google and type in Any Product review, and you'll get your fair share of honest reviews and feedback.
When forty nine out of fifty reviews say something is a piece of crap, then you know two things:
1) It's a piece of crap
2) The fiftieth reviewer is the same guy who faked up the benchmark posting under a pseudonym.
Re:Video drivers - PC Mag tests (Score:3, Insightful)
The 2D ZD WinBench had a string "The quick brown fox..." it rendered in different colors and sizes using the Windows GDI, and the IIT BIOS embedded it. I believe the parts were still ISA based, so embedded the string in a ROM on the card would actually offer more bandwidth than passing it over the ISA bus did.
Re:ATI's 'Quake' optimization. (Score:3, Insightful)
Re:Consumer Reports pays cash (Score:4, Insightful)
Based entirely on your comments, I would suggest that is the true strength of Consumer Reports' reviews--you have not just a ranking, but also a detailed explanation of how that ranking was arrived at.
The people who buy based only on a final arbitrary score or ranking are just as screwed as the people who choose a CPU based solely on its clock speed, or an audio amplifier based solely on its output power. Sure, such people exist, but there's useful content in CR for those who are willing to look.
One hopes that people willing to plunk down the cash for a copy of CR are also willing to spend a small amount of time reading the whole article before they buy a twenty thousand dollar vehicle....
Re:Truth - Advertising? (Score:2, Insightful)
Re:ATI's 'Quake' optimization. (Score:4, Insightful)
I mean, cheating on benchmarks is one thing (the card is just slower than it benchmarks), but the WHQL is supposed to be a stop gap measure: sure, it's Windows, we hate Windows, whatever. But where I work, we use it. And WHQL drivers are something that you're supposed to be able to lean on; they're drivers that may not be the latest, greatest, but they will work.
I can't tell you how long it took to track down that the RAID WHQL certified drivers were the problem. It's something you're supposed to be able to put a little checkmark next to when diagnosing problems, a "it can't be that!".