Consumer Reports – Still Clueless After All These Years

My disdain for Consumer Reports started in the late 70’s, when I was an audio salesman. They frequently gave lousy products a “Best Buy” rating.

Back then audio was getting better on an almost daily basis.  All receivers and amplifiers had Total Harmonic Distortion (THD) down to a fraction of a percent.  It was amusing to hear people insist they could hear the difference between .05% THD and .03% THD.  It was like claiming, “Most people can’t read the Surgeon General’s warning on a pack of cigarettes from 200 yards away, but I can.”

With the exception of some very high-end equipment attached to high-end speakers, the difference between receivers was inaudible.  There was one notable exception – the Pioneer SX series.  The most popular model was the SX-780.  The phono preamp (which was used to power an ancient artifact called “the turntable”) was as hissy as a nest of angry vipers, and the FM reception sucked.  Virtually any other receiver in its price range was superior, and it didn’t require a trained ear to hear the difference.  Consumer Reports gave it a Best Buy rating.

The most variable component in an audio system has always been the speakers.  These days even inexpensive speakers can sound impressive, but back then there was a huge variation in sound.  Customers who had grown up listening to crappy speakers were often most impressed with “tinkle boomers,” speakers specifically designed to impress inexperienced listeners with a shrill high-end and a thumpy bass.  Better quality, smoother-sounding speakers provided a more realistic sound and a much better listening experience, and it took some time to explain and demonstrate the difference to customers.

CR’s speaker ratings were all over the place.  They’d give equally high marks to tinkle boomers and great-sounding speakers.  It was as if they’d thrown the model names into a hat and plucked them out at random.

Many customers would come into the store carrying their beloved Consumer Reports and insisting on buying based on their ratings.  At first, I’d try to steer them away from crappy recommendations, but I learned my lesson quickly.  People would get upset, sometimes outright angry, when you dared to contradict CR.  One customer angrily waved a rolled-up copy of CR in my face and yelled, “These people have never lied to me!”  From that point on, when someone came in carrying a CR I just sold them whatever crap they recommended and moved on to the next customer.

My next gig was selling computers, and CR continued its fine record of recommending garbage.  The Leading Edge Model D got their best buy rating, despite the fact that after two or three hours of use it would overheat and lock up.  I seem to recall them also giving a high rating to Coleco’s horrible Adam, but I can’t confirm that.

As the years went by I continued to be amused by their mistakes and stupidity.  I remember a test of primers, where they painted samples and then tested them raw, without any paint on them.  It should be obvious that’s not the way primers are designed to work.  They rated a car unacceptable because if they were driving at high speeds and tried to make an abrupt turn, jamming the steering wheel in an instant, it didn’t perform well.  Because, yeah, we all try to make 90-degree turns at 60 MPH.  Milwaukee’s Best was at the top of their list as the best-tasting beer.  And their over-the-top attempts to tip over a Suzuki Samurai were appalling.

About ten years ago I was a computer tech for a large R&D center.  The company’s products include home appliances.  I was fixing a computer in an engineer’s office during a meeting.  They were trying to decide if they should degrade the quality of their refrigerators to get a better rating from CR.

Engineers have been tweaking and honing the performance of home appliances for decades, so there isn’t much room for improvement.  The performance of refrigerators or freezers varies according to the temperature of the outside environment.  Obviously, higher temperatures require more energy.  More importantly, a box designed to work best in a 68-75 degree range isn’t going to perform as well in 90 degrees. That would require a box designed to work in the hotter environment.

CR was testing in a 90-degree room, and the company’s refrigerators weren’t performing well because they were designed to be most efficient at normal room temperatures.  The engineers were discussing the merits of redesigning their refrigerators to work best at 90 degrees, degrading their performance in more realistic temperatures, to get a better Consumer Reports rating. There was some speculation that competitors who had earned better ratings were doing just that.  The meeting adjourned without them reaching a decision.

So not only was CR giving bad advice because of their crappy testing procedures, they were contributing to devices being downgraded to win their approval.

A few days ago I was visiting relatives, and my mother-in-law was going through her mail.  She handed me the latest issue of Consumer Reports, saying, “I know you don’t like these guys, but they have GPS’ in this issue.” (I’ve mentioned getting one.)

They also reviewed freezers.  Remembering the engineering meeting, I turned there first, and in the second paragraph they explained their testing procedures.

They tested them in a 90-degree room.

12 Comment(s)

  1. Yes, but the Pioneer Stereos have LRF support ?

    William Wangerien | Sep 4, 2009 | Reply

  2. Yes, they did. And that’s SO important. Almost as important as KPD.

    If you’re clueless about these important features, they’re explained in The QH Podcast “Audio Insanity.” http://www.davehitt.com/podcasts/index.html#Audio

    Hittman | Sep 4, 2009 | Reply

  3. This just in! Consumer Reports endorses Obamacare!

    From Le.gal In.sur.rec.tion

    Consumer Reports has decided to weigh in on the health care debate in favor of the Democratic health care proposals, or what CR euphemistically calls “health care reform.” Having decided to take sides, I think it is fair to hold CR to the standards it expects of the manufacturers and service providers reviewed monthly in its flagship magazine, and on its website.
    Here is the mission statement of Consumers Union, the tax-exempt parent company of Consumer Reports:

    Consumers Union (CU) is an expert, independent, nonprofit organization whose mission is to work for a fair, just, and safe marketplace for all consumers and to empower consumers to protect themselves. The organization was founded in 1936 when advertising first flooded the mass media. Consumers lacked a reliable source of information they could depend on to help them distinguish hype from fact and good products from bad ones. Since then CU has filled that vacuum with a broad range of consumer information…

    Brian Riley | Oct 5, 2009 | Reply

  4. Advertising didn’t flood the mass media until 1936?

    Wow. That’s just incredibly clueless.

    Hittman | Oct 6, 2009 | Reply

  5. It’s funny you mention the “tinkle boomers” because I went to school with someone who applied this concept (in the extreme) to car audio. This was back in the late ’90s were everybody was getting those huge trunk subwoofers (without upgrading any of their other speakers of course). Anyway, this guy decided to take a trip to radio shack and get his own set of tinkle boomers for his dodge neon. In the back was featured an “Optimus” brand amp and dual 12″ subwoofers, and in the front on his front pillars were mounted the tinniest, harshest-sounding tweeters ever known to man.

    And it sounded like shit. It was actually physically painful to listen to music in that car at any volume.

    Brian Stilson | Oct 7, 2009 | Reply

  6. I would also like to mention that CR gave the 2002 PT Cruiser high ratings. In the four years I’ve owned it I’ve had to have the water pump replaced twice, as well as my entire steering rack. They should have tied Chrysler to AMC and shot them both. After every government bailout, the cars get even shittier!

    Brian Stilson | Oct 7, 2009 | Reply

  7. It was actually physically painful to listen to music in that car at any volume.

    But I bet he had convinced himself that it sounded great.

    Wny not put that speaker system in your PT Cruser? Then you could have the worst of all worlds.

    Hittman | Oct 9, 2009 | Reply

  8. But I bet he had convinced himself that it sounded great.

    Oh he did. That’s how I heard it. He gave me a ride so I could hear how awesome it was.

    Brian | Oct 13, 2009 | Reply

  9. I remember Bicycling magazine skewering Consumer Reports back around 1990 for knowing absolutely nothing about bicycles, bicycling, and bicycle safety. I think the controversy was that they dissed extremely well made brands for brake placement. Consumer Reports wanted the brakes on middle levers, not on the side of the handlebars, if I recall right. Over the years, they learned – that kills people.

    Ben Madden | Dec 19, 2010 | Reply

  10. I had to come back and read this article as I’ve gotten into vintage audio and recently purchased a new receiver. I had to make sure it wasn’t the crappy one you were talking about. Luckily I got a Kenwood KR-6160 (that’s an ok one, right? I hope?) and not the Pioneer.

    Brian | Nov 17, 2015 | Reply

  11. Kenwood was good stuff. Very solid, good sound. I’ve got a Kenwood amp that’s thirty years old and still works great.

    Hittman | Nov 18, 2015 | Reply

  12. Awesome! Mine seems to be built like a tank as well.

    Brian | Nov 19, 2015 | Reply

Post a Comment