Déjà Vu All Over Again
By Tom Nousaine
Contributing Editor
At the Home Entertainment 2005 show in New York last spring (successor to the Stereophile hi-fi show), in a seminar titled “The Great Debate—Subjectivism on Trial,” John Atkinson “invited” Arny Krueger to get out from behind his PC and “confront in person those he criticized.” At least that’s Mr. ’Kinson’s spin on things—that Arny Krueger (and people like me and Peter Aczel) have been hiding behind their keyboards, afraid to discuss the issue face to face or in public. Quotes following are from Mr. Atkinson’s column “As We See It” in the July 2005 Stereophile, including some of his quotes of what Krueger had said.
Mr. Atkinson attempts to build a self-serving argument out of Mr. Krueger’s criticism of Stereophile, namely that it “willfully ignores much that is known about reliably evaluating audio products,” frequently reaching conclusions and making recommendations “that are improbable if not completely wrong” and not taking “enough pains to ensure that it is publishing correct information.” According to Mr. Atkinson, Krueger is basing his assertions “on the one criticism he has repeatedly made over the years that is correct: to wit, that Stereophile’s reviewers do not perform their listening evaluations under blind conditions.” Interestingly, Arny Krueger never mentioned blind tests in his criticisms. This straw-man argument is entirely Mr. Atkinson’s personal invention.
Mr. Atkinson has long pointed out that no reviews currently published in Stereophile use blind tests. This, of course, is true, but the literal criticism in Krueger’s words is not that blind tests aren’t used in listening evaluations but that what “is known about reliably evaluating audio products” has been ignored by Mr. Atkinson and Stereophile.
Mr. Atkinson then goes on to relate an anecdote about a 1978 blind listening test conducted by Martin Colloms, which showed that subjects were unable to distinguish reliably between three amplifiers, two solid-state and one tubed, in a blind comparison. Following that comparison, Mr. Atkinson claims to have made a solid-state buying decision based on those results but later he “began to realize that even though the sound of my system with the [solid-state] Quad was the same as it ever had been, the magic was gone…until I replaced the 405 with an M&A tube amplifier two years later.”
He then concludes that “normal listening had revealed what the blind test had missed” and that the original comparison failed to prove that the amplifiers sounded the same—therefore any criticism of sighted listening must be wrong. Furthermore, he makes an interesting observation about psychological bias—that “the nonaudio [i.e., psychological] factors were all working in favor of my not hearing any problem with the amplifier,” as if anybody would have conscious awareness of bias held subconsciously. He conveniently ignores that those “nonaudio factors” gathered over a two-year period are more likely to have influenced his opinion about the sound of the amplifiers. “Nonaudio factors” certainly could not have changed the physical sound the devices reproduced.
Even so, if the amplifiers truly did sound different, why wasn’t Mr. Atkinson able to prove that (figuratively) blindfolded in the first place? Mr. Atkinson must be arguing that the techniques used to control listener bias, by themselves, interfere with auditory perception. While this is literally true in the psychological perspective, there is no way that a figurative blindfold could have any effect on the acoustical sound being reproduced by a playback device or system.
There had been more than a couple of dozen bias-controlled listening tests published in the popular press by the early 1990s, all of which showed that any competent amplifier device, with flat frequency response into a loudspeaker load in an acoustic space and not overdriven, is capable of delivering a sonically transparent representation of the input signal into the load and into the listener’s ear. Thus it is patently clear that the Krueger criticism hits the nail right on the head. John Atkinson and his staff regularly publish reviews and recommendations about the sound of equipment such as amplifiers, cables, bricks, stands, and assorted “tweaks” that have never been shown to have a sonically audible effect in any way unless they have demonstrably and measurably worse performance.
One doesn’t have to use blind tests to recognize and acknowledge the existing evidence. Indeed, Mr. Atkinson’s entire argument and the main focus of Stereophile is to encourage readers (and industry personnel) to ignore existing evidence and rely on Stereophile’s reviews and other uncontrolled listening that simply promote existing high-end mythology. Indeed all the arguments about the problems of bias-controlled listening tests revolve around the fact that the controlled listening tests have never verified nonsonic artifacts.
In the final analysis, The Great Debate, like paranormal psychology, Nessie, Bigfoot, and Alien Visits, only keeps resurfacing among high-enders because they aren’t able to bring any real evidence to the table. Amp-sound guys are all café racers—the rides look good parked outside the coffee house, but don’t expect to see any of them at the proving grounds or the race track.
It’s also interesting that John Atkinson pointed out that he, personally, had participated in more blind listening tests than most “objectivists.” While this is probably true, it could also be pointed out that he apparently also has more practice in ignoring, misrepresenting, and misinterpreting controlled listening test results than anyone else.
Editor’s Note: About 12 years ago, I published in The Audio Critic the following comments on The Great Debate. “…the unending back-and-forth on the constantly shifting ground of this controversy has left many audiophiles with the impression that a fundamental clash of philosophies was taking place, the audio world’s equivalent of capitalism vs. socialism, religion vs. atheism, Republicans vs. Democrats, protectionism vs. free trade, batting pitchers vs. designated hitters, etc., etc. That is an enormous misperception. The way the lines are drawn today, the debate doesn’t have two arguable sides. It’s more like laetrile vs. the AMA or the Ku Klux Klan vs. civil rights.” A lot has changed in audio in the last 12 years but apparently John Atkinson hasn’t.