Last week, I posted a challenge to listen to two pairs of audio samples, and answer two simple questions about them. If you haven’t tried it yet, you might like to check it out before you read the answers.
Six months ago, I did a similar experiment to see if people could tell the difference between compressed an uncompressed audio. Although the results didn’t suggest that anybody could tell difference, it had one key flaw: it’s impossible to prove that nobody can do something. Without testing all the people all the time, you might miss somebody, and that somebody might be working for the Philadelphia Inquirer.
This time, I approached the problem from the opposite direction: is it possible for people to think they’re hearing differences that don’t exist, and are these false perceptions vulnerable to suggestion?
The test is fairly simple. In case you hadn’t guessed, all four audio samples are exactly the same.
After hearing two identical samples of music, people are asked if one sounds better than the other, or if they sound the same. This is question one.
Respondents are then told that the next two should sound different. The wording of the question suggests that there’s a correct answer. The third option is changed from “They sound the same” to “I can’t tell”. This is question two.
If people’s perceptions of what they heard were reliable, you’d expect everybody to pick “They sound the same” in the first question and “I can’t tell” in the second. If just one person doesn’t, then it means it’s possible for us to be mistaken about what we hear.
If our hearing is suggestible, you’d expect more people to perceive a difference between the second pair of samples than the first pair.
I asked 100 people. 56% claimed to hear a difference between the first two identical samples.
When the question was loaded to suggest the samples weren’t the same, that rose to 69%.
On this graph, a “right” answer is that the samples sound the same, and a “wrong” answer is that they’re different.
In both questions, most people thought they heard a difference where there was none. Even more people perceived an imaginary difference when the question was loaded.
That means that you can’t trust what people think they’re hearing, especially when they have preconceptions about what they’re going to hear.
This is important, because if we can’t count on our own ears then we certainly can’t believe anybody’s assertion that they heard a difference between two recordings unless they consistently perceived this difference under blind conditions, where they didn’t know which recording was which.
Under these circumstances, reviews like this one tell us nothing useful about the quality of recorded sound – there’s a good chance we’re just reading about the preconceptions of the listener. We might as well ask the critic if he is an above-average driver.
It’s worth remembering this next time somebody tries to sell you magic speaker cable. The salesman will encourage you trust your ears, because he knows they’ll do what they’re told.
It seems that this effect* isn’t limited to recording quality, but also extends to aspects of performance. Guardian columnist Ben Goldacre wrote about a similar study a few days ago. Women were filmed playing the violin while wearing a variety of different outfits. The videos were all dubbed with the same recorded sound, and then shown to an audience of trained musicians who rated them for technical skill and musicality. The viewers thought that girls in slutty outfits played badly, even when the music was exactly the same.
It seems even trained musicians aren’t much good at consistently assessing what they hear. It’s difficult to know if the same applies to critics. Perhaps they’re a breed apart. I guess the only way to find out is with an experiment. If I can think of a way to do it that’s not professionally embarrassing or physically invasive, perhaps that’s what we’ll do next. How the heck I’m going to get volunteers, I don’t know. Free drinks, maybe. That would work on me.
If you’ve investigated the limits of your own aural perception, I’d be fascinated to hear from you.
As always, the comments are here for you. I know it’s the Internet, but let’s at least try to keep it civil.
* which probably already has a name, but which I’m going to call “David Patrick Stearns Disease“. Its opposite – the Alex Ross Effect – is when, despite repeated listening, you don’t notice how genuinely awesome something is until it is pointed out to you by a trusted public figure.