April 1, 2005Vanished Audio Tests A curious thing has happened in audio reviewing. On the one hand, what was once a fairly specialized audio component -- the headphone -- has become virtually ubiquitous. Its still probably fairly rare that most of us would listen to music on our primary system using headphones, except perhaps late at night, but the proliferation of MP3 players and other portable devices means that a huge amount of listening is done with these convenient gadgets clamped on our heads or stuck in our ears. Yet no one seems able to evaluate them, either at the manufacturing or consumer level. As a result, makers of phones and their ultimate users both tend to take an entirely subjective approach to figuring out whats good and whats not.
However, there is some science that can take some of the guesswork out of measuring and reporting on headphone sound, and it proved satisfyingly reliable. Pity no ones used it since a series of landmark tests in AudioScene Canada more than 20 years ago. To my knowledge, no one else has ever published anything like these reviews, which means that a very valuable tool has been lost.
Testing headphones is not as straightforward as testing most other audio components, where flat frequency response is the ideal -- generally speaking, the closer most components come to achieving a flat response, the better they sound. But the very fact that phones interfere physically with the outer ears means that the designer must build into a headphone design some compensation for the bypassed aural functions. A signal that is flat when it reaches the outer ear is anything but by the time it gets to the eardrum, as there is considerable acoustic modification by the ear itself. A headphone alters this by deforming the ear and coupling very closely with the air in the ear canal -- even more so in the case of earbuds, which are placed inside the canal. For a headphone to sound natural, it must exhibit a frequency-response curve that duplicates the modifications the ear would make. Unfortunately, every human ear is different, so no single compensation curve will be ideal.
It is possible, however, to build up a family of curves that represent the range of responses required for the average person to hear a natural sound. These curves, first produced by Dr. Edgar Shaw at the National Research Council in Ottawa, Canada -- and adapted by the redoubtable Dr. Floyd Toole for use in magazine reviews -- are shown as the shaded area on the graphs below. A number of tests performed during the 1970s and 80s showed that the headphones whose curves most closely conformed to these shaded areas were those that listening panels judged to sound best.
The frequency-response measurements shown were made using a lifelike rubber replica of a human ear of median size, fitted with a microphone at the entrance to the ear canal. The replica was mounted to a form that simulated the side of a human head, and a support some 7" away held the other side of the headphone and allowed the headphone to exert its normal force. The lower curve in each case was measured in this way; the upper curve represents a similar setup, but with some pressure applied to the outside of the earcup.
The shaded area in each of the frequency-response curves indicates what the average human ear "wants" to hear. The measured curves should not be judged by their flatness, therefore, but by their relationships to the range of acceptable responses.
A panel of listeners using a variety of music evaluated each headphone subjectively, without reference to the measurements. There was a high degree of agreement among members of the panel as to the sounds of the various phones, and the subjective results correlated very closely to measured performance: The headphones whose frequency response closely approximated the shape of the shaded area on the graphs tended to sound good to the panel, while the ones whose curves strayed outside the shaded area tended to have problems.
These measurements were made in 1979, and none of the phones tested are available today, which is why Ive identified them here only by number. But these results are typical of what we might find today, at least to my aging ears, so it should be instructive to look at the range of measured results and see how they relate to the "ideal" range of curves. To get some idea of those relationships, here are some of the comments included in the original test reports.
These curves and comments represent a small, more or less random sample of headphones tested on a particular occasion many years ago, but the technique proved its worth over and over. When it was devised, we hoped that manufacturers and other reviewers might follow the lead, but to my knowledge none ever did.
Which is why headphones are still designed and bought "by guess and by God."
...Ian G. Masters