Experience teaches us how to organize our perceptions into categories, which for efficiency must be unconscious most of the time. Sometimes, however, one becomes aware of their existence in an amusing way. When young men started to wear their hair long, in the 1960's, I sometimes mistook a man for a woman. My error was absurd: Men do not look like women. But ‘hair length' was apparently a category my mind had established as efficient for telling men and women apart. When this category turned out to be misleading, it gave way to new categories, but only after some weeks of errors on my part. If men had never worn long hair, I would never have become aware of the existence of this particular category in my own mind.
It is only sensible to assume that there will be defects in the New, whether the New is digital recording, solid-state circuitry, analog tape or Edison cylinders. I was therefore startled when an executive of a well-known company involved in digital audio seriously claimed that his company's machine was perfect. In a phone conversation, he said, "You will never, under any circumstances, no matter what the source material, hear the slightest difference between Line In and Line Out." If this machine is indeed perfect, it marks the first time human beings have produced anything at all which is so.
A danger, however, in simply listening to the New on its own terms is that one may ‘normalize' on it. Normalization is the phenomenon in psychology whereby, for example, after extended wearing of eyeglasses which make straight lines look curved, you come to perceive the lines as straight again. (When the glasses are then removed, you see straight lines as curved in the other direction.) I was recently presented with an example of normalization in audio when I compared two amplifiers in a lecture. Most people in the audience agreed that one of the amplifiers sounded more like real music; but afterwards, one listener told me that he preferred the other amp. However, he went on, he also preferred its sound to that of a live orchestra. That is a person who has truly normalized on reproduced sound!
When evaluating microphones or speakers, one has no choice but to compare their performance directly to the live sound. For certain other components, however, one can use a "bypass test." In the double-blind listening test of digital audio in my Caltech lab, the sound heard by each listener had to come through transducers in any case; so we used as a reference, not the live sound, but a "direct feed" of that sound via microphones, amplifiers and speakers. In other words, live music was played in a concert hall, and our reference was that music as heard in the Music Lab via microphones, amplifiers and speakers, with no recorder in the chain.Then the recorder being auditioned could be inserted into this listening chain and a comparison made between its output and the "direct feed."
An informal listening experience underlined the importance of having an identified direct feed as a reference. In December, 1981, twenty listeners heard a half-hour of music direct from microphones, while the microphone feed was also recorded on a professional digital recorder. After a short break, everyone listened to the playback. In such a comparison, any difference must be a degradation. Yet two listeners preferred the digital playback to the live feed!
In our formal listening test, the decision to have a high-quality direct feed as a reference was rewarded by two particular comments. The first came from Prof. F. Brock Fuller, of the Caltech math department, who was verifying with me one day that the equipment was working properly before testing was due to begin. He said of one machine, "If you go down a sort of recording engineer's checklist in your head while listening, you would conclude it's perfect or near-perfect. But when you switch to the direct feed, you hear that the unit has taken away the beauty of the sound."
Rules of the Game
by James Boyk
Hi-Fi News & Record Review (England), January, 1983
Copyright © JB 1983, 1997. All rights reserved.
I came across another example when I visited Greenfield Village in Dearborn, Michigan, last year. (This is the Ford Museum's collection of interesting buildings brought from all over America and surrealistically plunked down together where none of them is in context. They might as well be replicas.) In Thomas Edison's laboratory, we saw a cylinder phonograph. The guide's talk included the fascinating statement that Edison ran "live vs. recorded" demonstrations, and that listeners found the playback indistinguishable from the live sound! Surely, it was just that their 19th-century mental categories did not allow for any sound that was not live! Surely, after a number of comparisons, they would have discriminated correctly every time!
The introduction of analog tape recording in the 1940s was marked by claims that it, too, was perfect; and the same happened with transistor circuitry in the ‘60s, at least among those who measured rather than listened. Whether the claims were true or not, the collisions of solid-state technology with the older tube technology, and of analog tape recording with the older direct-to-disc recording, have been fruitful for our understanding of how sound-quality relates to measurement, and for our awareness of our own mental categories of perception.
Currently we see another collision, between digital and analog recording techniques; this should also give us much insight into our "mental set" for evaluating live and reproduced sound. But, as always, the danger in evaluating a new technology is that the old technology has shaped our perception. If Old defects are corrected in the New, we will notice that fact vividly. What we may miss is the presence of New defects for which we have no established perceptual categories.
Ignoring such absurd claims, how should we adjust our mental categories for the New in sound reproduction? One answer is simply to live with it for a long time, the time being long not because the discriminations are so difficult but because the categories allowing them must first be established.
Some people may have no trouble making discriminations in any given case, of course, because they have different and less easily fooled mental categories. There were many people like me who mistook long-haired men for women, and there were many outraged comments about the long hair. (I imagine the outrage showed resentment at being fooled into reacting to a man as though he were a woman). But there were many other people who never made the error at all; apparently they were not using hair-length to identify gender. Just so, with any new audio technology, some listeners will not at first make the discrimination, while others will have no difficulty doing so.
Rather than simply listening to the New, therefore, one should compare its sound with the original. This is the appropriate test anyway, by the definition of the term ‘high fidelity.' Accurate reproduction is the accepted purpose of the equipment, and one tests accuracy by the most direct comparison.
Comparing reproduced sound with live music is educational but difficult. It is a prime activity in my course at California Institute of Technology, and students quickly learn how many unconscious conventions they have for listening; for example, the convention that loud reproduced sound is roughly as loud as live music -- with very rare exceptions, it's not.
Another reason that "live vs. reproduced" comparisons are difficult is that live sound is more convincing spatially than even the best two-speaker stereo. As an undergraduate, I heard a demonstration by Acoustic Research of a pair of AR-3a speakers on stage with the Fine Arts Quartet. At some point in the concert, the quartet switched from playing to miming, and the speakers took over. Placed among the quartet members and fed from an anechoic tape, they excited the acoustics of the hall very much as did the live instruments. With my eyes closed, I did not hear the changeover.
I don't think that AR-3a's were perfect speakers. I think that the ambience accuracy dominated the situation, leaving me, like Edison's audience, without convenient categories for telling live sound from reproduced. I think that on further listening, I could have learned to tell the AR speakers from the quartet.
The feed must of course be up to the job. Should one find that one or more tested units cannot reliably be told from the direct feed, then one is faced with an alternative: Either the units really are perfect or the direct feed is not good enough (or the listeners are not good enough). Once the direct feed is good enough that this does not happen, however, the test is reassuringly insensitive to just how good it is.
This suggests a ‘thought-experiment': What if we had a literally perfect recorder but we weren't aware that it was perfect; and what if, instead of listening to the direct feed and the digital recording, we had listened only to the playback of the two recorders. Those two listeners would have preferred the digital machine to the perfect machine, and the experiment would conceal the fact that they were preferring a degraded sound. This problem can be avoided only by using a labeled direct feed as a reference.
The other comment came independently from two listeners on the one day when tape hiss---exaggerated by frequency-response error in a line amplifier---gave a clear cue to which recorder was analog. The listeners did indeed favor the digital recorder strongly in this situation, but the interesting thing is that they did not choose it every time. Their comment was that both analog and digital degraded the direct feed, the analog by the hiss and the digital "in other ways."
Note the relative vagueness of the phrase "other ways" and Dr. Fuller's reference to "beauty" compared with the more well-defined remarks relating to categories taught us by the Old: "recording engineer's checklist" and "tape hiss." This seems indeed to reflect a lack of clear categories for the New.
Digital audio is not the subject here; it's just a convenient focus for discussion. Twenty years from now, we'll all agree about the quality of today's digital sound, as we now agree about the quality of transistor amplifiers of twenty years ago. (We do all agree, don't we?) The moral of my story is simply that in trying to reach a valid judgment, we must set up conditions and allow time not merely for new facts to be observed, but for the mental categories to come into existence through which those facts can be assimilated and organized. Then we must judge the New, not against the Old, but against the ultimate purpose of both. In audio, this means either a bypass test with a high-quality direct feed identified to the listener, or, if possible, a comparison with the live sound.
Many have a different attitude about all this. They regard audio equipment as a producer, not a reproducer of music. For them, there is nothing more to be said than "I like it" or "I don't like it." For me, however, audio can lead to careful observation of one's own perceptions and hence to deeper understanding of being human. And I find, as a performer, that my playing grows, both pianistically and interpretively, as the search for accurate recording and playback generates an ever-renewed appreciation of the beauty of musical sound, and of the crucial role played by the sound itself in the meaning of the music.