The Digital Challenge: A Report
Article By Stanley P. Lipshitz
University of Waterloo
Waterloo, Ontario Canada
This article is from the August - September 1984 issue of BAS Speaker

  Some readers may be unaware of the background to the "digital test" in which Ivor Tiefenbrun participated on February 23, 1984 [1-4]. To summarize briefly: Tiefenbrun has been quite outspoken about the inadequacies of digital audio recordings and the systems on which they are made, and his organization, Linn Products Ltd., was instrumental in publishing an analysis of the Sony consumer PCM-F1 digital audio adapter [5] outlining their objections. I, on the other hand, have been using this very system for a number of years now and have made close to one hundred recordings with it with superb results and not the "execrable results" reported by Tiefenbrun. I and my colleague John Vanderkooy have moreover conducted blind listening comparisons between the PCM-F1's input and its reconstructed output signals, and had yet to find anyone who can reliably distinguish between them on musical program material.

I therefore challenged the "anti-digital" community in general, and Tiefenbrun in particular, to participate in a blind listening test of the PCM-F1 to give them the opportunity to substantiate their claims of poor sound. The challenge details were spelled out in [2]. When I learned in February 1984 of Tiefenbrun's impending visit to Toronto, I reissued this challenge and was pleased to have him accept.

The test took place in the home of the local Linn distributor, Mr. Michael Remington, using his all-Linn/Naim system (Linn LP-12 turntable, arm and cartridge, Naim NAC 32 pre- and NAP 250 power amplifiers and Linn Isobaric loudspeakers) and his choice of program material (all LP records). Vanderkooy and Alan Lofft, editor of Sound Canada magazine, were also present. The atmosphere throughout was cordial and more relaxed than I expected.

The day began with two brief tests of the Tiefenbrun claim that undriven transducers (digital alarm watches, telephones, headphones, or other loudspeakers) in the same room audibly degrade the sound quality - a claim which forms the rationale behind their "single speaker" demonstration demand. Firstly, a digital alarm watch with piezoelectric "beeper" was held about 500 mm behind Tiefenbrun's head while he listened to the loudspeaker reproduction from his stereo seat on the couch, with the watch either fully exposed or clasped firmly between the palms of my hands. We were assured that the latter artifice would muffle any deleterious effects. This was thus a single-blind test: The testee did not know the covered/uncovered status of the watch at each trial, but the tester did know. A random series of 20 trials was conducted while Remington cued up the turntable (playing a female vocalist) on each occasion, as he did throughout the day. Tiefenbrun's result: 10 correct responses in 20 trials, an outcome which shows no ability to discriminate between the two situations.

The second test, also single-blind, used a Linn "Kan" loudspeaker as the undriven transducer. Again the female vocalist was used as source material. The loudspeaker lay on the thickly-carpeted floor behind the listening couch. It was placed either on its side (the "uncovered" condition) or on its face (the "covered" condition) according to a random series of choices. Ten trials were conducted during which Tiefenbrun achieved a score of 5 correct out of 10. Again, this demonstrates no discrimination ability beyond what one would expect purely on the basis of chance.

With these preliminaries out of the way (but just for safety all watches and headphones were left in another room throughout the entire day's proceedings), we proceeded to the main test of the day - that of the audibility of the Sony PCM-F1 digital audio processor in 16-bit mode when inserted into the audio chain, as proposed in my original challenge.

The PCM-F1's analog-to-digital (D/A) converter output was looped straight through to the digital-to-analog (D/A) converter input by connecting "video out" to video in". The complete encode-decode chain including the two low-pass anti-aliasing filters as well as the sample-and-hold circuits and the A/D and D/A converters was thus subject to the test. In order to remove the F1's polarity reversal it had been internally fitted with a digital inverter chip (as used in the later non-inverting Sony PCM-701 model) to interchange the ones and zeroes and hence perform a polarity correction.

The PCM-F1 was inserted into the signal path between the preamplifier and the power amplifier by means of an A/B/X switchbox of our design (see [6, 7]). The setup is shown in Fig. 1, above. When the switchbox was in the "A" position the PCM-F1 was inserted into the signal path, while the "B" position represented the "straight-through" configuration. The "X" position was a randomly selected choice made by the switchbox. For each trial "X" was chosen by the box to be either "A" or "B", but its identity was unknown to any of the participants, thus making the experiment double blind. The subject could at his leisure compare any of all of "A", "B" and "X" with each other. What he had to do was identify whether "X" was "A" or "B". Once his decision was made, the switchbox was interrogated to discover the true identity of "X", and then "X" was re-randomized in preparation for the next trial.

The gains of the "A" and "B" paths were matched in both left and right channels to within 0.05 dB at 1 kHz using the PCM-F1's gain controls. This was done by measuring across the amplifier output terminals. The match was then confirmed to be within ± 0.25 dB across the whole audio band. The PCM-F1's "peak hold" feature was used to keep a record of the peak signal levels passing through it during the test, especially in view of the relatively high sensitivity of the Naim power amplifier (<1 Vrms at clipping) and the relatively low listening levels chosen by the participants. More about this shortly.

After an acclimatization period, a set of 10 trials was conducted in an unhurried fashion before breaking for lunch, after which a further set of 10 trials was conducted. Tiefenbrun's score for the series was 11 correct decisions out of 20, a result which shows no statistically significant ability to discriminate between "A" and "B" any more accurately than would be expected on the basis of random guessing.

At this point I thought that I could reliably distinguish between the "A" and "B" paths on the basis of the slight noise level increase which occurred when the PCM-F1 was inserted into the chain, and which was marginally audible due to the high gain of the Naim MAP 250 power amplifier combined with the low peak signal levels through the F1, which the peak-hold meters showed to have risen no higher than -20 dB. (0 dB is the digital clip point, and these peak levels were somewhat unfair to the digital processor since 20 dB of its signal-to-noise ratio was being thrown away.) [In other words, for this segment of the test the F1 was in effect a 13-bit processor. - Ed.]

I expressed my desire to try the test, and Remington went to cue up the record again, but I requested to be allowed to undertake the test with no signal passing though the system. Before realizing the import of what he was saying, Vanderkooy interjected: "Ah! You're going to listen to the sound of the relays." Yes, there is indeed a slight audible difference between the acoustic "click" made when the "A" and "B" relays pull in. This is due to the unavoidable differences in the mounting positions of the relay on the A/B/X box chassis and, although slight, it can be heard if one listens for it. I replied that I was going to listen to the difference in background hiss, and the subsequent series of blind trials showed conclusively that the two signal paths could be reliably distinguished on this basis alone.

What conclusion can we draw from this? Tiefenbrun's random results show that he had not been aware of either the background noise difference - masked as it most likely was during his trials by the record surface noise - or the relay click difference. The null result of this first PCM-F1 listening test is thus valid in spite of the potential for non-blindness due to the subliminal noise differences. I had been aware of the hiss difference from the outset from listening while the stylus was off the record, but had decided to proceed with the test in the knowledge that if the test produced a null result, the hiss difference could not have affected the outcome - a decision vindicated by the actual data.

The "secret" about the slight relay noise difference was, however, now out in the open, and if listened for could from then on influence (albeit subconsciously) the blindness of any succeeding tests made using the switchbox. Nevertheless we decided to proceed with a second series of PCM-F1 listening tests, this time with the digital system inserted into the preamplifier's tape monitor loop, as shown in Fig. 2, above. Once again levels were carefully equalized and frequency responses checked to be very closely matched between "A" and "B". This insertion point precedes the preamplifier's volume control, where the signal levels are significantly higher, thus more fully exercising the F1's full dynamic range while at the same time rendering its output noise level inaudible at the replay volume setting then being used. This configuration removed the hiss difference as a potential factor.

A set of 23 double-blind trials was conducted listening to music through the system, with Tiefenbrun voting for the identity of "X" at each trial. It was then that Vanderkooy pointed out that, although the relays in the A/B/X box were switching normally during this series, no tiny electrical clicks were audible from the loudspeakers during their operation, as normally would be the case. Investigation revealed that the preamplifier's "source/tape" monitor switch had inadvertently been left in the "source" position, and as Fig. 2 reveals, the A/B/X box and digital system were thus not being inserted into the chain at all during this run!

Because the "A" and "B" positions were completely identical due to this error, an unbiased decision for "X" should have produced a 50:50 split of "A" and "B" votes. Interestingly, it turned out that this was not the case, Tiefenbrun's votes being 14 for "A" and 9 for "B". Could something have been influencing his voting, or did he just prefer the letter "A"? A comparison of the actual "X" choices showed that during this series Tiefenbrun voted incorrectly in 16 out of 23 trials, although the sound from the loudspeakers never changed. [This proportion of incorrect guesses would occur in a truly random situation less than 5% of the time. - Ed.] What does this mean? We cannot say for certain, but the most logical explanation is that he was (perhaps subconsciously) voting on the basis of the relays' acoustic click difference which we had now revealed. If so, his remembrance of the sound of the A and B clicks was inverted. Be this as it may, the error we had made by leaving the tape monitor switch in the "source' position turned out to provide an interesting sidelight on the question of personal bias. It also illustrates one of the potential pitfalls of high-resolution blind testing, and the danger of jumping to conclusions before very carefully checking the test setup.

With the tape monitor switch now correctly set, a rather rapid series of 37 trials took place with Tiefenbrun voting. The results: 10 correct decisions out of 37, a result far worse than would have been expected by chance alone. What was influencing the voting? Was it the relay sound? We do not know, but this seems likely. It should be remembered that using the A/B/X box, a direct comparison between "A" or "B" or "X" is always available, so a series of trials in which there is a tendency as above to produce a consistent reverse identification seems to implicate some extraneous factor unrelated to, but correlated with, the loudspeaker sound. Tiefenbrun did very little switching during this series, leaving the A/B/X box set to the "X" position most of the time. It does indeed seem possible that the identifications were being made on the basis of the relay sound incorrectly remembered.

To wrap up the day's work, we conducted a final series of tests to assess whether the relay contacts in the A/B/X box could have been electrically affecting the sound in any audible way. For this single-blind test the configuration was as in Fig. 3, above. The A/B/X box was powered up and left permanently in one position, and the relay contacts inserted into the signal path by switching the preamplifier's monitor switch to the "tape" position. The switching sequence (relay contacts "in" or "out") was determined by a random number sequence. Thirty trials were conducted, interrupted three times to provide Tiefenbrun with a reference comparison of the "direct" sound versus the relay contacts. His score in identifying the presence of the relay contacts was 12 correct out of 30 trials. This of course also shows no statistically significant ability to identify correctly when the relay contacts were in circuit.

In summary, then, no evidence was provided by Tiefenbrun during this series of tests that indicates ability to identify reliably:

(a)  the presence of an undriven transducer in the room,
(b)  the presence of the Sony PCM-F1 digital processor in the audio chain, or
(c)  the presence of the relay contacts of the A/B/X switchbox in the circuit.

The tests were conducted in an amicable rather than confrontational atmosphere, and the parties departed feeling that the day's work had been worthwhile. Further carefully-conducted blind tests will be necessary if these conclusions are felt to be in error.

I would like to acknowledge the friendly cooperation and assistance of both Michael Remington and John Vanderkooy in carrying out these experiments, and would like to take this opportunity to express to Ivor Tiefenbrun, the guinea pig of the day's experiments, our genuine admiration for having the sincerity and guts to put his professed beliefs on the line.

References:

[1] S. P. Lipshitz, "Views", HFN/RR, Aug. 1984, p.15

[2] S. P. Lipshitz, "Views", HFN/RR, Sept. 1983, p.19

[3] I. S. Tiefenbrun, "Views", HFN/RR, Jan. 1984, pp. 19, 21.

[4] I. S. Tiefenbrun, "Views", HFN/RR, June 1984, pp. 13, 15.

[5] A. Orlowski, "Digital Sound: The View from Scotland", HFN/RR, June 1983, pp. 34-37.

[6] S. P. Lipshitz and J. Vanderkooy, "The Great Debate: Subjective Evaluation", J. Audio Eng. Soc., Vol. 29, pp. 482-491 (1981 July/Aug.).

[7] D. Clark, "High-Resolution Subjective Testing Using a Double-Blind Comparator", J. Audio Eng. Soc., Vol. 30, pp. 330-338 (1982 May).

Enjoy the Music.com highly encourages our readers to join the Boston Audio Society by clicking here).

This article is copyrighted © by the author or the Boston Audio Society.
It is posted on Enjoy the Music.com with their permission,
and with all rights reserved.