April 2019

How To Do A Proper Listening Test
Part 2
Article By Ethan Winer

If you have not read Part 1, please read it first.

  In part 1 of this article, I outlined how to conduct useful listening tests. Now we'll look at testing specifics, beginning with software.

Testing Specifics — Software
The easiest type of listening test to set up is comparing two music files in a multi-track DAW program. You'll put the files on separate tracks, then link (assign to the same group) the track Solo (or Mute) buttons while they're in alternate states. Then clicking either button instantly toggles between Track A and Track B. These are just some of the types of things that can be compared using DAW software:

· Comparing recordings made through different converters and sound cards

· Assessing the degradation of one converter via multiple record generations using a loop-back connection (see the sidebar)

· Comparing lossy compression bit-rates, or any bit-rate versus an original Wave file

· Comparing EQ and other plug-in types when all parameters are set the same

· Comparing a hardware emulation plug-in to a recording of the same source sent through the original hardware

· Assessing whether sample rate conversion affects the sound audibly

· Comparing different dither algorithms, or even dither versus no dither

· Comparing the same Wave file played from different hard drives

· Comparing the recorded output of different preamps (described below)

In most of the above cases the files being compared will be the same volume. But if you're comparing different converters, or multiple loop-back generations, you'll probably need to adjust the level of one track up or down a little. All DAWs can vary the volume in 1 dB steps, but that's not good enough for matching levels. If your DAW doesn't have a fine enough resolution to match levels to 0.1 dB, the Sonalksis Free G plug-in (available at bottom of this article) lets you set the volume in increments as small as 0.01 dB. A link to this freeware plug-in is at the end of this article.

Comparing Hardware
Human hearing memory is poor, so it's important to switch between the two sources as quickly as possible. This is easy in a DAW program where you can instantly alternate playback from one track to another. As mentioned, if you link track Solo buttons while they're in alternate states, clicking either button toggles between Track A and Track B. But when comparing preamps or power amps you need to switch the output wires, and possibly the input wires too. That takes at least five to ten seconds or more, and even longer when switching stereo. Some people believe it's better to listen for a few minutes, then switch. I disagree because our limited hearing memory makes comparing more difficult. But if that's their belief, let them do as they please, even if it means they have less chance of being correct! Again, anyone who insists they can hear such an obvious difference should be able to tell A from B no matter how long they listen, or how long it takes to switch.

When comparing hardware you can either swap wires manually, or use a suitable switch. Passive line level switchers are not expensive, or you can make your own. You also have to match levels, which is more difficult if neither device has a volume control. Many power amplifiers don't include a continuous volume control, so you'll need a way to attenuate the level into whichever amp is louder. Figure 1 shows how to wire a potentiometer to create a simple passive (no added noise or distortion) volume control that can optionally be put into a small box with connectors. You could use a dual control to adjust both channels for stereo, but that's not recommended. First, it's easier and more direct to compare music coming from only one loudspeaker because that minimizes slight acoustic imaging changes as you move your head. Just as important, the left-right channel tracking accuracy of any dual volume control you can buy will surely be off by more than 0.1 dB at most volume levels.

Figure 1: When comparing audio devices that don't include a volume control, you can add one yourself to attenuate whichever device is louder. For stereo devices you could use a dual potentiometer with one section for each channel, but that's not recommended.

For line-level devices you can usually avoid switching input wires — simply split one signal to feed two devices using a Y adapter, for example to compare power amplifiers. But you still need to switch the loudspeaker outputs so only one amplifier is connected to the speaker at a time. A/B speaker switches are available commercially, or you can make your own for even less cost. The DPDT switch in Figure 2 accepts the Plus and Minus outputs from two separate power amps, sending one or the other to a single loudspeaker. DPDT stands for Double Pole (switches both the Plus and Minus wires) Double Throw (sends to either of two possible amplifier sources).

Figure 2: Any basic DPDT power switch rated at 5 Amperes or more will work safely with audio amplifiers up to 200 Watts at 8 Ohms. For higher powered amplifiers this formula calculates the switch's required current rating:

Aside from hardware that adds intentional color, you'll probably be listening to compare which device sounds clearer and with a correct tonal balance. So it's important to keep signal levels reasonable to be certain you're not clipping peaks even a little, because that of course affects audio quality. The best way to monitor levels for clipping is with an oscilloscope connected to the device output, but few people have that available. Unlike loudspeakers, most electronic gear is very clean right up to the point of hard clipping. So you could increase the input level until you just barely hear slight distortion, then back down by 6 dB to be sure you're within the device's linear region.

Wires
Wires are easy to compare. You simply plug and unplug line level RCA, XLR, and 1/4" phone wires to swap them. Speaker wires are also easy to compare, though it helps if the wires have banana plugs at both ends to allow switching more quickly than connecting stripped wire ends to binding posts.

Converters
Comparing sound cards and converters isn't difficult, and it doesn't matter if both converters are connected to the same computer or not. All that matters is recording the same source through both into suitable software. To compare the A/D sections you'll split the output of your source — whether a mic preamp or media player — using a Y adapter to send the same signal into both converters at once. It helps if at least one converter has an input volume control, otherwise you can insert the passive volume control in Figure 1, after the splitter, into whichever converter records hotter.

Even though you'll match the playback volumes when listening critically later, I suggest recording at similar levels too. Most modern converters are highly linear over their entire range, but a record level around –6 dBFS through both converters guarantees a clean capture. Of course, you could also record through both at –20 or even –30 to see how the converters compare when recording at lower levels. I suggest you record a 1 kHz sine wave from your smartphone or media player into both converters too, before the music starts. That will help you set the record levels now, and match the playback levels later. A 1 kHz Wave file that loops seamlessly is linked at the end of this article.

Comparing playback through the D/A part of a converter is even easier, though again it helps if one or both have output volume controls. As long as you added the 1 kHz tone mentioned earlier to the start of the music before making the audition recordings, that tone will stay attached to the music throughout the process. Then you can match levels using a voltmeter, or a recorder with an analog VU meter as described earlier.

Preamplifiers
Microphone preamps are surprisingly difficult to compare because you can't just use a Y splitter to send one microphone to both units. Well, you could do that, but the preamps might interact, and the increased load on the mic from driving two preamps in parallel is different than when only one is connected which could change the sound. Some preamps have variable input impedance, which further confounds matters. You could split the microphone's output using a special transformer having two secondary windings, but that too might change the sound more than the preamp differences. Therefore, a better method is a re-amp setup which lets you create identical sound sources repeatedly.

Re-amp is short for re-amplify, where a recorded source is amplified and played through a loudspeaker to be recorded a second time. Rather than recording live performances that will surely vary, you instead record one performance to create a new source, then play that through a high quality full-range loudspeaker. Unless you have a professional studio with serious isolation between rooms, you'll need to record each preamp's output anyway to compare later in a DAW program through loudspeakers as described earlier.

Figure 3: For this test setup I put a DPA 4090 microphone two feet in front of a JBL 4430 monitor. It's not clear from the camera angle, but the microphone is directly in front of the horn tweeter and pointed straight at it.

Figure 3 shows a re-amp setup in my home studio using a JBL 4430 monitor and DPA 4090 microphone two feet away pointed directly at the horn tweeter. A speaker playing music isn't the same as an original recording, but that doesn't matter. Whether re-amp'ing a loudspeaker captures exactly the same sound as a microphone in front of a singer or drum kit is irrelevant. The source simply is what it is, and a clean loudspeaker source is as valid as any other to audition tonal differences between preamps.

Before using a re-amp setup to compare preamps, you first need to set both preamps for the same amount of gain by playing a test signal through the loudspeaker. The noise level and sound of a preamp can vary slightly with different amounts of gain, so this ensures a fair comparison. Since the speaker and microphone won't move while matching levels, either pink noise or a 1 KHz sine wave can be used as the test signal. Start by placing the microphone a few feet in front of the speaker, at tweeter height, and close enough to avoid picking up too much room ambience. The best type of microphone is a small diaphragm omni condenser, because these tend to have the flattest and most extended response. Otherwise, just use the best mic you own or can borrow.

Next you'll connect the microphone to one preamp, and play the test signal through the loudspeaker at a reasonable volume such as 80 dB SPL. The gain amount needed depends on the mic's sensitivity, so you'll adjust the gain for a reasonable record level. Then connect the mic to the other preamp while the test signal continues to play through the speaker, and set that preamp to record onto a different track with the same amount of gain. As long as you record both preamps at the same reasonable level now, you can do the final level matching to within 0.1 dB when you play the recordings later during the actual blind test. Finally, play the test signal, then your music or other source, through the speaker while recording the microphone through one preamp. Then switch to the other preamp and record the same test signal and music source again. Now you have A and B tracks in your DAW that can be level-matched and played at random for the blind test.

Another option, besides re-amp'ing, is a player piano that's (hopefully) consistent every time it plays the same music. Yamaha makes a MIDI controlled acoustic piano, but it's very expensive so few people have access to one. To compare preamps for drums and other percussion instruments, you could drop a small ball bearing onto a drum or cymbal from a fixed platform above to create the same sound repeatedly. Then move the microphone output wire from one preamp to the other as you record each drop for later comparison.

Microphones
Microphones are very difficult to compare because they must be placed identically in the room and in relation to the sound source. Further, we usually compare microphones not to see if they sound the same—because most do not — but to decide which model sounds better subjectively with a given source. But blind tests are still useful, if only to discover that you like the sound of a particular modern $100 import you already own just as much as the $8,000 vintage model from your local rental shop. If the microphones are different sizes, or have different fixed pickup patterns, it's difficult to capture sounds exactly the same even if the mics are placed identically in the room. Microphone output levels also vary greatly, so that too needs to be accounted for. One solution is to record multiple takes in order to average the inevitable performance and positional variations.

First, connect one microphone and set a reasonable record level. With something as variable as a live performance it will be difficult to match volume levels later during playback. So just do the best you can. Now you can record someone singing or playing several times in a row onto different tracks. I suggest recording the same short vocal segment six times through Microphone A on Tracks 1-6, then again six times through Microphone B to Tracks 7-12. This way you need to position the microphone and set the record level only once for each mic. Of course, you could instead record an acoustic guitar, saxophone, or whatever source you think will best reveal the qualities of each mic. Be sure the performer stays the same distance from each microphone, and moves as little as possible from one take to the next.

Now there are 12 tracks in your DAW, clearly labeled mic A or B. Slide the recorded clips left and right if needed so they all start at the same time. Assuming you're the tester, play the clips by solo'ing tracks one at a time, ensuring that the subject can't see which track is playing. Again, it is best if the subject can't see your face either. And it's okay to play the same track two or three times in a row, with your subject choosing which microphone he thinks sounds better at least 10–20 times. After each playback, ask the subject which microphone he thinks he's hearing. Note each choice, and also note which microphone track was actually playing. After playing all 12 tracks a few times each, in random order, show your friend your notes so he can see how many times he was right.

Loudspeakers
It's even more difficult to compare speakers because their placement in the room has a huge effect on their sound, especially at low frequencies. And at mid and high frequencies, untamed reflections from nearby surfaces create comb filtering, which further skews the response. Earlier I mentioned several other reasons that speaker comparison is difficult, such as relatively high distortion that varies with volume level, and an off-axis response that changes constantly as you move your head even a little. Further, all rooms have numerous peaks and nulls that vary the response at low frequencies at different places in the room, even locations only inches apart.

Treating a room with plenty of bass traps reduces the low frequency response variation, but it's still present, and much larger than the response differences between most speaker models. Audio industry giant Harman International built a special listening room with a "loudspeaker shuffler" that uses a custom conveyor to put different sets of loudspeakers into the same physical locations in the room. A visually opaque but acoustically transparent curtain between the shuffler and the listeners hides which set of speakers is currently playing, making the comparisons truly blind.

You probably won't duplicate Harman's Herculean effort, so the next best thing is to audition speakers in mono. You'll put the two models being compared adjacent on a table that puts the tweeters at seated ear height. Books or similar will probably be needed under one or both speakers to ensure that both tweeters are at exactly ear height. Of course both speakers should be aimed at your head for the flattest on-axis response.

Then you'll match levels by placing an SPL meter (or small diaphragm omni-directional condenser mic) at tweeter height two to three feet from the speakers, centered as best you can to not favor either speaker. This is where the bl-noise.wav file mentioned earlier is needed. If you feed mono music into both channels of a stereo receiver, with one receiver channel driving each speaker, the receiver's balance control can set both speakers to read the same on the SPL meter. If you don't have a receiver with a balance control, you'll need some other way to ensure that both speakers are outputting the exact same volume. You could add a variable power resistor in series with the more sensitive speaker, but I don't advise that because it can affect the sound. If all else fails, the passive volume control in Figure 1 will do the trick, patched into whichever receiver or amplifier input is driving the louder speaker.

When you do the actual comparisons, I suggest sitting close enough to the speakers to minimize hearing room reflections, but not so close that the woofer and tweeter outputs are too close to "blend" properly to sound like a single source. It will be obvious from the source location which speaker is which, though you could have the subject enter the room literally blindfolded and be guided to the seat. It will still be obvious that one speaker is on the left and the other on the right, but at least the subject could choose a preference. One way around this, which I've done with some success, is to put one speaker on top of the other with the top speaker upside down. This puts the tweeters even closer together, which helps reduce their apparent physical separation.

Sidebar: Loop-Back Tests
Most modern digital converters — even affordable "prosumer" models — have very high quality. The distortion and noise of most is too soft to hear, with a frequency response so flat nobody can hear any quality loss no matter how carefully they listen. Devices this clean are considered to be audibly transparent. To make the degradation of high quality converters easier to hear in a listening test, it's common to record several "generation" passes. This way the distortion and other errors accumulate more than with a single recording.

The most common method plays music from a Wave file through a converter, with the converter's output connected back to its own input to be recorded again. This is called a loop-back test, because the output is "looped" back into the input as shown in Figure 4. If you do this several times in a row, each time playing the last recording as the new source, distortion and frequency response errors eventually accumulate enough to be audible. You might have to do this five times or even more to hear any degradation with a very high quality converter! This is where adding a 1 kHz tone to the front of the music really helps, because you'll be setting record levels five or ten times or more.

Figure 4: A loop-back test patches the output of a sound card or external converter back into its own input, to assess the degradation from multiple recording passes.

These are the band-limited pink noise and 1 kHz sine wave files on the author's web site:

EthanWiner.com/bl-noise.wav

EthanWiner.com/1k-sine.wav

Below link is the Sonalksis Free G freeware volume control plug-in:

www.Sonalksis.com/freeg.html

About The Author
Ethan Winer has been an audio engineer and professional musician for more than 45 years. His Cello Rondo music video has received nearly two million views on YouTube and other web sites, and his book The Audio Expert published by Focal Press, now in its second edition, is available at Amazon.com and Ethan's own web site EthanWiner.com. Ethan is also a principal at RealTraps, a USA manufacturer of high quality acoustic treatment.