Comparing changed to our systems

[Flint]

I've written about it before, but I never really focused on it....

When we compare any two audio systems, or compare the difference in sound between two components in a system, it is absolutely critical to get the gain levels exactly the same. As I have written, I've participated in and edited research on how a mere 0.1dB change in SPL for the listener can be perceived as being an entirely different system. Between the change in loudness, you also have to take into account the "equal loudness curve" which dictates that any shift in overall loudness will also change your perceived frequency balance as well.

Take a look at the equal loudness curve:

If you are listening to your carefully chosen and endlessly studied auditioning material at an average of 70dB SPL and you replace amp A with amp B which has a difference in gain of 0.5dB, then the change in the loudness of everything below, say, 80Hz may sound like it is louder by 1.0dB SPL while the midrange is only louder by 0.5dB SPL. You will think the new amp has "more control over the bass" or "better impact", while in fact, all you did was increase the listening level by a scientifically measurable amount of 0.5dB SPL.

The same is true for comparing balanced versus unbalanced interfaces - you are not assured that the gain structure of the balanced interfaces is identical to the gain structure of the unbalanced interfaces.

What I've found when comparing anything in the electronics chain is that taking a few minutes to carefully calibrate the output with an AC voltage meter with a 1,000Hz sine wave ensures what you are comparing is apples to apples. It is bit more time consuming, but it makes a HUGE difference in how you will perceive the difference between any two parts of your system. With speakers is isn't as simple as that.

With speakers you have to estimate a similar SPL by measuring the frequency response. You can use a bandwidth limited Pink Noise signal which only includes about 600Hz to 3,000Hz and match the two speakers identically. Or, you can measure the frequency response, compare their SPL response on a response chart and try to adjust one of them until it appears to overlap as much as possible in the 200Hz to 7kHz range. But it isn't as easy because no two speakers has the same frequency response... and at different levels they will react differently in their output.



My advice to all of you is this....

If you want to know if your brand new amp, CD player, tube rolling, or whatever, is really different, get a good AC voltmeter and measure the output voltage at the speaker connector or headphone jack and ensure that both configurations are putting out the exact same voltage from the same source signal.

 

[PaulyT]

Yes, that makes sense, especially and most importantly if you're doing immediate A/B switching between two pieces of equipment. And I can see doing that with speaker amps, where there is often an independent gain setting on the amp, separate from the main volume control (which is generally an attenuation of the input signal, rather than a change in amp gain).

Unfortunately, headphone amps rarely (actually never in my experience) have gain controls, only a master volume. I suppose in theory you can approximate the same thing by adjusting the volume so that the output at the headphone jack has the same voltage... but it's not quite the same thing. The other practical issue is that, for me, my demo tracks are wildly different in their average loudness level, so I'm continually changing the volume during my evaluations, anyway. I wouldn't be able to set the volume by volt meter and leave it alone.

So I'll just have to live with un-scientific testing.

Actually my listening procedure is based on methods you described way back @ S&V - keeping track-by-track notes of what I'm hearing, and also my emotional response to the music. Yeah it's subjective, but ultimately this is all subjective at some point anyway, until audio engineers devise "perfect" tests.

 

[Flint]

Paul, you bring up an important issue.... how can you make an honest and fair comparison when the gear doesn't offer the features necessary to get it right?

Back in the day, I learned that isolating the change being compared and calibrating levels made it extremely obvious as to what the listeners heard. It took more time and work, but it was worth it in the long run. This hobby is filled with opinions based on the placebo of other people's opinions. We all know how powerful the placebo effect is when it comes to sound. As such, I am a proponent of audio skepticism and carefully designed listening tests over just trying it out to see if the self-proclaimed experts are right. I have had hundreds of people fall for the idea that something sounds a specific way merely because I said it would happen despite absolutely nothing being altered.

Words and ideas are powerful, and smart people will add to the illusion they were coached to believe by very slightly altering their descriptions of the sound Ina way tat aligns with the placebo language. For example, if we are told the bass will be more controlled, we may believe that claim but tell ourselves the bass is less dominant or tighter, which is a different way of describing the same concept, and thus we heard it but interpreted it in our own way.... all despite any real change in the sound.

This crap makes it impossible to know if we don't have the tool, features, or patience to ensure a fully calibrated comparison.


Of course, this is just a search for the truth. Regardless of what we really hear or what we believe we hear, as long as we are enjoying our hobby, all is good?

 

[PaulyT]

Paul, you bring up an important issue.... how can you make an honest and fair comparison when the gear doesn't offer the features necessary to get it right?

...

Of course, this is just a search for the truth. Regardless of what we really hear or what we believe we hear, as long as we are enjoying our hobby, all is good?

I wouldn't go that far. There is a middle ground. And just how objective you need to be depends a lot, in my opinion, on the purpose of the evaluation. If you're a manufacturer selling speakers or other gear, you need to be (ideally) as objective as possible in describing the benefits of your product, otherwise we just descend into snake oil territory. However, if you're an end consumer, what is the point in spending a lot of effort towards some elusive objectivity, when the end result really boils down to "do I prefer A or B?" I submit that it's irrelevant or even impossible to make a purely objective choice.

That said, the goal of my evaluations is to try to get some level of objectivity, or at least try to overcome the bias that you talked about because we've inevitably heard other people's opinions. Maybe it's all just a placebo and/or power of suggestion, maybe not, but if I can hear the difference between tube A and tube B, or SS smp C or amp D, and I happen to end up agreeing with opinions that I've read, does that make the opinion wrong or the evaluation invalid? I don't think so. I spend over an hour in each test, listening to my set of tracks, trying to arrive at some sort of conclusion, which usually presents itself after a few tracks and is confirmed by the rest (as I mentioned before). Is this objective or subjective? There's no way to tell for sure. And it's even worse when I try to describe in words on this forum what I'm hearing and concluding, because words are ultimately inadequate to describe sound. So yeah maybe it sounds to you and Jeff like I'm out in la-la-land; maybe I am, maybe I'm not, but there's no way to convince you one way or another - or myself for that matter - because the experience is at least in part subjective.

But I believe it's at least theoretically possible to overcome bias and be honest with one's self about what we're hearing. I think simply being aware of potential for bias and being intentional about carefully evaluating what we're hearing rather than what we've been told, will go a long way. Maybe it's impossible to eliminate bias completely, unless we're creating everything we use from scratch and never buying an externally manufactured product, or ever letting anyone else hear our gear. Most of us don't work that way, though, so we muddle through the best we can and try to avoid bias as much as we're able, while recognizing that the results may be imperfect or subjective.

People are not measurement devices. If we had some sort of readout that we could calibrate for ideal sound reproduction, then we wouldn't need all this discussion, and we could go about our happy objective lives. (Sorry, I'm teasing a bit here. :teasing-neener: )

So there's a place for objectivity, and a method to accomplish it (blind a/b, careful calibration, etc.) but most of the time such efforts are unnecessary or impossible. So then what do we do? Do we give up? No, we try to intelligently evaluate the gear as best we can, with the hopes that eventually we're happy with the result.

 

[PaulyT]

And then if you throw price into the mix, the subjectivity increases by another order of magnitude. That is, assuming I find more expensive device B to be superior to less expensive device A, is the improvement in going from A to B worth the difference in price? And if we're being honest with ourselves, is B perceived to be better than A because it's more expensive?

Anyway, my point in all this is that there are so many steps along the way where we can't be objective, that insisting on objectivity is not, in my opinion, useful. Rather, we accept that there is some level of subjectivity, and we should do our best to evaluate gear honestly and without bias as much as is humanly possible. That is the intention of my listening method. Not to convince you all that I'm right (not possible, and frankly I'm not quite egotistical enough to try - not usually anyway ), but to attempt to hear for myself whether a particular component is better than another, and, ultimately, whether I prefer to spend a lot of time listening through one component or another. And whether I can afford to do so...

 

[heeman]

I have recently had a couple examples of changes, I could call them upgrades to our HT System. One was the change from the Paradigm Studio 60/20's to the MB Quart Vera Bookshelves. The other the addition of the second XS30 sub.

Let me talk a little about the MB Quart - Paradigm change. I did document this pretty extensively in the other thread, however much time has passed since then. The latest is that I am still very pleased with the Vera's performance over the Paradigms, however there is some music material where I believe the Paradigms still sound better, however as a whole, the Vera's out perform them in Accuracy. Many of you have heard me say that I would NEVER get rid of the Paradigms, however the bargain to risk the huge discount on the Vera's pushed me over the edge and I am glad I did. I still have my 60's set up in the Den and I did sell the matching 20's.

The Sub upgrade was much more tedious, because I knew that the one sub was lacking due to placement and even when that was improved after John's GTG by using the simulator, it was still having some nulls. When adding the second, it really, REALLY became apparent to me that placement was everything especially with the second to eliminate the nulls and peaks that best that I could. I detailed this in the other thread also.

So, what am I trying to say here??????????????

The Vera to Paradigm is truly subjective and I have ZERO proof that they out perform the Paradigms, however my perception is that they are a better sounding speaker in my space.

The Sub upgrade is much more of an objective improvement. I could actually measure very easily and confirm the measurement by listening. I then could move both subs around, measure, listen and come to a very accurate conclusion, at the sweet spot.

Accurate measurement along with listening can help us draw conclusions, however the subjectivity to how and what we like to hear introduces the variables that make this hobby interesting.

:twocents-mytwocents: