HEAT, NEVE 8048 Listening Challenge

[wizprodux]

I thought they were clear that this was a summing comparison and NOT a mix ITB vs OTB. Obviously with a Neve console you'd need to rely that the UAD NEVE modeling was on point.
In my experience mixing with the UAD NEVE plugs (1073, 1081) coupled with HEAT has made my clients extremely happy!

[bolau]

yeah, that's what I mean. In the old test they did the complete mix in vs. out the box. I like the new test better, but I would have preferred it the ABX way...

[Mt.Everest]

Originally Posted by kinghand

Actually, I first choose all the answer between Neve and Heat opposite, because I choose those I like best as HEAT, but I fouind they are wrong, and I made the opposite answer again, then most of them are all right. So, the best result to my ear is from Neve, but not HEAT, but obviously, HEAT is better than without HEAT. My monitor is Digidesign RM2.

Huh? You chose HEAT cuz you liked that best most consistently and since you expected the best to your ears to be NEVE you answered NEVE. So how dies this turn into you saying the best result to your ears is NEVE?
If you picked HEAT as the best sounding, and therefore chose NEVE cuz of your bias to expect NEVE to be better, all that means is that you preferred HEAT the most, not NEVE.

Am I missing something in what you said?

[WernerF]

Exactly, it seems as though: 1) people don't understand why a double blind us the most accurate test and, 2) don't understand that their own bias will effect the outcome if they are allowed to know which source is which. Anyone who doesn't think that their mind can talk their ears into hearing what they want is naive at the very least.

[SOUNDDECISION]

Originally Posted by WernerF

Exactly, it seems as though: 1) people don't understand why a double blind us the most accurate test and, 2) don't understand that their own bias will effect the outcome if they are allowed to know which source is which. Anyone who doesn't think that their mind can talk their ears into hearing what they want is naive at the very least.

1) First of all this isn't a "double blind test" a "double blind test" by definition is when the testers themselves don't know which is which either. This isnt the case here.

2) The main reason the test is flawed is because it REQUIRES us to be be biased in order to even make the selection. We need to pick the best sounding mix then assign it to what choice we expect to be the best sounding. So If I assume the NEVE to be the best I need to pick NEVE when I hear the best one. Obviously this doesnt work because if HEAT is actually the best...I will get most of the answers wrong.


The bigger point is the mistaken assumption that since so many people are getting so many wrong that they all must sound the same. This couldnt be further from the truth since I would assume that most of us agree they all sound very different.

[Eric Lambert]

Originally Posted by SOUNDDECISION

The bigger point is the mistaken assumption that since so many people are getting so many wrong that they all must sound the same. This couldnt be further from the truth since I would assume that most of us agree they all sound very different.

You've been right on the money with several posts in this thread. I'm impressed.

I'm interested in the differences between the Neve and HEAT - as everyone is - but the results are being falsely interpreted.

[kinghand]

Originally Posted by Mt.Everest

Huh? You chose HEAT cuz you liked that best most consistently and since you expected the best to your ears to be NEVE you answered NEVE. So how dies this turn into you saying the best result to your ears is NEVE?
If you picked HEAT as the best sounding, and therefore chose NEVE cuz of your bias to expect NEVE to be better, all that means is that you preferred HEAT the most, not NEVE.

Am I missing something in what you said?

It seems you understood me, did you ever do this test? If you did, you should know, you don't know which is which before you submit the answer, so I guessed the ones I like most is by HEAT, but I'm totally wrong, then I made a oppsite choice, guess the ones I like most is by Neve, and most of the answers are right, so I can tell, to my ear, the best result is from NEVE but not HEAT. Understand?

[Tom Graham]

Originally Posted by SOUNDDECISION

1) First of all this isn't a "double blind test" a "double blind test" by definition is when the testers themselves don't know which is which either. This isnt the case here.

2) The main reason the test is flawed is because it REQUIRES us to be be biased in order to even make the selection. We need to pick the best sounding mix then assign it to what choice we expect to be the best sounding. So If I assume the NEVE to be the best I need to pick NEVE when I hear the best one. Obviously this doesnt work because if HEAT is actually the best...I will get most of the answers wrong.


The bigger point is the mistaken assumption that since so many people are getting so many wrong that they all must sound the same. This couldnt be further from the truth since I would assume that most of us agree they all sound very different.

Just to wrap up my thoughts on this test and respond to you:

We are attempting to remove any bias or 'tipping of the scales' by not telling you what is the Neve, Pro Tools Only or PT HEAT mix. That way you are not biased. You are listening with fresh ears, and if they sound different to you, you can attribute which mix you think is which format. I personally feel they are pretty darned close.

IMHO, this test is not about picking which one is the best sounding mix. That question, about which you prefer (and the question about how hard was it) does not weigh in on the scoring results. You are only judged on your ability to identify correctly which mix is summed/done on which format. Most people have answered that the test was very hard to hear a difference, which is in contradiction to your comment that they sound very different.

I get that there are multiple ways to do this type of testing/experiment and we spent a lot of time, effort and dollars to make something we felt would be fair and eye opening at the same time. Sorry this doesn't work for you... Maybe we can do another version early next year. Thanks for checking it out though...

[Tom Graham]

Post Script :

780 People have taken the test as of this morning.

78% said they were not sure how they did, that it was pretty hard.

The average correct identification is 30%

[SOUNDDECISION]

Originally Posted by Tom Graham

We are attempting to remove any bias or 'tipping of the scales' by not telling you what is the Neve, Pro Tools Only or PT HEAT mix. That way you are not biased. You are listening with fresh ears, and if they sound different to you, you can attribute which mix you think is which format. I personally feel they are pretty darned close.

Sorry this doesn't work for you... Maybe we can do another version early next year. Thanks for checking it out though...

I think we all appreciate the concept of a test like this since we all would love to know how people perceive HEAT on a mix. The problem is the results of this test serve no purpose to actual engineers. Without any point of reference the results don't have any significance.

If it was important for the test to be done this way what would have made the results more relevant would have been to FIRST play 3 examples and tell us which is which, then ask us to identify the rest of them based on having a reference to what each option sounded like.

Or if the idea is to give no bias then just ask us to pick the best sounding etc.

But to have to try to attribute what is what based on what we might expect it to sound like would by design produce irrelevant results.

Also, the test doesn't even tell us how the Neve mix got back into protools. Different AD converters could also produce very different results.

At the very least we should have heard samples of each so we could have some idea of what we were trying to identify.

To sum it up (pardon the pun) The reason I took part in this and am taking the time to comment on this is I really would like some real world reactions on how others perceive HEAT vs other options but I don't think this test served that purpose.

I do appreciate the time and expense you put into this, but I thought it was important to provide this feedback.