AAE-9173 - Ran Out Of CPU Power

[dankin]

Originally Posted by treason

I was able to enable all plugs natively without spiking. Here are the screen shots and the Mac Info (also in my sig).

So do you ever get spikes? Are the 9173s limited to native PT? Mine come and go but, I get them everyday, multiple times and have since 11 came out. Both my 2010 12 core 3.46, and my 2014 2.8 MBP. Large sessions (100+ tracks) on my MBP usually end up around 40-45% of CPU, but at any random moment that will spike to 100% and stop playback. Same thing on my 12core, except the same session on that machine will average around 25-30% of CPU, but still randomly jump to 100%.

[jjnssn]

Help me understand what you believe this “Native Mode is”.

Do you mean plugins running on a CPU vs DSP chip?

Are you thinking HDX vs Native Cards?

Are talking HD va Vanilla software license?

Or some combination of the above.

For example; my understanding is you need an HD license to run HD Native or HDX cards, I have had both. Thus if I run ProTools on my MacBook Pro It launches as HD however I am NOT using the HDX card.

I am going to run your tests. I am curious regardless of system conf. You’ve obviously put a great deal of effort into this. I would like to help so understanding your hypothesis is my goal , (what do you believe to be true).


Sent from my iPhone using Tapatalk Pro

[treason]

Originally Posted by dankin

So do you ever get spikes? Are the 9173s limited to native PT? Mine come and go but, I get them everyday, multiple times and have since 11 came out. Both my 2010 12 core 3.46, and my 2014 2.8 MBP. Large sessions (100+ tracks) on my MBP usually end up around 40-45% of CPU, but at any random moment that will spike to 100% and stop playback. Same thing on my 12core, except the same session on that machine will average around 25-30% of CPU, but still randomly jump to 100%.

My sessions are in the range of 190 to 300+ voices. My spikes appear to be correlated to the Native Plugs I use - in other words, the more AAX-DSP I utilize, the more stable ( i.e. no -917x spikes) the sessions.

BTW - I typically hit 60-85% CPU load on the Native side, utilize in excess of 90% of 2 UAD Octos and max out my HDX2 cards. I have extensive session routing as I mix each instrument (about 16-22) w/it's own VCA, Master Strip and subgroup AUX, and a normal session uses about 8-10 HDX DSPs for the "Mixer" function. The very seed of this thread is how I work - and it works well.

I've been iterating on sessions, and recently started removing instances of Kush Omega, UAD console strips and Slate VCC (using Plugin Alliance's new bx_consoles instead). These changes have made my sessions Native CPU usage almost a flat line of stability.

Native CPU stability has been a quest of mine since switching to PT HD11 - and my solution has been to instantiate as much AAX-DSP as I can. With the choice of great plugs in AAX-DSP these days, it's been getting easier.

Hope that helps,

[wonder6oy]

To the above responders,
Thank you for your participation!

As I have been discussing this subject on several threads now, I'm realizing that it may be unclear what I am hoping to establish from the results (my fault guys- this has been quite a process for me). I recently sent some step-by-step directions to Avid so I'll go ahead and present those here with a few updates:

The purpose of the test is to identify two things:
a. The characteristics of a vast, undesirable difference in some system's CPU capability resulting from instability caused by native plugins enabled on tracks being fed audio from other tracks (live monitor input audio tracks, Aux busses, Master busses) vs audio tracks either empty or loaded with audio clips and not set to a live monitor input.
b. Whether the system being tested has this issue or not.

Directions:
1 - Load the test session located here:
https://www.dropbox.com/s/9zrf9qm8py...test1.ptx?dl=0
2 - Without replacing the already-enabled Tone Generators, enable as many PitchII plugins on the Audio tracks only as you can (OR REPLACE WITH ANY OTHER PLUGINS YOU LIKE) while maintaining the ability to engage stable, uninterrupted playback. If you run out of plugins available, please duplicate tracks to add more instances the maximum number your system is capable of is reached.
3 - It's likely your system should run out of power at just above 95% CPU usage. For reference, the number of plugins some have been getting at this point seems to range from 350-475 instances. More powerful systems will produce higher numbers. Please make a note of the number of plugin instances you can run at this point.
4 - Now DISABLE all of those enabled plugins with the exception of the tone generators - please leave them enabled.
5 - Now enable as many PitchII plugins on the Aux tracks only as you can (OR REPLACE WITH ANY OTHER PLUGINS YOU LIKE) while maintaining the ability to engage stable, uninterrupted playback. If you run out of plugins available, please create more Aux tracks continuing the series workflow (Aux feeding Aux) until the maximum number your system is capable of is reached. Now please make a second note of the number of plugin instances you can run as well as the average CPU usage at this point.

Results are still being reviewed, but so far systems that have this issue exhibit a discrepancy between the two numbers by roughly 400-500% (i.e. 340 vs 77) with CPU usage maxing out on the Audio tracks portion of the test, but reaching only roughly 35% on the Aux tracks portion of the test. Systems that do not have this issue should exhibit similar numbers and a maxed out CPU on both tests.

My (admittedly optimistic) hope regarding all this are that we are able to in some way help Avid engineers diagnose the root cause of the years-long enduring issue of incessant -9173 errors on what should otherwise be healthy systems and identify the differences between those systems that experience it and those that do not.

[Darryl Ramm]

What IO Buffer size are you working at? This is the primary impact on performance here as folks have been pointing out. You are mentioning everything under the sun except that most critical thing.

What the CPU meters display is not a highly accurate magical view of what might be going on. Do not spend a lot of time worrying about what they show. All that matters is avoiding CPU errors, and the best that you as a user can do is properly optimize the system, run with an IO buffer large enough to be stable (and live with whatever latency that causes) and/or change workflow especially freezing and committing tracks. In some cases a better interface/driver may help, or you need to buy a bigger computer. This stuff is dealt with by folks all the time, Avid should already be well aware of stability issues in AAX, I suspect they may not have the deep technical/architect folks left who can address this. I suspect your efforts are best spent in working on what you can do to get things running as best as you can for your needs.


Sent from my iPhone using Tapatalk

[wonder6oy]

Originally Posted by Darryl Ramm

What IO Buffer size are you working at? This is the primary impact on performance here as folks have been pointing out. You are mentioning everything under the sun except that most critical thing.

What the CPU meters display is not a highly accurate magical view of what might be going on. Do not spend a lot of time worrying about what they show. All that matters is avoiding CPU errors, and the best that you as a user can do is properly optimize the system, run with an IO buffer large enough to be stable (and live with whatever latency that causes) and/or change workflow especially freezing and committing tracks. In some cases a better interface/driver may help, or you need to buy a bigger computer. This stuff is dealt with by folks all the time, Avid should already be well aware of stability issues in AAX, I suspect they may not have the deep technical/architect folks left who can address this. I suspect your efforts are best spent in working on what you can do to get things running as best as you can for your needs.


Sent from my iPhone using Tapatalk

Hello Darryl. Buffer size makes no difference with this issue unless set to extreme low settings, which of course magnifies the problem as it taxes the CPU further. I will be sure to add this to the list in post#37 in the next revision.

[wonder6oy]

Originally Posted by jjnssn

Help me understand what you believe this “Native Mode is”.

Do you mean plugins running on a CPU vs DSP chip?

Are you thinking HDX vs Native Cards?

Are talking HD va Vanilla software license?

Or some combination of the above.

For example; my understanding is you need an HD license to run HD Native or HDX cards, I have had both. Thus if I run ProTools on my MacBook Pro It launches as HD however I am NOT using the HDX card.

I am going to run your tests. I am curious regardless of system conf. You’ve obviously put a great deal of effort into this. I would like to help so understanding your hypothesis is my goal , (what do you believe to be true).


Sent from my iPhone using Tapatalk Pro

Sorry jjnssn - It is indeed a bit difficult to define isn't it?
Honestly, I would welcome feedback from anyone who is willing to give it a try. HD Core, HD Native, or vanilla Native. One intent of this is to find out if any particular software or hardware configuration is directly tied to the matter.

If you have the capability to run both (like HD on a Core card vs HD run in Native mode) that might be very helpful!

[Darryl Ramm]

Please let us know what range of io buffer sizes you are checking over. Also as mentioned before make sure dynamic plugin processing is disabled and reduce additional latency is not checked. I don’t have Pro Tools with me (traveling without iLok, duh) but I hope to check this out later.

And the big difference with HDX is the mixer plugin itself is running on the DSP not native. Core stability issues that have been seen in AAX make me suspicious that even the mixer is unstable, I’ve wondered before if that might include the mixer itself sufferings from denormalization issues.


Sent from my iPhone using Tapatalk

[wonder6oy]

Originally Posted by treason

My sessions are in the range of 190 to 300+ voices. My spikes appear to be correlated to the Native Plugs I use - in other words, the more AAX-DSP I utilize, the more stable ( i.e. no -917x spikes) the sessions.

BTW - I typically hit 60-85% CPU load on the Native side, utilize in excess of 90% of 2 UAD Octos and max out my HDX2 cards. I have extensive session routing as I mix each instrument (about 16-22) w/it's own VCA, Master Strip and subgroup AUX, and a normal session uses about 8-10 HDX DSPs for the "Mixer" function. The very seed of this thread is how I work - and it works well.

I've been iterating on sessions, and recently started removing instances of Kush Omega, UAD console strips and Slate VCC (using Plugin Alliance's new bx_consoles instead). These changes have made my sessions Native CPU usage almost a flat line of stability.

Native CPU stability has been a quest of mine since switching to PT HD11 - and my solution has been to instantiate as much AAX-DSP as I can. With the choice of great plugs in AAX-DSP these days, it's been getting easier.

Hope that helps,

treason, I appreciate your input as it is helping me identify some factors that might need to be nailed down to provide valuable feedback from others. Your system is using a core card - is that correct? My last experience with non-native HD hardware was with an HD Accel system and in addition to plugins, it handled the processing of voices and bussing directly on the card itself. Is that the case with these newer systems as well?

EDIT: Darryl appears to have answered that in the post above. Thx Darryl.

[wonder6oy]

Originally Posted by Darryl Ramm

Please let us know what range of io buffer sizes you are checking over. Also as mentioned before make sure dynamic plugin processing is disabled and reduce additional latency is not checked. I don’t have Pro Tools with me (traveling without iLok, duh) but I hope to check this out later.

And the big difference with HDX is the mixer plugin itself is running on the DSP not native. Core stability issues that have been seen in AAX make me suspicious that even the mixer is unstable, I’ve wondered before if that might include the mixer itself sufferings from denormalization issues.


Sent from my iPhone using Tapatalk

Buffer sizes have been checked from 64 - 1024 samples in size in my case. As stated in the list, dynamic plugin processing engaged or disengaged makes no discernible difference. Signal generators are included in the session to eliminate the possibility of denormalization issues.