AAE-9173 - Ran Out Of CPU Power

[guitardom]

Originally Posted by wonder6oy

So are you saying this is totally normal behavior for a healthy system?

Yes, it has ALWAYS been this way. Live inputs that require minimal latency require heavier CPU usage. How big that gap is has a lot to do with how well your system is setup and configured. inefficient drivers, background processes-programs, misconfigured Bios, etc. will all limit how well your system will perform at low buffers/live inputs.

[wonder6oy]

Originally Posted by guitardom

Yes, it has ALWAYS been this way. Live inputs that require minimal latency require heavier CPU usage. How big that gap is has a lot to do with how well your system is setup and configured. inefficient drivers, background processes-programs, misconfigured Bios, etc. will all limit how well your system will perform at low buffers/live inputs.

But these are not live inputs, and shouldn't be processed as such. In addition to that, I find it very difficult to believe that, even when done intentionally, any kind of horsepower boost required to handle live processing on Aux channels (or Master Faders or even live Inputs) would result in a CPU usage boost of nearly 5X the normal amount.

And it still falls short of explaining the erratic CPU spiking behavior, one example of which can be seen here...



Clearly the capabilities of this system are being extraordinarily underutilized here. I am at a complete loss how anybody could view this as "normal" behavior.

[treason]

I really appreciate this discussion and the information shared. I think it's vital for both users, as well as for Avid to have a sampling of how their products are working, or not working, as intended. Of course it's frustrating for those who are having issues, and frustrating for those who can't relate to those same users. Nevertheless, I encourage a continuation of this conversation. While Avid hasn't historically enlightened the DUC, this community has generally helped each other out. More than once I've come across posts that have improved the stability of Pro Tools and my sessions.

Thank you to all participating. This is invaluable!

[wonder6oy]

Originally Posted by treason

I really appreciate this discussion and the information shared. I think it's vital for both users, as well as for Avid to have a sampling of how their products are working, or not working, as intended. Of course it's frustrating for those who are having issues, and frustrating for those who can't relate to those same users. Nevertheless, I encourage a continuation of this conversation. While Avid hasn't historically enlightened the DUC, this community has generally helped each other out. More than once I've come across posts that have improved the stability of Pro Tools and my sessions.

Thank you to all participating. This is invaluable!

Thanks treason.
FWIW, if there is a system out there that will run this session correctly (without the spiking behavior), I would very much like to know it's hardware configuration. To date, all systems across both platforms have done this so far, which really makes the issue a matter of a choice of workflow - using Aux busses for heavy-lifting processing like parallel compression, saturation, emulation etc - and funneling them down to a print track, which is very common nowadays in modern mixes. I think this is the reason the 9173 problem has been so difficult to pin down. It's not the hardware; it's the choice in workflow that seems to determine how soon that point of instability is reached.

The test session is simply a way to magnify that bad behavior to bring it to the forefront and not be confused with other legitimate issues that might be at fault.

[guitardom]

The "spiking" will vary with the plugin selected and buffer size selected. Some plugins want more "time" with the cpu than other plugins. Those types of plugins will not work as well at lower buffer settings. Again, its always been this way. Any plugins known as "cpu heavy" are heavy BECAUSE they need more "time" with the CPU. When the HW Buffer is set low, the system (Pro Tools) is calling the cpu FAR more often in an attempt to reserve and take cycles that it can dedicate to latency. This is getting in the way of or "interrupting" other processes that are wanting more attention, like some plugins. Its why problematic drivers or drivers with heavy footprints like wireless drivers for instance will ALWAYS reduce system performance at these times and why bios settings that cause throttling and cpu fluctuations cause problems. Again, plugins and software determine the need, but overall system power and how well your system is optimized is the overall determining factor of the difference between high and low buffer settings with live inputs.

[wonder6oy]

Originally Posted by guitardom

The "spiking" will vary with the plugin selected and buffer size selected. Some plugins want more "time" with the cpu than other plugins. Those types of plugins will not work as well at lower buffer settings. Again, its always been this way. Any plugins known as "cpu heavy" are heavy BECAUSE they need more "time" with the CPU. When the HW Buffer is set low, the system (Pro Tools) is calling the cpu FAR more often in an attempt to reserve and take cycles that it can dedicate to latency. This is getting in the way of or "interrupting" other processes that are wanting more attention, like some plugins. Its why problematic drivers or drivers with heavy footprints like wireless drivers for instance will ALWAYS reduce system performance at these times and why bios settings that cause throttling and cpu fluctuations cause problems. Again, plugins and software determine the need, but overall system power and how well your system is optimized is the overall determining factor of the difference between high and low buffer settings with live inputs.

The origin topic of this thread was -9173 Errors, and while the original author's issue remains somewhat vague, the discussion of the reoccurring 9173 issue has continued forward with some interesting new discoveries that might help identify the root cause of the problem. Some of this has been covered in other threads, which have also been referenced, but I can understand where one might mistakenly draw the conclusion that simple matters like buffer sizes, CPU power expectations and hardware/software configuration issues might have been completely overlooked.

In case it might prove helpful to you, here's a summarized breakdown of where we're at so far...

Originally Posted by wonder6oy

Yesterday I sent another test session to a friend of mine who mixes on a Mac (I'm on PC). The session allows you to check the difference in the number of plugs you can run on individual audio tracks vs how many you can run on a series of busses. The difference on both our systems was both similar and staggering. On his (a 2015 iMac), he can run over 400 Pitch II instances on just the audio tracks and maintain total stability during playback, but when he moves the plugs from the audio tracks to the aux busses, that number drops to a remarkably low 80 instances. On my machine the results are similar: on audio tracks, I can get to 340 instances running smooth vs only 77 on the aux tracks. In both our cases, that's over a 400% increase in CPU usage on the Aux busses! He was as stunned about this as I initially was, and began to recognize in retrospect where he had faced work-halting issues in the past that might easily be explained by this phenomenon.

Is it a bug? Bad code? A feature "restriction" gone unmentioned by Avid? I don't know.

But I do know what it's NOT.

-It's NOT poorly coded 3rd party plugins (and yes - I even tried moving them out of the plugins folder)
-It's NOT a lack of power
-It's NOT a lack of memory
-It's NOT a hard drive issue
-It's NOT a neglect to follow optimization procedures from Avid or anyone else
-It's NOT a denormalization issue
-It's NOT a licensing check issue
-It's NOT a Dynamic Plug-in Processing issue; the problem persists with or without it
-It's NOT a Delay Compensation issue (I can get the same disastrous results with only 8 (yes 8) instances of Scheps Parallel Particles and a grand total of roughly 595 samples of PDC going on.
-It's NOT an unreasonable choice of workflow

And, given the reproducible symptoms across both Mac and PC platforms:

-It's UNLIKELY a driver issue
-It's UNLIKELY due to faulty hardware
-It's UNLIKELY the fault of the OS

So what is it?

Here's that session btw...
https://www.dropbox.com/s/9zrf9qm8py...test1.ptx?dl=0

And as mentioned in a recent response in a troubleshooting case w/ Avid...

"It's a test designed to highlight the problem in way that is impossible to ignore. The problem translates in a real working mix environment when a workflow involving some heavy processing on Aux tracks funneling down to a print track is used. This is a VERY COMMON style of workflow for MANY of us out there who do this for a living! Typically, however, it doesn't rear it's ugly head until a working session reaches around 65-70% in CPU usage. But when it does, it brings productivity to a GRINDING HALT with a CONSTANT slew of never-ending 9173 errors that can't be easily resolved because the only way to realy resolve them is to start disabling IMPORTANT AUX BUSSES."


If you would genuinely like to help out, I (and I am sure many others) would like to know the EXACT hardware configuration for any system able to run those plugins on series-connected Aux busses in a manner that far exceeds the dismal results I'm seeing currently. A viable test result to confirm this would prove informative as well.

Also, in a strange remark from Avid regarding the above case, they claimed that plugins on Aux busses should in fact be consuming FEWER CPU resources than ones on Audio tracks, contrary to what you've explained here. I'm not sure if he is mistaken about this or perhaps didn't understand the problem fully, but I'll update with any further responses as I get them.

[treason]

Originally Posted by wonder6oy

Thanks treason.
FWIW, if there is a system out there that will run this session correctly (without the spiking behavior), I would very much like to know it's hardware configuration.

I was able to enable all plugs natively without spiking. Here are the screen shots and the Mac Info (also in my sig).

[wonder6oy]

Originally Posted by treason

I was able to enable all plugs natively without spiking. Here are the screen shots and the Mac Info (also in my sig).

THANK YOU!

Question for you...
I notice you're using an HDX system. All of the systems I've seen test results for outside of yours are Native systems. I don't recall for sure, but I seem to remember that newer HD licenses allow for Native operation as well - like for use on portable laptops and such. If that's the case, do you know if you might be able to conduct the same test in Native mode? (Obviously I'm trying to isolate whether this might be an HD vs Native issue)
Or perhaps PCIe vs everything else?

[treason]

Originally Posted by wonder6oy

THANK YOU!

Question for you...
I notice you're using an HDX system. All of the systems I've seen test results for outside of yours are Native systems. I don't recall for sure, but I seem to remember that newer HD licenses allow for Native operation as well - like for use on portable laptops and such. If that's the case, do you know if you might be able to conduct the same test in Native mode? (Obviously I'm trying to isolate whether this might be an HD vs Native issue)

Correct - I have the possibility to run Native Only - however, I have no IO outside of HDX, therefore rendering me useless. <sigh>

This is a dedicated HDX system - thats all it does.

[wonder6oy]

Originally Posted by treason

Correct - I have the possibility to run Native Only - however, I have no IO outside of HDX, therefore rendering me useless. <sigh>

FAR FROM IT!

It can now be firmly established without question that this behavior is absolutely 100% NOT the intended behavior of the software, which for whatever reason has been difficult to prove up until now!