AAE-9173 - Ran Out Of CPU Power

[YYR123]

S1 is pretty good about latency when it’s paired with Presonus hardware iirc.

[Darryl Ramm]

Originally Posted by wonder6oy

Indeed, it is the combination of the length of the Aux bus chain and whether or not any heavy processing is taking place on that chain. And it is definitely an issue with Native systems only (HD or Vanilla).

After weeks of waiting, I got a generic "low latency processing increases CPU usage" response from Avid. Case closed.

As I don't have the time to deal with this any longer and don't get any indication that the Avid engineering team is going to regard this with any amount of seriousness, I began the transitional process to another DAW a couple of weeks ago. It's a painful one, but the results have been undeniably positive so far.

20 years invested in Pro Tools.

This does not surprise me. I think you had posted before about a Avid customer support person working with you and I thought they were being a little careless in setting your expectations that this would go somewhere.

Do you have a real-world session (not using Pitch II plugins, or all plugins on very long bus chains) that shows a problem that you can share?

I know folks are hitting -9173 errors, and I'm curious why, I've found the Pro Tools 12 releases I've used to be relatively stable.

[wonder6oy]

Originally Posted by Darryl Ramm

This does not surprise me. I think you had posted before about a Avid customer support person working with you and I thought they were being a little careless in setting your expectations that this would go somewhere.

Do you have a real-world session (not using Pitch II plugins, or all plugins on very long bus chains) that shows a problem that you can share?

I know folks are hitting -9173 errors, and I'm curious why, I've found the Pro Tools 12 releases I've used to be relatively stable.

A "real world" session obviously has far fewer cascading busses, but in my case - and in several others I've seen here - a typical session would hit a brick wall of un-playability at @ 65% cpu usage. And due to the nature of the bug, freezing a few tracks does little to help; the busses remain and continue to further instability even when frozen. Without the use of cascading busses and substituting the use of more parallel busses instead, I am able to utilize close to the full potential of the system. If you are using a hardware-based version of Pro Tools, you will not experience this issue. If you are using a Native version, than it's likely your workflow does not require an extensive use of cascading busses.

[Darryl Ramm]

I was just curious what plugins or workloads are causing these problems--on my 4 core MBP I'm able to run quite large sessions, including your test session if I don't go crazy chaining super long runs of Aux tracks. I think your original idea to provide a sessions that can reproduce this problem was a good one but I doubt the session as is is going to convince anybody there is something that needs to be fixed--maybe it performs a lot worse on other computers (I can't tell).

I was surprised that increasing playback buffer size does not make as a huge a difference to stability as I thought it would when in that mode of killing things with a long chains of Aux tracks.

The ~65% CPU comment worries me. The CPU meter is fantasy ********, especially with hyperthreading enabled. It concerns me you are mentioning it like getting errors at 65% is a problem. At least disable hypeerthreading and see what the meter shows.

[wonder6oy]

Originally Posted by Darryl Ramm

I was just curious what plugins or workloads are causing these problems--on my 4 core MBP I'm able to run quite large sessions, including your test session if I don't go crazy chaining super long runs of Aux tracks. I think your original idea to provide a sessions that can reproduce this problem was a good one but I doubt the session as is is going to convince anybody there is something that needs to be fixed--maybe it performs a lot worse on other computers (I can't tell).

I was surprised that increasing playback buffer size does not make as a huge a difference to stability as I thought it would when in that mode of killing things with a long chains of Aux tracks.

The ~65% CPU comment worries me. The CPU meter is fantasy ********, especially with hyperthreading enabled. It concerns me you are mentioning it like getting errors at 65% is a problem. At least disable hypeerthreading and see what the meter shows.

Plugins don't cause the issue. And a typical session of my own might have a chain in it somewhere consisting of 6-7 links total, factoring in assorted processes of parallel compression, grouping, bus processing, and mastering - far from "going crazy". The problem persists regardless of platform, processor power, or winning personalities. Changing buffer size to relieve stress does not help because the CPU is not undergoing any stress. The problem is clearly in the code, and there is absolutely nothing anyone outside of the team of Avid engineers who can do anything about it. The fact that I am able to provide a session that highlights the issue by shutting down Pro Tools at a mere 25% CPU usage and still struggle to convince those who have not experienced the problem firsthand that there even IS an issue is indicative to me of the outcome I believe my efforts at this point will yield.

[Darryl Ramm]

I think you seem fixated on the CPU meter meaning more than I think it does. The CPU errors are telling you when things die because of too much latency.

The CPU meters are reporting something very different, and typical statistical sampling used to generate those numbers and the compete abberation of hyperthreading likely means they are not reporting interesting stuff much of the time. I know that Bharath Venkatesan who was one of the Avid engineers who used to pop up in DUC and provide helpful information was concerned about folks overrelying on the CPU meters or misunderstanding what they mean. He had promised at one point to write an Avid blog post about that, but left Avid for before he got to do that.... if anything in all this that would be something very useful for Avid folks to provide more info on that.

[Burbree]

I'd argue Darryl that people are instead fixated on achieving reliable consistent performance in Pro Tools. At present that doesn't seem achievable for many. The CPU meter in Pro Tools mirrors quite closely Apples own activity monitor CPU usage with hyper threading enabled or disabled.

Screen Shot 2017-02-21 at 22.05.04.jpg

These discussions were all covered thoroughly back in February:

http://duc.avid.com/showthread.php?p...95#post2414395

http://duc.avid.com/showthread.php?p...77#post2414277

Interestingly 59.9hz said in this post here:

http://duc.avid.com/showpost.php?p=2357974&postcount=7

A word about msconfig. This tool allows you to instruct Windows to only use a certain amount of logical processors after the next reboot. This feature is mainly used in debugging software, and simply instructs Windows to ignore some of the processor cores/virtual cores. This does not disable Hyper-threading. Hyper-threading is enabled/disabled at the motherboard firmware level, i.e. from BIOS.

I would guess that the Instruments method on OS X is also a software level debugging feature as it's reset on restart.

In a roundabout way I feel that the cyclic arguments on these threads are placing blame at the users door and are counterproductive to highlighting an obvious flaw
in the software.

It appears to me there are three problems, that dependent on system, session, and plugins in use are conspiring or not to cause the issue:

1.
Wonderboy has highlighted an interesting and valid flaw in the aux architecture. Others me included too have noticed and reproduced this problem with sessions that are routed through cascaded auxiliaries.

2.
I notice this problem is also exacerbated by certain plugins. For example Wonderboys session fell over with only a few BX Meters placed on the aux.

Which the following quotes from Avid themselves backup:

Hi Alex

Thanks for making contact with Avid Customer Care.
We are sorry for your inconvenience.
And also we are very sorry for our slow response.

We got the same kind of report from many customers recently
and looked into their .dlog files and found that
in most cases, the issue was caused by some 3rd party plugins including:

Magic AB
Waves plugins
Brainworx plugins
BlueCat Patchwork
Native Instrument plug-ins
and
UAD plug-ins.

The last dlogs from Pro Tools 12.7, shows AAE -9173 on the following plugins:

Eleven On Audio Track 2TRK REF DEMO_bip.dup2 at Insert C
Abbey R Abbey Road Plates On Aux Track Vb1.dup1 at Insert A

Thank you and have a good day.

Warm regards,
Markus
Avid Audio

3.

And then theres the graphic/window interaction that others are noticing, for example
here's a version of the Avid demo song with Avid only plugins which will -9173 if you flick through the memory locations sequentially

https://www.dropbox.com/s/46ve73qeej...%2001.ptx?dl=0

[Darryl Ramm]

Originally Posted by Burbree

I'd argue Darryl that people are instead fixated on achieving reliable consistent performance in Pro Tools. At present that doesn't seem achievable for many. The CPU meter in Pro Tools mirrors quite closely Apples own activity monitor CPU usage with hyper threading enabled or disabled.

Yes but CPU % reported in both does not tell you what is happening with these real time latency sensitive processing systems. Hoping to max out CPU consumption before having that real-time latency sensitive system fall over just may not be a good assumption. All reports of %CPU with hyperthreading enabled are fantasy. Counting the hyperthread pair as having 100% of a virtual CPU is just a falsehood and probably helps explain part of why %CPU may be so low in some of the problems people report here.

I would guess that the Instruments method on OS X is also a software level debugging feature as it's reset on restart.

Not sure what you are worried about here. XCode Ibstruments does the right thing to disable one side of hyperthread pairs.

And thanks for the pointers to past comments on plugins. The UAD ones I do remember as folks having issues with, maybe no surprise there given the high latency overhead of going out to the UAD DSP. I'll go have a play with those other sessions....

I know this is causing people problems, but I still hope that folks affected still try to troubleshoot and hammer out issues as best as they can... I do wonder how many of these problems could have been reduced if Avid had provided better developer/test tools with AAX. Then it's intersting how many developers rely on the JUCE development toolkit, including Avid for their own AAX plugin. It would be intersting to know how many plugins had issues that were more early JUCE to AAX related.

[Burbree]

Thanks for shedding more light on this, it makes sense that this could be a latency problem rather than a CPU usage problem. For infos sake a session that was falling over at 48% with hyper threading enabled registered 68% with hyper threading disabled and fell over in the same way.

If you want to see this problem in action heres some videos of a Avid only session with various different graphics, wifi and fan settings..

http://duc.avid.com/showpost.php?p=2400733&postcount=28

[bleen]

I've been getting a ton of -9173 errors on my normally extremely stable rig, particularly one session that doesn't have many tracks, but regularly throws up the error and then crashes.

I made it all day today reviewing mixes for nine songs with a band (most of them were 60-80 tracks) then opening an 111-track session to make an instrumental version and everything ran perfectly smoothly. After the day I opened the problem session to try and get some work done; after 15 minutes, BOOM! Error then crash. Always seems to be this:

Crash reason: EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x0

Thread 43 (crashed)
0 CFnd + 0xb9ead
rbx = 0x0000700014d77100 r12 = 0x0000000000000007
r13 = 0x0000000000000007 r14 = 0x0000700014d79000
r15 = 0x0000000000000000 rip = 0x000000010d0ecead
rsp = 0x0000700014d76c50 rbp = 0x0000700014d76c60
Found by: given as instruction pointer in context
1 libsystem_platform.dylib + 0x2b39
rip = 0x00007fffca9b5b3a rsp = 0x0000700014d76c70
rbp = 0x0000700014d76c70
Found by: stack scanning
2 libsystem_kernel.dylib + 0x19d41
rip = 0x00007fffca8d4d42 rsp = 0x0000700014d76d28
Found by: stack scanning

Maddening. I'm going to rebuild that session in Logic tomorrow... (I've already made a new PT session and imported everything to no avail)