HDX PT 2018 CPU Spikes AAE -9171 Halts Playback

[Howardk]

Further testing shows that with edit and mix window closed CPU spikes are greatly reduced, however the system still halts during the unattended test. . . ran it multiple times. . . it made it 83 minutes and 64 minutes before am AAE -9171 Halt, even thou the Total CPU shows around 25% peak at the start.

Although it doesn't look like it will be the solution, I found a Gigabyte AMD Radeon R9 280X (7970) 3GB for Apple Mac Pro card on ebay from what appears to be a responsible seller. Claims to have Apple boot screen etc. . . I have to sort out how to best power the video & HDX cards. . .

[the.engineer]

That is the exact video card that I have. I spliced one of the graphics card power cables together with the HDX card power cable. Works a charm.


Sent from my iPhone using Tapatalk

[Howardk]

Originally Posted by the.engineer

That is the exact video card that I have. I spliced one of the graphics card power cables together with the HDX card power cable. Works a charm.
Sent from my iPhone using Tapatalk

Instead of splicing, I might get a SATA splitter and split the connector in the Optical Bay (I have an Optical Drive and a 6TB Scratch Drive in there), and run a power extension to the HDX card. Did you find a spec on the HDX current card draw? I remember someone saying it is just for the fan, but I have not verified.

[Howardk]

Hi James

The steps to upgrade to 12 Core 3.46 GHz looks pretty straight forward and not that expensive. Anything that I can do within reason, to push things past this damn AAE-9171 Halt threshold is well worth it. . . I have little/no patience, especially when there are clients around, for this sort of thing (which I had expected Avid would be well past in 2018 considering how solid PT 10/HD has been).

Here are a couple links:

Outline of Process
Example 1 of eBay Kit
Example 2 of eBay Kit

I have 2 x 2010 Mac Pros(I wanted to have a spare computer around, to help insure I don't miss a deadline due to a hardware/software issue),
a) 2.66/12 Core, and
b) 2.93/12 Core. . .
I think the Processor tray is the same (I will check). . . if so, I may as well upgrade the 2.66 to 3.43 GHz

Sounds like you have gone thru this already. . . I am an electronics tech from another life, so I am good with this stuff, but anything more you think that I need to know or consider to get this done right?

[Howardk]

Summary of where things are at for me

After many many tests and configuration tweaks and inquiries. . . I can not find a way to make a Pro Tools 2018-10(or -12) with HDX card, work reliably with an acceptable buffer size, without CPU Spikes which cause AAE-9171 errors and Playback or Recording Halts.

Even with 512 buffer and no Native plugins, this still occurs unpredictably, but infrequently with large buffer. . . tested with Sierra and El Capitan and I have tried two different Mac Pros (2010, 12 cores).

Closing the edit and mix Windows reduces the CPU Spikes/Surges on the Total CPU Meter, but the Halt still will occur sooner or later. I have tested with lower resolution monitor settings, HDMI, DP and DVI, and it still occurs. There is also some randomness to these Halts. . . sometimes the system will be fine for hours with 256 or 512 buffers, and then out of the blue it will Halt frequently, and a restart does not make a difference, nor does deleting Prefs, forcing the cron scripts with Cocktail, etc. . . this poor performance varies, randomly.

Disabling Hyper-Threading did not seem to alleviate the CPU spikes and Halts.

I have tried with just a single Interface, with no difference. Have tried a single AVID 16x16, single Aurora 16, single Digidesign 192 i/o, no difference noticed. HDX Card passes Digitest, but there are some interface related errors, which I understand are normal, passes clean with no interface attached.

These Pro Tools iterations, on my configurations could never be used to record a live show with any level of confidence. . . as compared to my Pro Tools 10 HD Accel rig, which was/is very reliable but unfortunately is not supported by certain VIs I wish to work with and does not have the Freeze or Commit feature.

I have tried with all extra cards out, fresh OS X and PT installs with only the AVID plugins in the folder (El Capitan and Sierra), with no perceivable improvements. I believe I have tried everything on AVID's optimization notes with no perceivable improvements.

I also get a delay of about 5 seconds each time I click on solo for a playlist lane, even when there is only a hundred playlists, making it impossible to productively audition Takes.

Pro Tools HD 12.8.1 on Yosemite does not have these problems. At the suggestion of James, I tried this combination and found it does not show these CPU Spikes and my system seems to work reliably with a 64 buffer and multiple AAX.native plugins. Also, soloing playlist lanes works fine, without the delay. I had zero problems in these areas in during a 6 hour recording session today. Also, my Test Session has run multiple test cycles without a Halt/Error with a 64 buffer (two x 2 hour recording test cycles, and one 4 hour recording test cycle). With Pro Tools 2018 and a 64 buffer, my system often can’t run for a second without a Halt, and also halts from time to time with a 512 buffer, which is also really hard to work with when recording midi/VIs, Native Guitar Amp sims, etc. . .

I am not the only one with this problem. What a step backward!

I am curious if anyone with a 2010 or 2012 Mac Pro can run Pro Tools 2018 with some native plugins and small buffers without similar problems?

Click here - a link to my test session on Google Drive.


System Summary
Mac Pro 2010 12 Core 2.93 GHz 48GB Sierra (10.12.6)
Video: ATI Radeon HD 5770, 34" Samsung SE790C (DP or HDMI) 20" Dell 2005FPW (DVI @90 deg rotation)
Drives:
- System: OWC Mercury EXTREME Pro 6G SSD
- Sessions: Samsung SSD 850 EVO 500GB
- Samples: Samsung SSD 860 EVO 1TB
- Trim Enabled
- Sonnet Tempo SSD Card (PCIe 2xSSD), System and Sample SSDs
PCIe USB3/eSata Card (2x2)
PT Ultimate HDX 2018-12
Interfaces: Lynx Aurora 16 (x 2) + Digidesign 192 i/o (x 2)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[LDS]

Great test!

I will giv it a spin when I am back in front of my computer tomorrow.

[the.engineer]

Great work Howard, thanks for being our guinea pig!

What were your conclusions around the feature differences between 2018.x and 12.8.1? Any major features missing?

And did you get your plugins working?

[Howardk]

Originally Posted by the.engineer

Great work Howard, thanks for being our guinea pig

What were your conclusions around the feature differences between 2018.x and 12.8.1? Any major features missing?

And did you get your plugins working?

I have not spent enough time yet on Yosemite and PT 12.8.1, not tested Virtual Instruments, and only done minimal editing, . . . . but so far I have not found anything that I need missing and it works much better than 2018-12. I have only had one Pro Tools Crash and used PT 12.8.1 for about 12 hours to date. These infrequent crashes occur with 2018-12 as well, resulting in lost work, require a restart, sometimes deleting Prefs is required. My Plugin Installations on Yosemite was not done normally/properly. . . I simply dropped all my plugins from PT 2018-12 into the 2012.8.1 Plugins folder, and skipped anything that was already there (be sure to first start PT 12.8.1 once so it creates the Plugins folder and installs the defaults). Not a good move, I know, which will rear its ugly head, but it was a quick way to get going quickly and the essentials worked.

Also, I noticed there are a lot of bug fixes in PT 12.8.2, especially to MIDI, which means these issues are unresolved in PT 12.8.1. Unfortunately PT 12.8.2 is not listed as compatible with Yosemite, Pro-Tools-Operating-System-Compatibility-Chart. In tests I found that PT 12.8.1 exhibits CPU Spikes on El Capitan and Sierra, so I presume 12.8.2 will as well, but I have not tested 12.8.2.

These CPU Spike and display lags must be "old news" considering how long ago these issues appear to have crept in, so it is ridiculous that we are Guinea Pigs. I am still in disbelieve these problem even exist. . . I am hoping someone will swoop in and say their 2010/2012 Mac Pro system works perfectly with Sierra, or High Sierra and a PT 2018 version, and all I have to do is "x" to resolve. Wish I could borrow a TrashCan with a Sonnet PCIe expansion chassis, to see if these problems exist in that setup. Regardless, AVID should take responsibility and make it very clear there is a problem, so customers can make an informed decision and start in a better place, or stay in a better place. I am into this for about 60 hours of my time over that last 2 months, as I pulled my hair out trying to find a workable place to land. Multiple that out across the user base and we are talking millions of dollars in wasted time and lost productivity. Fixing this issue should be job 1 for AVID IMO. Not cool!

I suggest there should at the very least be a concise "Sticky", once others substantiate my findings.

[smokeydan]

Hi Howard - I am sorry to say - we have a trash can with a Sonnet Chassis and are experiencing similar issues with our HDX system since we got it last year. High Sierra installed, loads of RAM, 12core processing. PT HD 12.8.3 (Don’t really want to update to a more recent version half way through an album project unless someone can guarantee it will fix these issues!)

Dissappointed in the extreme.

I went all round the houses trying to find the cause/remedy -
Disable Hyperthreading - not much improvement.
changed all thunderbolt and digilink cables - no change
Tried a different Sonnet Chassis, different HDX cards, Different interfaces. No change.
Connected to a smaller/lower res screen - appeared to allow ProTools with lower CPU usage.
I have tried running with both edit and mix windows minimised and then closed altogether, it doesn’t appear to make a noticeable difference on my system.

In certain circumstances, a session which is running reliably at low buffer will fall over with a CPU spike if I hit record or input monitor on any audio channel which is routed to an aux with a Native Plug in on it.
When I run the session with “Built In Audio” selected in the playback engine, the session will run reliably at the lowest buffer setting and very low and stable CPU usage.
However, when I tried running the session with an HD Native PCIe card on a completely different computer, similar issues returned - so my current theory is that it might have to do with the Avid HD driver?

You mentioned in one of the earliest posts about latency when using high buffer settings - As a work-around - if you route the audio track you are recording onto directly to the output (rather than to the mix bus where everything else is going), there’s “no” latency, regardless of buffer setting. (as Protools now has a small input buffer which is fixed, and the buffer you set in the playback engine is the “output buffer” as far as I understand it.)
Also - I’m not certain about this, but I think that every time the audio goes from Native to DSP(HDX card) or back again, it runs through the user set buffer, so at higher buffer settings you are multiplying your monitoring latency issues.

We have to be careful about the order of plug ins when mixing DSP and Native plug ins in a track (or the session generally) to minimise the number of times the signal path goes from Native to DSP and back again. If you have to mix Native and DSP plug ins on an audio track, try to make the order is Native first then DSP and don’t go back to Native after that. Native plug ins on Auxes cause particular problems as the audio is already in the DSP domain when it hits the Aux, so putting a Native plug in on an Aux forces the audio to return to native and then back again to the HDX card at the bottom of the Aux channel. Some Native plugins are worse than others - Altiverb in particular can cause CPU spikes in this situation when running at a low buffer setting. Waves Abbey Rd Plates and Kramer Tape, certain UAD plug ins (which protools treats as Native for these purposes) Clariphonic eq, NI Solid Bus Compressor and several others use a lot of CPU when used on an Aux channel with low buffer settings.

Generally speaking, we have been tracking this way, with the buffer set to max and have had more or less reliable performance this way, but now we’re at the latter stages of the project, we are tracking more midi/keyboard lines with virtual instruments, and the high buffer setting is adding latency to the midi when tracking - and I can’t think of a work around for that. Which is a pain, so I have had to revisit each session and try to find ways of getting them to run reliably at the lower buffer settings. Re-ordering the Native/DSP signal flow as described above, Freezing tracks here and there, making muted channels or bypassed plug ins inactive etc.
I have also recently updated to the latest version of High Sierra and update the iLok driver, after reading posts saying that a couple of older versions of the iLok driver were causing instability issues - Don’t know if its a placebo, but this does seem to have improved the situation with regard to the CPU spiking to a small degree. One way or another I have been able to modify each session in such a way that I can run at the minimum buffer setting. Some of the adjustments are less than ideal though (eg freezing aux channel with reverbs on)
Whilst doing this I have come across two sessions which are running several instances of Altiverb on Auxes, and even on low buffer settings, these sessions have no CPU spikes - and I can't figure out what is different on these sessions compared to every other session where running Altiverb on an Aux is absolutely out of the question. So this is the next thing for me to investigate - I’ll see what happens when I import these "good" Aux/Altiverb tracks into the sessions where I am having problem. And vice versa.

interesting post about running Yosemite and 12.8.1 - When I find time I’m going to have to create a Yosemite boot disk then - Then I’ll have to find an installer for 12.8.1 - unless anyone here can tell me that 12.8.3 will run on Yosemite?

cheers and good luck - from the look of this forum, seems like most people who are running HDX systems are having to run with high buffer settings in order to run the plug ins they want to use.

[Howardk]

Originally Posted by smokeydan

I am sorry to say - we have a trash can with a Sonnet Chassis and are experiencing similar issues with our HDX system since we got it last year. High Sierra installed, loads of RAM, 12core processing. PT HD 12.8.3

Disappointed in the extreme.

Sounds like you are seeing the same issue, and have put a lot of time into trying to resolve, to no avail. . . yes, very disappointing.

Originally Posted by smokeydan

but now we’re at the latter stages of the project, we are tracking more midi/keyboard lines with virtual instruments, and the high buffer setting is adding latency to the midi when tracking - and I can’t think of a work around for that.

Nor can I. As you know VIs need to be in-time to feel responsive and get any kind of feel when tracking. I use guitar amp sims as well, such as Positive Grid and Helix, which are also native and subject to this issue making it next to impossible to be productive.

I will read your post more carefully, with regard to mixing DSP/Native plugins etc. . . but I do not use an Native plugins on buses I am routing thru, but I do always have a Stereo bus that all tracks are routed thru, with a mix buss compressor, currently ML4000 from McDsp (AAX.dsp). This Mix bus is also routed to an Audio track which enables me to print mixes, while doing manual moves for reference for artists, punching in mid song on a reference mix or touch up, etc. . . I have been working this way forever. . . it saves oodles of time, and has a myriad of benefits.

Although not a new issue, Auxes with FX returns and plugins such as a Universal Audio Plate Reverb add latency to tracks in record. . . I just remove the compensation delay on these tracks when recording. . . and the latency for the musician's tracks in record reduces accordingly.

Originally Posted by smokeydan

Whilst doing this I have come across two sessions which are running several instances of Altiverb on Auxes, and even on low buffer settings, these sessions have no CPU spikes - and I can't figure out what is different on these sessions compared to every other session where running Altiverb on an Aux is absolutely out of the question. So this is the next thing for me to investigate - I’ll see what happens when I import these "good" Aux/Altiverb tracks into the sessions where I am having problem. And vice versa.

That is both strange and hopeful. Good luck! I have had inconsistent performance behavior I can not explain as well. . . but have not nailed it down to a specific session or setting. . . I my case it seems to be random.

Originally Posted by smokeydan

interesting post about running Yosemite and 12.8.1 - When I find time I’m going to have to create a Yosemite boot disk then - Then I’ll have to find an installer for 12.8.1 - unless anyone here can tell me that 12.8.3 will run on Yosemite?

AVID's info says 12.8.3 on Yosemite is not compatible. There are a lot of fixes in 12.8.2, so I am a bit concerned about 12.8.1, but I had another very good workday on it today. Maybe these newer versions are ok.. . 12.8.2 & 12.8.3, but not on AVIDs OS/Pro Tools version grid. . . there is a link to the page my post back a few.

Send me a PM if you need 12.8.1. . . It would be helpful if AVID provided access to earlier Pro Tools versions so we can do this kind of testing, to get our systems stable so we can work productively. . . and it would be great if AVID pointed out issues like this, and directed up toward the optimum combo of OS/PT Version for our situation in the notes.

BTW, I believe I did try 12.8.1 on El Capitan and Sierra and the CPU Spikes "problem" came back (need to check my notes to be sure).

Originally Posted by smokeydan

cheers and good luck - from the look of this forum, seems like most people who are running HDX systems are having to run with high buffer settings in order to run the plug ins they want to use.

Yes, that is what I hear in my limited access to Pro Tools HDX users running newer versions of Pro Tools/OS X. . . still hoping someone has found the magic bullet and can enlighten us all. . . .

Meanwhile people running old Mac Pros like mine are doing huge mixes with no issues using 3rd party interfaces, such as Universal Audio Apollo. . . with UA's stuff. Between Native processing, and the UA PCIe DSP horsepower there is a lot to work with, and for only for dimes to our dollars in investment.

I really wanted to stay with AVID DSP so I could record large groups live with monitoring from within Pro Tools, and be able to use a little bit of native for VIs and such, as I have been doing since 1998 with the Mix system and every Pro Tools platform since. . . . I hope this is not as widespread as it looks. I encourage AVID to take responsibility and provide clearer information to their customers, and ask that fixing this become Job 1. . . seems like some kind of computer bus contention issue (bottle-neck) which is likely going to have some Apple/OS issues as part of the problem. . . way beyond my pay grade.