Looking for advice on a new system for large virtual instrument sessions

[tbonesteak4dinner]

Hey everyone,

First of all, I know these posts suck, so thank you to anyone who reads through it. I'm looking to get some direction in a new system. Generally I feel like I'm on top of this stuff since I've been building computers since I was a kid, but I feel like heavy Virtual Instrument and plugin use has some variables that I'm missing. I like to stay as much "in the box" as I can, but I'm not opposed to including more outboard gear.

My current setup is posted below, and it's struggling against my large sessions. I run in excess of 32 instrument tracks with VIs, all with multiple plugins per track. With the buffer set high it will absolutely push through a huge session, but there's a CPU overhead I can't explain and it's only in the Pro Tools system usage window. Windows barely sees the CPU cap 50%. I'm starting to use more outboard gear, and running the buffer down at 64 samples with so many VIs and plugins is unrealistic, but necessary at the same time for the signal to leave, go through outboard effects, and return on time. When I was running completely in the box sessions, I could set the buffer high while I was composing and then render offline at the end, but that's becoming less of an option with the direction I'm going.

If money wasn't an issue, but I still wanted to be efficient and not just throw money at equipment I don't need, what kind of system would I need to run the following with zero issues and CPU overhead to spare?

1. Audio Buffer at 64 or 32 samples, or the equivalent.
2. Work at 88.2 and 32 bit float, if possible.
3. Run 32+ (more, if possible) virtual instruments and heavy synths with multiple additional plugins per track. All on highest resolution settings.
4. Mastering plugins running live over the session.
5. No freezing tracks.

That seems like a short list, but I know that's asking a lot from a system. Am I looking at a dual Xeon server class workstation? Am I looking at running multiple slave machines? Is my interface the choke point? If I knew I wouldn't be asking you fine people.


Current setup:
Core i7 3820, speed step, C states disabled, locked at 3.8GHz
16GB of DDR3 1333
4 SSDs, 3 Platter Drives, all meeting/exceeding Pro Tools requirements
Windows 10 Pro, independent installation just for Pro Tools
Pro Tools 11 and 12
loopmidiBE
Finale v25 and 2012
Digidesign 003 Rack

My workflow:
I compose in Finale, and while writing I pipe over midi to Pro Tools using virtual midi cables. Pro Tools is record enabled and hosts all the plugins on the other side. When finished writing, I export the midi file, import into Pro Tools, and do my final editing and automation from there.

Most used VI's and Plugins:
Komplete 11
East West Hollywood Series and Composer Cloud titles
Serum
Various Waves plugins
Various iZotope plugins
Slate Digital
Dozens of others....

And yes, I have trashed preferences.

[musicman691]

If you're using Ozone - cpu hog
Kontakt 5 - cpu hog if what you have uses a lot of scripting
Reaktor 5 & 6 - cpu hogs
if you're using the Arturia MiniMoog - major cpu hog
EW Play & the Hollywood series - plenty of disk throughput needed unless you load all to ram

And that last thing brings up a major issue - you really need to have more than 16 gig of ram. That ain't nowheres near enough for the loads you're talking about running.

[tbonesteak4dinner]

Originally Posted by musicman691

If you're using Ozone - cpu hog
Kontakt 5 - cpu hog if what you have uses a lot of scripting
Reaktor 5 & 6 - cpu hogs
if you're using the Arturia MiniMoog - major cpu hog
EW Play & the Hollywood series - plenty of disk throughput needed unless you load all to ram

And that last thing brings up a major issue - you really need to have more than 16 gig of ram. That ain't nowheres near enough for the loads you're talking about running.

Thank you so much for your reply.

With all of this in mind, I'm wanting to put together a new machine that could run all of these plugins, whether they are known CPU hogs or not with the interface buffer set to the minimum. RAM would of course be much larger in the new build, I'm thinking at least 64GB. I don't want to avoid or drop plugins for being CPU intensive, I'm trying to simply muscle through that with whatever goes into my next system and interface.

Ozone - yes, I'm aware of how heavy it can get, but I like to use it when it fits the project.

Kontakt 5 - nothing too heavy with scripting as far as I'm aware, usually drum sounds with this and battery

Reaktor - I don't use it generally, so no impact on my current projects, but it's something I want to get into and therefore want the power to do that alongside everything else.

Arturia - Haven't used it.

EW Play - here's where things are a little weird. Even with 16GB, I don't have RAM issues. The four SSDs in my build are dedicated to EW libraries and I use disk streaming. In my testing, if I loaded everything to RAM the CPU usage would go up and if I kept disk streaming on, disk usage went way up (nowhere near maximum), but the CPU usage went down. On my largest projects I'm using 8-10GB of total system memory including the OS, as the samples are streaming from disk. Keeps both my RAM and CPU usage low and puts the workload on the SSDs, which they're more than capable of handling. RAM is clearly a low point for my build, but I don't believe it's causing my current bottleneck, as I'm only using about half of it on any given project.

[Marsdy]

I'm a very heavy VI user. I must be using a lot more Kontakt than you because 16GB would get me nowhere. I have a sample server PC that is maxed out with 32GB of RAM and have another 40GB lof samples oaded locally on my Mac running Pro Tools.

Originally Posted by tbonesteak4dinner

1. Audio Buffer at 64 or 32 samples, or the equivalent.

This is pretty optimistic I'm afraid. I can't run at anything like that and I have a top of the range 12 core Mac Pro. Most of the time, the minimum I can get away with is a buffer of 256 and that often has to increase to 512. I'm also using VE Pro which is multiplying the buffer by x2. For even moderate VI use you'll be lucky to get away with 128. I can hardly ever do that. I can cope with a buffer of 256. I guess I've got used to the latency but some singers and musicians I record are more sensitive to it than I and complain. In this case I might have to freeze up a few tracks to get latency down. At 256 I have a system delay of about 640 samples, I'm sure it would be significantly more than that if I wasn't running HDX.

2. Work at 88.2 and 32 bit float, if possible.

Again this is optimistic. Bear in mind many VIs, especially sample libraries will upsample to that resolution so for me there's not much point in running at higher resolutions. Personally I'm quite happy to work at 48K 24 bit. I have tried to work at 88 and 96 but there's such a massive overhead it's just not practical for me.

3. Run 32+ (more, if possible) virtual instruments and heavy synths with multiple additional plugins per track. All on highest resolution settings.

To get anything like the performance you are looking for, you really are going to need Vienna Ensemble Pro to host VIs in my opinion. I personally couldn't use Pro Tools without it and would have work with Logic. Bear in mind I am using huge numbers of Kontakt instances in my template.

4. Mastering plugins running live over the session.

I wouldn't be able to do this simply because the best mastering plug-ins like Ozone and Waves L316 add so much latency. You might even find the latency get so big when running plug-ins and VIs that you go over ProTools' maximum delay compensation allowance of 16k.

5. No freezing tracks.
I never need to do this so you should be okay there.

I'm sure this isn't what you want to hear but I can only speak from my experience and as I say, I find Vienna Ensemble Pro essential for my workflow.

I also use HDX. For some reason latency feels lower at any given buffer setting than it does with native. Also hardware inserts are not a problem. I'm surprised you have to use a buffer of 64k to keep your hardware outboard in time. I can use any buffer provided I don't go over ProTools delay compensation maximum and as I say, that is a possibility if you use mastering plug-ins on your mix bus.

[tbonesteak4dinner]

Originally Posted by Marsdy

I'm a very heavy VI user. I must be using a lot more Kontakt than you because 16GB would get me nowhere. I have a sample Server PC that is maxed out at 32GB of RAM and have another 40GB loaded locally on my Mac running Pro Tools.



This is pretty optimistic I'm afraid. I can't run at anything like that and I have a top of the range 12 core Mac Pro. Most of the time, the minimum I can get away with is a buffer of 256 and that often has to increase to 512. I'm also using VE Pro which is multiplying the buffer by x2. For even moderate VI use you'll be lucky to get away with 128. I can hardly ever do that. I can cope with a buffer of 256. I guess I've got used to the latency but some singers and musicians I record are more sensitive to it than I and complain. In this case I might have to freeze up a few tracks to get latency down. At 256 I have a system delay of about 640 samples, I'm sure it would be significantly more than that if I wasn't running HDX.

2. Work at 88.2 and 32 bit float, if possible.

Again this is optimistic I'm afraid. Bear in mind many VIs, especially sample libraries will upsample to that resolution so for me there's not much point in running at higher resolutions. Personally I'm quite happy to work at 48K 24 bit. I have tried to work at 88 and 96 but there's such a massive overhead it's just not practical for me.

3. Run 32+ (more, if possible) virtual instruments and heavy synths with multiple additional plugins per track. All on highest resolution settings.

To get anything like the performance you are looking for, you really are going to need Vienna Ensemble Pro to host VIs in my opinion. I personally couldn't use Pro Tools without it and would have work with Logic. Bear in mind I am using huge numbers of Kontakt instances in my template.

4. Mastering plugins running live over the session.

I wouldn't be able to do this simply because the best mastering plug-ins like Ozone and Waves L316 add so much latency. You might even find the latency get so big when running plug-ins and VIs that you go over ProTools' maximum delay compensation allowance of 16k.

5. No freezing tracks.
I never need to do this so you should be okay there.

I'm sure this isn't what you want to hear but I can only speak from my experience and as I say, I find Vienna Ensemble Pro essential for my workflow.

I also use HDX. For some reason latency feels lower at any given buffer setting than it does with native. Also hardware inserts are not a problem. I'm surprised you have to use a buffer of 64k to keep your hardware outboard in time. I can use any buffer provided I don't go over ProTools delay compensation maximum and as I say, that is a possibility if you use mastering plug-ins on your mix bus.

Thank you so much for your input.

Really the only true requirements of the above are a low buffer and the potential to run 32+ VI tracks. I play nothing live, but the reason I want to run the buffer all the way down at 64 or 32 samples is because I'm record monitoring the input from finale. In realtime, the synth needs to output to my analog signal chain and return fast enough that I don't notice the handful of ms of latency. The Pro Tools playhead isn't moving a muscle while I'm composing in finale, only the last stage where I import the midi file and edit. If I were playing back MIDI inside of Pro Tools I'm sure it would compensate better. Not playing anything live is also the reason why I don't care about delay imposed by plugins on the master bus - as long as anything I'm running through analog effects sends and returns before that point, then it doesn't matter since it's all sequenced playback anyways.

That may also be part of the issue. I may need to just get comfortable writing music in the piano roll inside of Pro Tools. I'm no stranger to it, but it's pretty cumbersome, and I haven't found a DAW that can handle notation. Pro Tools and Cubase both pretend to, and they don't pretend well. But the fact that the CPU overhead is miles lower while Pro Tools doesn't have a ton of VI tracks record enabled might be the tipping point for me. My biggest issue right now is with synths. I've got a couple synth heavy projects, and the insane polyphony in some of my massive and serum patches will bring the Pro Tools CPU load into the red real quick.

I do know that a viable answer is to work strictly inside of Pro Tools and maybe occasionally freeze some of those heavier pads, but holy crud am I sick of workarounds. I'm honestly entertaining the idea of a dual Xeon workstation just so I don't have to do anything but write music. 32-40 logical cores might get that done for me, but that price tag doesn't justify a "might."

[musicman691]

Originally Posted by tbonesteak4dinner

That may also be part of the issue. I may need to just get comfortable writing music in the piano roll inside of Pro Tools. I'm no stranger to it, but it's pretty cumbersome, and I haven't found a DAW that can handle notation. Pro Tools and Cubase both pretend to, and they don't pretend well. But the fact that the CPU overhead is miles lower while Pro Tools doesn't have a ton of VI tracks record enabled might be the tipping point for me. My biggest issue right now is with synths. I've got a couple synth heavy projects, and the insane polyphony in some of my massive and serum patches will bring the Pro Tools CPU load into the red real quick.

Have you taken a look at Digital Performer? I don't do notation inside DP but from what I've seen of it it can do the deed. 30 day fully functional demo and no iLok needed. One version that does everything with no artificial limitations like PT.

[Marsdy]

Originally Posted by tbonesteak4dinner

Thank you so much for your input.

Really the only true requirements of the above are a low buffer and the potential to run 32+ VI tracks. I play nothing live, but the reason I want to run the buffer all the way down at 64 or 32 samples is because I'm record monitoring the input from finale. In realtime, the synth needs to output to my analog signal chain and return fast enough that I don't notice the handful of ms of latency. The Pro Tools playhead isn't moving a muscle while I'm composing in finale, only the last stage where I import the midi file and edit. If I were playing back MIDI inside of Pro Tools I'm sure it would compensate better. Not playing anything live is also the reason why I don't care about delay imposed by plugins on the master bus - as long as anything I'm running through analog effects sends and returns before that point, then it doesn't matter since it's all sequenced playback anyways.

That may also be part of the issue. I may need to just get comfortable writing music in the piano roll inside of Pro Tools. I'm no stranger to it, but it's pretty cumbersome, and I haven't found a DAW that can handle notation. Pro Tools and Cubase both pretend to, and they don't pretend well. But the fact that the CPU overhead is miles lower while Pro Tools doesn't have a ton of VI tracks record enabled might be the tipping point for me. My biggest issue right now is with synths. I've got a couple synth heavy projects, and the insane polyphony in some of my massive and serum patches will bring the Pro Tools CPU load into the red real quick.

I do know that a viable answer is to work strictly inside of Pro Tools and maybe occasionally freeze some of those heavier pads, but holy crud am I sick of workarounds. I'm honestly entertaining the idea of a dual Xeon workstation just so I don't have to do anything but write music. 32-40 logical cores might get that done for me, but that price tag doesn't justify a "might."

I'm really sorry to rain on your parade again but I think you'll be disappointed to a degree. Even with 32 cores you would still have problems because Pro Tools can't spread the load of any given VI across muiltiple cores. One core is quite often not enough for really greedy CPU hogs, ESPECIALLY when they are record enabled. Pro Tools has a dual buffer engine, so when you are record armed you are at the buffer set in Playback Engine. When a track is in playback Pro Tools goes to a much higher buffer in the background. This isn't working properly, (long story!) but what you will notice is that CPU usage on demanding Kontakt instruments for example goes UP noticeably when they are record armed. In other words, your workflow is quite demanding of CPU unfortunately!!! Yes you would indeed be better off doing your sequencing in PT so you don't have to record arm all those tracks AND you can run at a higher buffer.

I actually wish I'd bought an 8 core Mac Pro because each indivual core is quite a lot faster than the 12 core. It's not just the number of cores it's how fast they are!

As I said, hosting your VIs in VE Pro would help a LOT. I couldn't work without it. Yes it's a workaround but it's a very, sophisticated and reliable one. Another option is Logic which is much more CPU efficient with VIs and coincidently has quite good scoring features though no match for Finale.

[tbonesteak4dinner]

Originally Posted by musicman691

Have you taken a look at Digital Performer? I don't do notation inside DP but from what I've seen of it it can do the deed. 30 day fully functional demo and no iLok needed. One version that does everything with no artificial limitations like PT.

I did try DP a while ago and unfortunately it wasn't my thing. I've actually gone through most of the major DAWs at some point, testing notation along the way. Cubase was the closest, but their visual quantization vs actual quantization for notation was the nail in the coffin for me. I think me and the piano roll are going to need to get nice and cozy...

Originally Posted by Marsdy

I'm really sorry to rain on your parade again but I think you'll be disappointed to a degree. Even with 32 cores you would still have problems because Pro Tools can't spread the load of any given VI across muiltiple cores. One core is quite often not enough for really greedy CPU hogs, ESPECIALLY when they are record enabled. Pro Tools has a dual buffer engine, so when you are record armed you are at the buffer set in Playback Engine. When a track is in playback Pro Tools goes to a much higher buffer in the background. This isn't working properly, (long story!) but what you will notice is that CPU usage on demanding Kontakt instruments for example goes UP noticeably when they are record armed. In other words, your workflow is quite demanding of CPU unfortunately!!! Yes you would indeed be better off doing your sequencing in PT so you don't have to record arm all those tracks AND you can run at a higher buffer.

I actually wish I'd bought an 8 core Mac Pro because each indivual core is quite a lot faster than the 12 core. It's not just the number of cores it's how fast they are!

As I said, hosting your VIs in VE Pro would help a LOT. I couldn't work without it. Yes it's a workaround but it's a very, sophisticated and reliable one. Another option is Logic which is much more CPU efficient with VIs and coincidently has quite good scoring features though no match for Finale.

Unfortunately I've had first hand experience with that multiple times, I was just hoping a more robust CPU would be able to spread those CPU heavy plugins to one of the many cores and spread the load evenly. I should have known it'd be about the same. Pro Tools is in love with CPU core #4 for my system, it's got seven other perfectly good cores and doesn't give them half the attention. It even overrides OS processer affinity when I force it manually to stay away from core #4. I assume that's because one big plugin is just camping out there and taking all of its resources.

As for VEP, I've actually used it multiple times to get through projects in the past, just never in a slave machine, so I do have a couple questions. If I run a heavy synth or two on the slave machine, does that actually reduce the CPU overhead on the master machine? In my testing, I ran a server on the same computer as Pro Tools, as I just wanted to take advantage of the additional buffers. The ASIO driver load never went down, so the only performance gains were from the additional buffers. If I run it from a slave machine, would it actually offload those resources to that machine even if I chose not to add an additional buffer?

[Marsdy]

Originally Posted by tbonesteak4dinner

I did try DP a while ago and unfortunately it wasn't my thing. I've actually gone through most of the major DAWs at some point, testing notation along the way. Cubase was the closest, but their visual quantization vs actual quantization for notation was the nail in the coffin for me. I think me and the piano roll are going to need to get nice and cozy...



Unfortunately I've had first hand experience with that multiple times, I was just hoping a more robust CPU would be able to spread those CPU heavy plugins to one of the many cores and spread the load evenly. I should have known it'd be about the same. Pro Tools is in love with CPU core #4 for my system, it's got seven other perfectly good cores and doesn't give them half the attention. It even overrides OS processer affinity when I force it manually to stay away from core #4. I assume that's because one big plugin is just camping out there and taking all of its resources.

As for VEP, I've actually used it multiple times to get through projects in the past, just never in a slave machine, so I do have a couple questions. If I run a heavy synth or two on the slave machine, does that actually reduce the CPU overhead on the master machine? In my testing, I ran a server on the same computer as Pro Tools, as I just wanted to take advantage of the additional buffers. The ASIO driver load never went down, so the only performance gains were from the additional buffers. If I run it from a slave machine, would it actually offload those resources to that machine even if I chose not to add an additional buffer?

I guess you're seeing the dreaded pyramid of CPU usage where the middle cores get hammered most. That evens out noticeably when you are not record armed but that is obviously no help with your workflow.

Yes you would need to use the additional buffers in VE Pro. At least I always have done and have them set to x2. That is the price you pay for using VE Pro. However, I find it is definitely less demanding overall than running the same VIs in ProTools at a higher buffer because there is MUCH less CPU spiking. On my slave PC, the CPU usage is right up there so there is definitely a LOT of processing being offloaded. Ultimately, I find it way more solid and stable than running the VIs in Pro Tools and get very few -9171/3 errors. I also get significantly better performance running VE Pro locally on the same Mac as ProTools. It's all about juggling compromises!

At least the piano roll editor in PT is a very good one and second only to Cubase IMO!

[musicman691]

Originally Posted by tbonesteak4dinner

I did try DP a while ago and unfortunately it wasn't my thing. I've actually gone through most of the major DAWs at some point, testing notation along the way. Cubase was the closest, but their visual quantization vs actual quantization for notation was the nail in the coffin for me. I think me and the piano roll are going to need to get nice and cozy...

What version of DP did you try? DP9 has come a long way from the olden days.

What didn't you like about DP?