Thanks for all the input. I guess it's nice to know we're not the only ones with these problems. I've been keeping an eye on our ram usage with Activity Monitor all this week and it seems all of those problems have disappeared.
The numbers in Activity Monitor don't always seem to make since but I've found the best way to tell is if you click on the PT process and hit "inspect" then in the memory tab keep an eye on virtual memory size. As long as I'm below 3.5 or so the system feels pretty safe.
If it's more, we've been printing the tracks we can (tracks with a bunch of plugins) and only using virtual instruments inside of vienna ensemble pro. It gets tricky and time consuming on large mix sessions where you need to match all routing and be aware of key inputs to plugins and all. As well as the fact that our vocals are often using the most plugins and are the most likely to be tweaked down the road. (Still, the best solution in this waiting period for 64 bit and AAX dsp 3rd party support would be freeze track!!)
What I don't understand about Activity Monitor is sometimes the virtual memory size and real memory size add together to be over the 4GB limit. Not sure if this is a GB - billions of bytes / 1024 kind of thing or what.
Anyway, I'd really like to get a definitive list like this from AVID:
https://www.ableton.com/en/articles/...out-of-memory/ (Maybe it's out there and I've just missed it somehow??)
It's frustrating when you spend $400 on an app and they give you all the tools to get around these problems and helpful info on how to deal with it, but you spend $20,000+ on hardware and software and...