Disk Cache oddity

[Frank Kruse]

I'm having weird things happening with disk cache. So I have 48GB of RAM. I assigned 16GB to disk cache. I currently have a massive session containing 20 episodes along the timeline to assemble a couple of new ones.

A bit extreme I know but there's no other way to do this efficiently.

PT throws up an error from time to time that I'm running low on RAM and that I should lower the disk cache to let it use more and then the confusing part of the message: "you currently have 16GB of unused space in disk cache"

When I look at the system usage I indeed see 9% of disk cache used. Why on earth are only 9% of 16GB used when I have 200 tracks of BG and FX track-lay for 20 45minute episodes of film in my session? That seems like a miracle to me. Or does PT only cache the surrounding of the current cursor position? I always thought it will try to fill the entire cache.

So I lower the disk cache down to 8GB just to see what happens. MacOS says I now have 32GB available memory total. Memory usage is down to 83% (it was at 99% before) but now disk cache is at 100%.

So before:


dc assigned: 16GB
dc used: 9%
system memory available: 10GB
PT usage window: Memory 99%

after switch to 8GB
dc assigned: 8GB
dc used: 100%
system memory available: 25GB
PT usage window: Memory 83%

So then I set DC back to 16GB. Now Memory in system usage is at 100%. MacOS says I have 32GB available and disk cache is at 0% and doesn't really re-cache after I hit play.

Very odd. What am I not getting? Seems very buggy to me.

Thanks!

Frank.

[BScout]

It does try to fill the cache but it does it very, very slowly if you get to work immediately after opening a session. Try opening the session and wait until the disk cache goes green before starting to work on the session.

(and of course, video is never put in the disk cache)

[Frank Kruse]

Yes I know. I had been working for hours when I started noticing it. I can see it start caching when set to 8GB and then it fills to 100%.

The odd thing is that when set to 16GB it caches only up to 9% and according to PT it's at 100% memory usage (which throws up the warning) bit I still have 10GB of available RAM. Plus it says I have a ton of unused disc cache that it wants me to better give to PT.

The numbers in my first posting are all after DC has settled down.

F.

[Mark Ziebarth]

You have 10 GB of RAM left, where do you get this information from? Activity window?


Best

Mark

[Mark Ziebarth]

The Avid Video engine is a separate thread and takes a lot of RAM, too. Maybe there is miscommunication between Mac OS and the Pro Tools threads? Just a guess...

Best

Mark

[Frank Kruse]

Yes the physical ram left info I get from a little helper that shows that figure.
So it's the total RAM free including all running processes.

[garnoil]

I have 96 gigs of RAM so I can set my cash very high, like 48 or even 64 gigs. I have a session where I put the entire session into the cash, and I still get "could not read from disk fast enough'. I also noticed that the more cash I assign, the more Pro-Tools slows down after several hours of editing. In my system after 4 hours of solid editing, Pro-Tools is so slow that I have to re-boot. When I had cash set to normal, Pro-Tools did not seem to slow down that much. I am on a 5,1 Mac, 96 gig Ram, USB3 or firewire 800 for audio and separate firewire 800 for video. System drive is an SSD, internal.

[Frank Kruse]

What is surprising is that with 10GB free, PT says memory usage is at 100% and needs more RAM and with 24GB free it's at 80% of memory usage.

The same session works just fine on a laptop with only 16GB of RAM total.

[Darryl Ramm]

This is all too confusing without knowing exactly how all these numbers are derived. There are many different meanings of memory use and capacity limits. Especially say the memory size potentially allocated for the disk cache and the amount of that actually in use. I kinda hope Pro Tools memory warnings works from the max size allocated, even if there is currently cache memory unused maybe those warnings could be improved or supressable.

“Little helper” does not mean much, what exact product and what exactly is it measuring.

—-

And I would be curious if errors about not being able to read from disk fast enough are caused by write to disk to free space at the same time as reading in new content. Hard for us users to diagnose, but somebody really eager can spy on the cache behavior with the right tools. This is more and area where Avid ought to have some documentation about what is going on in extreme cases and best practices for handling that

What is the underlying disk in those cases? I do worry about folks relying on disk cache to cover for slow HDD or RAID or NAS systems. That may be just asking for corner case issues.

[Farhoof]

Frank, set "parallel task optimization" lower or disable it.