PT10's Disk Cache or RAM Disk. Why not use the same concept for VI memory space?

[sunburst79]

As the title of the thread says. Why not apply the same concept to Virtual Instruments and their RAM needs. If your composing these are the true memory hogs. Could we load the VI Samples into a dedicated external RAM Space? If the timeline is Disk Cache why not load the samples into Sample Cache? Sample Cache or Memory Server whatever you would like to call it the concept is much the same. I'm assuming that if this was easily done it would have already been incorporated into PT 10. I'm going to guess that any RTAS VI's must share PTs memory space. But I wanted to raise the subject. Could this be done in a version upgrade of PT or as part of a AIR instrument rewrite? Good idea? Bad Idea?


I'm sure someone will point out several reasons why its not feasible but it seems with disk cache we are already halfway there. I like the idea.

[Dism]

I think what we have to take away from that video is that is just a taste of things to come.

I look at it this way... They could have demonstrated a 64 bit Pro Tools with some sort of offline rendering. They could have loaded up a session full of VIs and said, "Hey, look! We can finally do what other hosts are doing!"

Realistically, that would be boring, because this is what we are already anticipating/expecting in the versions to come. Why show what we're expecting already, instead of showing entirely new features that actually challenge the competition rather than playing catch up?

It has genrated far more interest and discussion. Otherwise, we would have had an 8 page long thread of people saying, "Yay, it's coming!"

[sunburst79]

The way I see it it's that it should be possible. All they need to do is recode the plugs to offer a choice between streaming samples from a drive or hosting the samples in RAM. At that point you would simply point to the ramdisk instead of the mechanical drive.



Sent from my iPhone using Omelette and Home Fries.

[John_Toolbox]

Originally Posted by sunburst79

As the title of the thread says. Why not apply the same concept to Virtual Instruments and their RAM needs. If your composing these are the true memory hogs. Could we load the VI Samples into a dedicated external RAM Space? If the timeline is Disk Cache why not load the samples into Sample Cache? Sample Cache or Memory Server whatever you would like to call it the concept is much the same. I'm assuming that if this was easily done it would have already been incorporated into PT 10. I'm going to guess that any RTAS VI's must share PTs memory space. But I wanted to raise the subject. Could this be done in a version upgrade of PT or as part of a AIR instrument rewrite? Good idea? Bad Idea?


I'm sure someone will point out several reasons why its not feasible but it seems with disk cache we are already halfway there. I like the idea.

Right now, Kontakt Memory server does this for mac users, I think PLAY also does it. Like jbridge on a PC, it attaches the VI memory to a separate process for each 4GB of RAM you need. It definitely works, but it's very CPU intensive. I tested KMS with both Logic and PT9, and in both instances with the same MIDI and VI's loaded the CPU usage was the same. If you load the same VI's in Logic, but start Logic in 64 bit mode and turn off KMS, the CPU usage was almost cut in half.

So it will work, but not nearly as efficiently as a true 64 bit app hosting 64 bit plugins.

I think with true 64 bit, you have one Kontakt engine taxing the CPU. With KMS, I think what happens is you have a separate Kontakt Engine taxing the CPU for each 4GB of memory. I tested with about 13GB of samples loaded(16GB of RAM available in the computer). So in this scenario I essentially had 4 Kontakt engines running simultaneously, where if you run one Kontakt engine in 64 bit mode, it handles the 16GB of RAM as a single process, thus cutting way back on CPU usage.

[sunburst79]

Originally Posted by John_Toolbox

Right now, Kontakt Memory server does this for mac users, I think PLAY also does it. Like jbridge on a PC, it attaches the VI memory to a separate process for each 4GB of RAM you need. It definitely works, but it's very CPU intensive. I tested KMS with both Logic and PT9, and in both instances with the same MIDI and VI's loaded the CPU usage was the same. If you load the same VI's in Logic, but start Logic in 64 bit mode and turn off KMS, the CPU usage was almost cut in half.

So it will work, but not nearly as efficiently as a true 64 bit app hosting 64 bit plugins.

I think with true 64 bit, you have one Kontakt engine taxing the CPU. With KMS, I think what happens is you have a separate Kontakt Engine taxing the CPU for each 4GB of memory. I tested with about 13GB of samples loaded(16GB of RAM available in the computer). So in this scenario I essentially had 4 Kontakt engines running simultaneously, where if you run one Kontakt engine in 64 bit mode, it handles the 16GB of RAM as a single process, thus cutting way back on CPU usage.

I'm figuring its possible. What I'm thinking is that it would be FAR more efficient to let PT handle all memory server processes or act as a VI RAM Macromanager and either handle all outside sample processes itself or hand them of to the OS for management. Regardless of bit depth this would be good for those VI's that primarily stream samples. Managing samples streaming in and out of a RAM disk shouldn't be any more difficult that managing audio on the timeline. Running out of sample space is a bigger concern the CPU cycles for a lot of people.

[sunburst79]

Also I realize that there are some existing solutions out there. I'm just thinking that something centrally managed by PT itself would be a lot more elegant and effective.

[John_Toolbox]

If you had 16GB of RAM, and Pro Tools saw 12GB of it as a disk, and didn't need 3 separate 32 bit processes to access this disk, then it theoretically should be much more efficient than having to use a separate process for every 4GB of RAM you need(the way KMS works).

One slight gotcha, is that most sample libraries are set up to either load everything in RAM only (with no disk streaming), or load the first part of a sample and stream the remaining part of it from disk. In order for this to work, the sample library would need to be set up to stream the entire sample from disk, which I'm not sure is possible with any VI's or virtual samplers right now. Not that code for this couldn't be an update that VI developers add in a later revision, but the VI's that do support streaming from disk do still have a pretty significant amount of sample data that by design MUST load into ram since it is the start of the sample. As SSDs keep getting faster and RAM continues to get cheaper, I think we'll start to see more of the samples streamed from disk and less of them loaded in RAM, especially if RAM disks prove to be a viable solution.