informal benchmark, interesting buffer results

[huzzam]

Hi everyone, I just ran a simple Pro Tools benchmark on my two systems, and noticed a very odd result. Namely, the performance at buffer sizes of 64 and 1024 were *really* close. I really expected the system to be able to do a lot more with a maxed-out buffer, but it just didn't happen that way.

In a 24-bit 48kHz session, I made a ~52 second mono recording (of myself playing a pretty dense line on clarinet), and added a single instance of eleven lite with default settings. Adjusted clip gain & channel volume. No other plugins on inserts/sends/master. Duplicated this track a bunch of times, and if the system could play cleanly to the end of the recording, it passed. If not, fail. Count the tracks it can pass with.

Systems are as in my signature, one M1 Macbook Air with 16gb ram, and one hackintosh i9-9900k with 16gb ram. Both on latest Ventura 13.6.1 and PT 2023.9. Dynamic plugin processing on, Optimize at low buffer sizes off. Turbo Boost on on the i9. RME Fireface UCX II with latest (DriverKit) driver v4.08. Sessions were stored on internal fast ssd in both cases, but PT's meter showed very low disk activity, so the test was really CPU-bound.

Here are the results:
M1 @64 samples: 43
M1 @1024 samples: 44
i9 @64 samples: 129
i9 @1024 samples: 132

Doesn't it seem strange that the performance at small & large buffer are so similar? I'd really expected a significant increase in track count with larger buffer sizes, like double or something. Instead it's a ~2.5% difference on both machines.

Do you all see significant performances improvements in your workflows with a larger buffer? Am I looking at the wrong thing?

Also, though not at all the main point: I didn't expect the i9 to slaughter the M1 so handily... synthetic benchmarks show them much more closely matched. But the i9 had exactly (!) 3x as many tracks before failure. Guess I'll hang on to it a few more years!

food for idle thought
~peter in athens

NB I did also play around with some other buffer sizes, though not thoroughly. The results were similar, just very slightly better or worse.

[Darryl Ramm]

When benchmarking things you have to be very careful that the benchmark test is relevant... and often they just are not for a variety of reasons. And here you hit a trap, so great of you to suspect something was not right.

Pro Tools, like many DAWs, uses split/dual buffers. When you are playing back audio in Pro Tools it uses the disk playback buffer (sometimes called the high latency buffer or Avid folks talked about the high latency domain) and when you are recording/monitoring a live input Pro Tools uses the H/W buffer.

Disk buffer is fixed at 1024 at lower sample rates and 2048 samples at higher rates (96 kHz and above IIRC). The H/W Buffer is the thing you get to set the size of.

Of course if you are mixing this may be a good test, but if you are tracking it may not.

To do tests that exercise the small H/W buffer performance of Pro Tools you typically route out an output or bunch of outputs into live hardware inputs on the interface.

As for comparing the i9 and M1. You are comparing a fast desktop processor with a low-end of the Apple product range M1 laptop.

M1 MacBook Air only has 4 p-cores and 8GB of memory. Jammed into a thermally constrained fanless compact enclosure. It's just not a laptop for running Pro Tools on and to some extent makes this comparison not so interesting.

The i9-9900k *is* a reasonably fast processor. 8 physical cores. And here you have 16GB in that vs 8 in the Air. You don't state clock rate, turbo boost on will likely help it some.

I think the MacBook Air did pretty well, it would be interesting to say watch the thermals, and make sure it is not running out of memory during testing.

The main thing is is this workload representative of your work... other traps for folks is just running lots of instances of the same plugins and getting better CPU core scaling than they would with one or two bottleneck plugins, esp with plugins in deep chains. Or just a VI that is fragile/causes problems for the whole systems, if you need to use that VI then... other benchmarks may not mean anything.

[huzzam]

Interesting info, thanks Darryl!


So the old advice (or maybe i've just been confused for years)—to use a low buffer for low latency while tracking, and a high buffer while mixing—doesn't really hold true, then? Since during playback the disk playback buffer will always be used anyway?


I might try another round where i'm recording, just to see how they fare. Though I've already spent too much time benchmarking for today.

[Darryl Ramm]

Originally Posted by huzzam

Interesting info, thanks Darryl!


So the old advice (or maybe i've just been confused for years)—to use a low buffer for low latency while tracking, and a high buffer while mixing—doesn't really hold true, then? Since during playback the disk playback buffer will always be used anyway?

Yes to some extent that went away when dual-buffers were added, but still its not uncommon for folks do stuff where some tracks will be in the low-latency domain and you may need to increase H/W Buffer size while mixing.

[huzzam]

Originally Posted by Darryl Ramm

Yes to some extent that went away when dual-buffers were added, but still its not uncommon for folks do stuff where some tracks will be in the low-latency domain and you may need to increase H/W Buffer size while mixing.

Aha right. I guess if using any outboard gear, for example, right? Since you have to record that back in.

[Darryl Ramm]

Yes.