"System held interrupts too long" error: Could it be HUI?

[j20056]

I am having crippling problems with PTLE 7.3.1. I use the DM4800 as a HUI control surface which works very well via USB. However, since I started using this setup, PTLE keeps complaining about the annoying "the system held interrupts for too long error", especially when I use memory and CPU intensive plugin such as BFD (although my new 8-core CPU is well at ease actually). The only thing I can do is increase the HW buffers to 256 or 512, which makes it impractical for tracking given latency. I wonder if HUI is somehow crippling PTLE? The other thing I do is that I connected the DM4800 to the Digi002 Rack via the spdif connectors for stereo out. I use the ADAT optical for 8 IO, and use the DM4800's clock and tell PT to use the external ADAT clock. it works well and I get no clicks or pops, but I wonder if it leads to my interrupts problem.
Any help greatly appreciated!

Digi002 Rack / Mac Pro 8-core / 5gb RAM / 3X 500GB 7,200 SATA DRIVES / OSX 10.4.10 / PTLE 7.3.1cs5
Plugins: BFD v1.56 (with Andy Johns samples) / Guitar Rig v2.2 / SampleTank v2.5
One HD for system / One HD for BFD and SampleTank samples / One HD for PT Sessions