Avid Pro Audio Community

Avid Pro Audio Community

How to Join & Post  •  Community Terms of Use  •  Help Us Help You

Knowledge Base Search  •  Community Search  •  Learn & Support


Avid Home Page

Go Back   Avid Pro Audio Community > Pro Tools Software > AAX Plug-ins

Closed Thread
 
Thread Tools Search this Thread Display Modes
  #1  
Old 10-25-2011, 08:53 AM
reichman's Avatar
reichman reichman is offline
Member
 
Join Date: Aug 2000
Location: New York
Posts: 2,411
Default A Programmer's Perspective on the AAX Transition + Q&A

This was posted on the Daw-Mac list recently. Frank Filipanits generously gave me permission to share it here. I think his (2) posts are required reading for everyone:

Transition to AAX: A Real Programmer's Perspective

Quote:
There's a lot of almost-correct or sort-of-right information floating around regarding AAX, RTAS, and TDM. I'd like to try to set the record straight about a few things.

Unified AAX architecture:
The AAX spec strives to unify development so that the same algorithm can run natively or with DSP hardware acceleration on the new "HDX" TI-based floating point DSP cards, but there will remain a distinction between AAX Native and AAX DSP, with some (mostly third party) plug-ins only supporting one or the other. While Avid appears to intend to offer all of their plugs in one version only that runs on both, it is quite possible to create a plug-in that is either AAX Native only, or AAX DSP only.

Bit depth vs. internal processing:
There seems to be much confusion regarding how bit widths apply to different aspects of plug-ins.

TDM passes audio around using a 24-bit fixed-point pipe. Internal processing can be 24 bits, 48 bits, 56 bits or more. I lost track of what the mixer uses but it is very, very large.

RTAS passes audio around using a 32-bit floating-point pipe. Internal processing can be 32 bits, 64 bits, 80 bits, or more. I don't know what the native mix engine uses but again it's very very large.

AAX also uses a 32-bit floating-point pipe to pass audio around, for both Native and DSP. Internal processing can be 32 bits, 64 bits, 80 bits, or more.

Application word size:
When folks talk about an OS or APPLICATION being 64-bit, what that refers to primarily is the maximum size of a memory pointer, which places limits on the maximum amount of memory an application can use. It has nothing to do with the size of the data streams, or how efficiently the CPU can handle 32 vs. 64 bit float operations. A 32-bit application is limited to directly addressing 4Gb of memory.

Whether a plug-in (or host) is "64-bit ready" or "runs as 64-bit" has nothing to do with the word size of the audio pipes between plug-ins, which may well remain 32-bit float. Even if the audio pipes between plug-ins are 64-bit, the internal calculations may still use 32 bit operations since they are considerably faster than 64-bit operations (regardless of whether the host app or plug-in is "32-bit" or "64-bit").

Why a whole new format?
When the TDM spec was written in the early 1990's, the state of the art was 68000-based Quadras running System 7 and hard drives in the tens of Mb. It had provisions for black and white monitors. RTAS was added as a branch of that spec, and Windows compatibility was grafted in. Quite honestly, it is amazing how long that spec lasted -- through the transition from 68k to PPC, NuBus to PCI, OS9 to OSX, PPC to Intel. But facing continuing aggressive evolution of OSX and fundamental structural incompatibilities with operation in a 64-bit host application environment (among other concerns), it was time to rewrite the plug-in spec to meet the needs and capabilities of today's and tomorrow's systems.

Some question why Avid didn't simply adopt AU or VST -- a few simple reasons are these: AU is a mac-only spec, and VST simply doesn't have the power and flexibility to do everything RTAS/TDM and AAX are capable of.

Believe me, I am less than thrilled that the 17 years of expertise, techniques, tools, and libraries I built around the RTAS/TDM spec are now as useful as COBOL. But AAX was a necessary move on Avid's part, and not one they took lightly.

Personally, I think it is remarkable that Avid made PT10 a transitional product - one that runs BOTH the old (RTAS/TDM) and new (AAX) formats (with the one limitation that TDM and AAX DSP can't coexist simultaneously). It would have been much easier to simply make a clean break with the old.

"It's just a scam to make more money":
Some people seem to think third parties are dancing with delight at the opportunity to force people to repurchase their plugs in a new format -- nothing could be further from the truth. This is a HUGE pain in the ass for third parties, and plug-in manufacturers are stuck between a rock and a hard place: they have to spend considerable money and effort to port their software to the new spec, because no one is going to continue to buy the old format. But on day one, the market size for the new format is exactly zero. And existing customers often expect to get the new version for free or close to it. It's an unpopular, very costly exercise for third party companies.

It's not unlike having an expectation that since you bought an 8-track of "Led Zep II" for $5 back in 1969, that Atlantic Records would have provided you over the years a new copy -- at no cost to you -- in LP format, then cassette, then CD, then DVD-Audio, then mp3, then SACD, then Blu-Ray....

So take a deep breath. Remember that the TDM//RTAS system you have today works just as well and is just as capable as it was last week, and no one is going to force you to stop using it. PT10+ and AAX have some real advantages moving forward, and when those advantages overcome the cost disadvantages for you, bite the bullet and upgrade. Just like you did going from 68k to PPC, or NuBus to PCI, or OS9 to OSX. And give your plug-in companies the benefit of the doubt; this change is hard and expensive for them too.

-Frank

Frank Filipanits
Cool Stuff Labs, Incorporated
  #2  
Old 10-25-2011, 08:54 AM
reichman's Avatar
reichman reichman is offline
Member
 
Join Date: Aug 2000
Location: New York
Posts: 2,411
Default Re: Transition to AAX: A Real Programmer's Perspective

AAX DSP/Native vs. TDM/RTAS

Quote:
Posted by: "Nathaniel Reichman" <> on Sun Oct 23, 2011 6:51 am (PDT)
I was at AES on Friday, and despite some controversy over high upgrade prices, and developers dragging their feet, it was completely evident that Avid has put a tremendous effort into Pro Tools 10. I used systems there, and they were snappy and powerful. The plug-in menus on the demo rig showed that there are separate "DSP" and "Native" menus, similar to RTAS and TDM now. However, Avid product specialists where quick to point out that the underlying sonic code is identical, so hopefully developers won't have a hard time making both versions of the new AAX plug-ins (would love to hear from Frank Filipanitis here).
Quote:
Posted by: "Ben Cox" <> on Sun Oct 23, 2011 6:56 am (PDT)
What I want to know is, does it also make it so that developers won't have a hard time making plugins that only work in Native or DSP modes, and thus charging separately for them? I'd love it if _all_ AAX plugins would run in Native or DSP, and it was not possible for a plugin vendor to charge separately for DSP vs. Native.
Quote:

RTAS and TDM require two completely separate implementations due to the fundamental differences in architecture; RTAS is written in C or C++ in a floating-point environment, TDM is written in 56k assembly language in a fixed-point environment. While the GUI and "glue" code could be shared between the two, the actual processing code had to be written completely separately for TDM and RTAS. This made ensuring bit-for-bit matching and synchronization of changes difficult.

HDX changes the DSP hardware to a floating-point architecture and AAX enables the possibility of writing a single implementation of the algorithm in C that compiles to both the native and hardware-accelerated versions. Note I said "enables the possibility". The realities of writing an algorithm for the constrained architecture and feature set of the HDX hardware and TI DSP makes it significantly harder than simply writing a plug-in to run on the host. There are many differences to factor, ranging from the straightforward (buffer size) to the complex (subtle differences in mathematical operations btw Intel and TI). And while simple algorithms can indeed run identical code on Native and DSP, many will still require some conditional code that is different on Native vs. DSP.

In general however, it should be the case that if the algorithm runs as AAX DSP it can run as AAX Native with little additional effort (though not necessarily optimized for speed). The converse, however, is not true... while it is MUCH more straightforward to adapt an AAX Native algorithm to AAX DSP than it was to adapt RTAS to TDM, there is still a considerable amount of effort and additional testing involved. So I expect we may see some companies elect to do plugs that run as AAX Native but not AAX DSP, at least initially.

I hope that we won't see stratification in the market, with different price points for AAX Native and AAX DSP+Native; I think it is much simpler to have a single SKU and pricepoint for both. But the fact remains that making a plug run on the DSP requires additional cost, effort, and support -- so it either gets baked into the price of a unified (DSP+Native) plug, or companies can attempt to recoup that additional cost only from those customers using the DSP side, while keeping the Native side a little cheaper.

I should also mention that attaining bit-for-bit matching between Native and DSP may no longer be practical for many algorithms.... expect to see manufacturers changing that bullet point to "matches to -96 dB" or "difference below -110dB" or something to that effect.

One last note regarding "developers dragging their feet"... the AAX spec was rolled out on a VERY aggressive schedule. Some vendors were able to refactor their plugs for AAX quickly, others were not. Avid provided a tremendous amount of support and encouragement to get as many third-parties ready for the roll-out as possible, but the amount of effort required varies tremendously from plug-in to plug-in. AAX also obsoleted a graphics framework used by some plug-ins, so vendors whose products were based on that not only have to refactor the processing code for HDX but also completely redo the UI code. Some companies are also better positioned to endure the painful parts of being on the bleeding edge, while others need to wait for things to settle out and stabilize before committing resources.

Again, while it is going to be a painful transition for many people -- users and developers alike -- I do believe that this is a good and necessary move for the future.

-Frank

Frank Filipanits
Cool Stuff Labs, Incorporated
  #3  
Old 10-25-2011, 08:25 PM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

First, let me thank Frank for making such an in-depth post about what AAX really is and why we we had to tackle a new plug-in platform. I was tempted not to reply at all because his comments are so comprehensive, but I decided to post to validate his comments as well as provide any additional information I could provide about AAX, both the reality of what we've shipped as well as some of the back story that drove us in this direction.

AAX is truly just a plug-in specification, as Frank mentions. Early on, we were really just trying to add HDX support to the legacy plug-in spec. (I'll admit, back then it had a different name, and a MUCH different architecture to just accomplish that goal) But when we started to look at the feature requests the plug-in developers were asking for to create amazing new plug-ins for all of you guys and our internal roadmap of products we'd like to create, that old plan just didn't make a lot of sense. This infrastructure had been growing by bolting on features for 17 years and it just wasn't easy to add even seemingly simple features to the SDK (Software Development Kit) anymore. Not to mention all of the 64 bit issues that come along with 17 years of development...

So, about a year ago we decided to take a more aggressive stab at setting ourselves and our third party plug-in developers up for the future. We looked at the backlog of feature requests, we looked hard at our road-map, we spent a lot of time studying all the intricacies of our current formats (RTAS and TDM), and we started designing AAX. We had several really talented teams working on this from multiple angles to make sure it satisfied all of our requirements and was simple. (And I don't mean limited in features, I mean simple to develop for, which hopefully in the long run leads to fewer bugs.) We had to preserve preset compatibility between formats. We had to make all of the new types swap out seamlessly with TDM and RTAS when you round trip your sessions. We had to make the porting process for plug-in developers as straightforward as possible. We needed to support complex engine interactions with both real time plug-ins and non-realtime (AudioSuite). We needed to make sure that new plugs-in mixed in with legacy plug-ins (and not break those, by the way). The list goes on and on and I just can't tell you all how proud I am of the teams that worked on this project.

There's the story, or enough of it at least... So, where are we now? Truthfully. We've shipped AAX, which solves all of the legacy issues that were preventing us from getting to 64 bit, which I could go on and on about. It also provides a much simpler path for plug-in developers to create DSP accelerated plug-ins, which is important in a world where DSP isn't always needed and developers may not be as motivated to do a TON of extra work to develop for that platform. (It's still extra work for them, so please be understanding) We have a design that is modular that will enable us to do some pretty darn cool features/products in the future. (Trust me, we're just getting started with this...) We've incorporated a ton of feedback from plug-in developers that will enable them to make cool new products. In fact, I think we closed out 14 of the top developer feature requests the day we shipped AAX. I'm personally looking forward to what these guys can do with the new platform.

And finally, with the incredible support of our third party plug-in developers, we're launching PT10 with shipping third party AAX plug-ins (Native and DSP) from some companies as well as firm commitments from a ton of others. All of the feedback they provided during their development process was critical in the refinement of AAX. Hopefully, they all feel like this platform is as much theirs as it is ours. With their continued efforts and feedback, we'll be able to make it even better.

We're still polishing up the last details on HDX, so I might not be able to answer questions quickly, but I will follow this thread and answer any additional questions as quickly and candidly I can.

Thanks again Frank for dispelling the myths and giving your very candid opinions on all of this.

Dave Tremblay
Sr Engineering Manager - DSP Group
Avid
  #4  
Old 10-25-2011, 08:39 PM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by TLarvenz View Post
One interesting note is the discussion about 32 bit vs. 64 bit for the application and saying that it really doesn't apply to tdm or rtas processing but rather the memory addressing. Doesn't that actually go against the argument put forth by Avid that the app HAD to be 32 bit to support legacy hardware? Thanks very much for the perspective and thorough explanation. It's refreshing to hear facts vs. beliefs. ;-)
I understand your point here, but the addressable memory issue (32 bit or 64 bit) affects lots of things in software like ours.

1) The communication channel between the Host CPU and the TDM Cards.
2) The drivers all have to be fixed for 64 bit.
3) All plug-ins would have to be rebuilt to communicate with the DSPs.
4) All plug-ins would have to be rebuilt to remove non-64 bit safe graphics frameworks on OSX.

Actually, the list is quite a bit longer, but suffice to say it's a ton of work for both us and for 3Ps.

Dave
  #5  
Old 10-26-2011, 08:36 AM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by jeremiahmoore View Post
My question is: Besides the reliability and latency factors (a big issue for the high-end market, where downtime is way more expensive than hardware) are there other technical decisions that went into continuing on the DSP path vs. abandoning it for a native-only system?
It is definitely true that Native systems are very powerful these days, which is why we released the HD Native product. It is quite possible that Native will work great for many people and meet their needs, but DSP based systems have a few advantages as you mention in your question. Low latency is a very difficult problem to solve in Native systems because the OS (windows or OSX) just isn't designed to have ultra high priority threads that are the dominant priority. The newer audios specs like Core Audio are pretty good, and we can run 32 sample buffer lengths to get latency down, but it comes at a price. The system becomes much less reliable as you crank down your buffer settings. (thats why we ask you to turn them back up if you start glitching :) ) So, in a way, a Native system is a compromise between low latency and reliability. Our DSP systems are not. They are extremely low latency and rock solid because all those chips do is process audio.

Additionally, an HDX system still has all the benefits of Native in that you can run a ton of AAX Native plug-ins up on the host AND get additional power (and low latency) down on the DSP cards.

So, that all sounds like marketing speak, and I apologize for that, but it's true. Many of our customers demand a high reliability and/or low latency solution, and these cards are designed to do that.

I will add a fun little technical tidbit which might make people think a little. So, Native cores have all this power. We've all seen the numbers of the ridiculous FLOPS and other specs, but one thing to keep in mind is that modern processors all have a cache because accessing memory is kind of slow. The cache isn't that big and worse yet, that cache (or much of it) is shared between all of your cores. In the audio world where you have a ton of data from your audio tracks to your plug-in states, the cache becomes a rather precious resource, and cache contention can become a major bottle neck that hobbles the real performance of your native cores. This can be mitigated somewhat by larger buffer sizes, but as you shrink that down, this cache problem starts to become pretty major as every process has to hit memory directly. Now, what if each of your individual cores had it's own memory and it's own cache and didn't have any contention with other cores? That would be cool, right? Well, look at the HDX card. It has 18 individual processors with their own memory and cache. No contention. These chips, while seemingly less powerful if you look at the raw numbers can be run much closer to the bleeding edge.

Dave
  #6  
Old 10-26-2011, 08:45 AM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by rockridge View Post
So RTAS and AAX have identical sound quality?
RTAS and AAX Native have the same audio precision coming into and going out of the plug-in, 32 bit floating point.

AAX DSP also has 32 bit floating point data path, so it shares that higher precision, which wasn't true with TDM which was 24 bit fixed point.

Because the data path and processing code can be identical between AAX Native and AAX DSP, you can use the full precision inside your plug-in without making a compromise to have your plug-in sound the same between Native and DSP.

For an individual plug-in, I guess the answer is yes, but it all depends on how the plug-ins are written. For the system as a whole, things got a whole lot better because all plug-ins are 32 bit floating point precision and won't likely have internal clips or clipping between the plug-ins for that matter.

Some of our plug-ins use 64 bit double precision internally for higher sound quality in the processing code, but I believe 32 bit floating point is more than adequate for the data paths between plug-ins and it makes the system perform much better because you don't have to move twice as much data (memory) around.
  #7  
Old 10-26-2011, 08:53 AM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by LukeHoward View Post
Probably. But going 64-bit at the application level also implies other changes: for example, the Carbon UI framework on the Mac is no longer available. Better for all to address both issues in one update.
It also has to be removed from all plug-ins, which we've done in the AAX infrastructure, but plug-in developers might still have to migrate to newer graphics frameworks. By releasing PT10 with AAX support we are trying to give developers more time to do this work without putting customers in a position where they have to choose between their existing plug-ins and the new ones coming out.

Transitions like this are very difficult to do, but we're convinced that this transition plan was the right thing to do for our customers and third party plug-in partners.

Dave
  #8  
Old 10-26-2011, 12:17 PM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by jeremiahmoore View Post
1) in HDX, running RTAS or HDX-Native plug-ins on auxes or masters, will we incur a similar voice and latency cost as we see now? i.e. running RTAS reverb on an aux, without ADC on, is like inserting a predelay, as the audio streams out of the DSP mixer, into host processing, and back into the DSP mixer.
If I understand you correctly, yes this is still true. That has more to do with the fact that the host system can't run at the same latency as our cards, so we have to buffer up data to move it back into the host and buffer it up to move it back. If the host OS could run a 1 or 4 sample buffer sizes, we might not have to do that. :)

Quote:
Originally Posted by jeremiahmoore View Post
and 2) native performance: clock vs. cores

Last year I did a bunch of research into native processing, and whether clock speed or number of cores was the better advantage.

The answer came down to: Clock speed is more important for now.

i.e. a 6-core 3.33GHz Westmere would perform better (presently) than a 12-core 2.93GHz, for Pro Tools anyway.

Any thoughts on what to expect when we move into the next generation of app? Might PT be fully multithreaded? Perhaps it's speculation
Your clocks vs. cores results are actually a little surprising to me, but maybe I can give some additional insight into how we use the cores and such. The Native engine is multi-threaded and not just for processing, but for things like control updates and stuff. Whether you can efficiently use all of the cores might really come down to your session though. If you run plug-ins in serial, like on a single track from insert to insert, multiple cores can't buy you much because we can't process those plug-in in parallel. If you have multiple tracks, now we could potentially processes in parallel, right? In most cases, yes, but there are some other dependency rules that come into play when you use particular bussing schemes and such. I remember being in a discussion a while back where we came up with a worst case session bussing scheme which would prevent the system from ever using any other cores... So, in some sessions, you might see the load balancing work quite well. In others, maybe not. I'd guess that other Native DAWs have similar limitations unless they add latency to break the dependencies, but I don't really know for sure.

Since multi-core is obviously becoming the future, we are and will continue to be looking at even better ways we can intelligently use all of your cores.

Dave
  #9  
Old 10-26-2011, 05:05 PM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by sunburst79 View Post
I'm curious if the HDX cards and the 11Rack use the same DSP chips and AAX programing code.
Good question. The chips are not identical, but they are similar. I think it is fair to say that we learned a lot when developing 11R that eventually made it into the designs for HDX and AAX. But 11R was developed before AAX, so it isn't built on top of AAX or anything like that.

Dave
  #10  
Old 10-27-2011, 04:50 PM
DaveTremblay DaveTremblay is offline
Member
 
Join Date: Oct 2011
Location: Colorado
Posts: 191
Default Re: Transition to AAX: A Real Programmer's Perspective

Quote:
Originally Posted by b1daly View Post
1) is my understanding of how dither is used in the TDM signal path correct?
2) is it different in PT 9 native mixer concept ( i.e. dither not needed between plugins)
3) is the mix engine different in PT 10, even running RTAS plugins? (meaning old mixes might sound better!)
4) Do the AAX plugins offer sonic advantages over RTAS?
1) I don't have enough direct technical knowledge of legacy TDM systems to answer authoritatively.

2) I would say that dither is not needed between plug-ins in a 32 bit floating point signal path. Floating point has fundamentally different numerical representation that makes use of dither awkward. Dither is really intended when you are reducing the bit depth to a final integer representation, like right before you go to your converters. Since all plug-ins are now 32 bit signal path, DSP and Native, you shouldn't need dither between any plug-ins, only before you hit your converters.

3) Good question, and not simple to answer. I'm not positive, but I believe the native mixer stayed the same. For the HDX Mixer, it has obviously been rewritten for the new DSPs and now uses 64 bit floating point accumulators internally, which is a significant improvement.

4) Not necessarily, the signal quality in and out of the plug-in are both 32 bit floating point. I will say that in our efforts to move over to AAX, we took some opportunities to modernize pieces of our code, internal to the plug-ins, which could/will make them sound better and perform better. That said, there is a benefit that there aren't all these conversions back and forth between 24 bit for TDM and 32 bit float for RTAS.

Dave
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Transition to AAX: A Real Programmer's Perspective reichman Pro Tools 10 289 03-02-2013 10:59 AM
perspective control 24 youbringmesuffering ICON & C|24 10 01-31-2010 09:27 AM
PC to Mac....looking for a little perspective Studio66 003, Mbox 2, Digi 002, original Mbox, Digi 001 (Mac) 15 10-09-2008 03:50 PM
Reason / PTLE (different perspective) basis3708 003, Mbox 2, Digi 002, original Mbox, Digi 001 (Mac) 14 09-06-2001 07:58 AM
Female perspective Doc 003, Mbox 2, Digi 002, original Mbox, Digi 001 (Mac) 6 07-16-2001 09:21 AM


All times are GMT -7. The time now is 12:48 AM.


Powered by: vBulletin, Copyright ©2000 - 2008, Jelsoft Enterprises Limited. Forum Hosted By: URLJet.com