When you write a DAW you can make it handle plugins any way you want.
But Pro Tools and most other DAWs load the plugins at startup. They are instantiated at a low level within the system, and really become a part of Pro Tools. There are stability arguments both ways here, and likely deep technical reasons why this would be hard, but not impossible to do.
For many workflows that also just may be a good choice. Folks likely don’t want to customize plugins that are available per session, they want to set stuff up once how they like and be done with it. And once plugins are loaded you get to do nice things like be able to invoke them on the fly.
It’s not a crazy idea but it’s just not something Pro Tools or most other DAWs were designed to do. And for my money there is a lot more important issues facing Avid/Pro Tools than worry about things like this. Including because Avid has been greatly downsized over the years and has a small fraction of the technical resources it has had in the past. I would rather see effort spent on obvious work needed on core AAX performance (Windows users should not be needing to play with Windows priority/affinity to avoid CPU errors), increasing HDX voice counts, migration to Apple/ARM for Mac users, and ongoing bug fixes, etc.