[Felix]

For anyone coming across this someday.. I found a work around that works good enough for me. Create a Multi-Output device in Audio Midi Setup. Deselect Speakers or Headphones as desired using the "Use" check boxes. You can also unplug / plug in headphones without causing it to force close the session... but *you must have headphones unchecked before unplugging...
Also, I don't see a way to include the mic as an input but this doesn't matter for me.
One more thing, when in Multi output, the volume keys can not adjust the volume. You need to go to the System Sound Preferences and select either headphones or speakers to assign the volume control to whichever you desire.