Randomly ran into this problem a few weeks ago, Pro Tools would get stuck on the "waiting for video engine to start" progress bar, and continue waiting until cancel is eventually clicked. No specific cause that I could trace, it was working one day, then booting up the next day it was not. Updating to 11.2.0 did not provide a fix either.
Reinstalling PT 11 seemed to fix it for me (at least for now…). Deleted the files listed on Avid's
manual uninstall PT 11 page, but did not delete IO settings or plugins. They do leave out a few other folders that I deleted:
/Users/username/Documents/Pro Tools
/Users/username/Library/Preferences/Avid
It seems this bug is sporadic and affects different machines differently, so I hope this will help at least a few people.