Is there a Master track with any plugins? Those would cause this issue(master track plugin latency is not compensated for, so you would be hearing late, causing you to play late). I was under the impression that most Apollo users would be working inside the Apollo "mixer" window(wondering if that would improve things for you). As for LLM, personally, I almost never use it as its often either broken, or causes other issues(like doubling in the headphones). Its my preference to set the playback buffer to 64, leave LLM off and be careful of what plugins are in play while recording(any plugins I use for mixing that exhibit more than a few samples of latency are inactive while I am tracking). Not sure if any of this helps, but maybe altering your method a little can make recording work better for you.
You might also check out LUNA(free to all Apollo owners) and see if that could be a solution
Last thing to check; look at the Delay Compensation numbers and watch for any that change when you put that track into record-ready. If that track suddenly gains a lot of compensation that makes no sense, it could be a send that has no destination(make sends inactive and see if the Delay Compensation number drops). This has happened to me a few times and it caused over 1900 samples of latency on the record-enabled track
