Knowing the exact gear you are using might help get better advice. In any case, LLM is something that(for me) would only be used for a last-minute recording in a huge session that has forced me to use a high buffer setting(which would mean tons of latency without LLM).
For general tracking of players, skip LLM and set the playback buffer to 64 or 128. For several years, I ran a Nashville studio that tracked lots of full bands(drums, bass, multiple guitars, keys, vocalist) and the headphone system provided 6 stereo headphone mixes(so in most cases, each player got their own mix). That was done with AUX sends on all tracks in a session. Once you get mixes dialed in to where players are happy, save that session(without media) as a session template. That way, each time you create a new session(from that template), everyone's headphone mix is good right from the start(I rarely spent more than 2 minutes adjusting headphone mixes)
All the above requires is enough physical outputs and some way of powering the headphones. In my case, I used an ART HeadAmp Pro 6, but in an unusual way. It has 1 set of stereo inputs(on XLR's) that normally would feed all of the 6 outputs. Each of the 6 sections also includes a TRS input and a knob to balance between the main input(the XLR's) and the AUX input(the TRS). By setting those balance knobs all the way to the AUX side(on all but #1), I could feed separate mixes into those AUX inputs. So, HP1 was fed by the XLR inputs and 2 thru 6 were fed by separate signals to their respective AUX inputs. There are several headphone amps that can do the same(some with fewer possible outputs) but the beauty of the ART unit is that all the inputs are on the back(to keep cabling neat and hidden). As the engineer, I could patch my headphones into each of the 6 sections on the front panel so I could hear the exact same mix that went to each player
