Don't set a separate output path(you are using 3&4). Then anything solo'd should mute everything else
The only reason to give a separate set of outputs is if you WANT to be able to hear both signals at the same time
(something I do, but in a different way, with multiple sets of speakers)