Sync and Wobble

[digifriend]

I have a question about recording with the Optical output/Input. Basically I am trying to make a video of myself playing rockband for later upload to youtube.
What I am doing is recording the output of my TV (LG HDTV) with a optical cable and into the Digi-002. I am recording at 24/48k. The problem comes when I do multiple takes of the same song. I have taken some of these takes and tried to line them up to see if I could do 1 take on the right side and one take on the left. But when I did this they seem to match. But I can notice that the songs seems to wobble in the stereo picture. They are both recorded from the same setup and samplerates. So is it that the digital clock on the digi is not doing something right? I want the highest quality sound so if Im some how recording bad audio like this I want to know.