Actually, in early seasons of QL (thoguh they changed it later), it was at least implied that it was much like it was here, at least described. It's the MIND going back in time. Even if it looks to us, the viewers, like the body is different. So we the viewers would be seeing the self-image.
They quickly changed it for QL of course, and I believe the prevelant explanation was that Sam's body was actually being "spatially mapped" on to the body, so space was warped such that he always fit into the body - people who looked into his eyes weren't looking up, we just saw it that way as a viewers.
Of course, then they screwed with that, by having Sam be able to walk while the person he leaped into had no legs. I liked QL, but continuity was not its strong point.
Here I don't really know how it works since they see each other as heir 'real' selves (which doesn't make any logical sense at all), but mostly it looks like it's the self-image idea. Sort of like phantom limb syndrome, I guess... phantom body syndrome, they still see what their mind expects to.