[1] First, take a deep breath [2] Second, I don't think it's stalkery or controlling to go. Just don't make it sound like you're checking up on him [3] Say you got an extra long lunch and wanted to see if he wanted to grab something or something like that