"Yeah, well. After hearing these stories for so long your mind sort of goes along some well established paths. Sorry." So his mind leaped towards certain ideas when he heard traveling stories. Sue him. Better yet, blame Jack. It was easier.
"We've got a pet pterodactyl. I, um. She came through the rift one fine day and I used her to secure a job with Torchwood three." The rest of the circumstances were conveniently shoved under the proverbial bed. Rose didn't need to hear about his betrayals and failures.