Speaking to someone doesn't make you friends, even if you speak 'often'. Being invited to a party once doesn't make you friends, especially if you cannot name the person who invited you. If you refer to some as 'That' and all insistence is on their end, you are not friends.
This is not how friends work nor how friendships are formed.