It's not just small to large objects; it's a mutual attraction. Any two things that've got 'mass' pull at each other, but the one with the stronger attracting force is the 'bigger' or 'heavier' of the two. So between something like a planet and something like a human being, there's no contest as to who ends up doing more moving.
Oh yeah, gotta point out-- there's a difference between 'mass' and 'weight'. What you normally think of as the 'weight' of something is really affected by two things; its 'mass', and the amount of gravity being exerted on it.
When you're in zero-gravity conditions like true space, things don't 'weigh' anything at all-- that's why they float around. But they've still got mass. So if you lobbed a rock at someone even in zero-G it'd still give them a decent bruise.