As soon as the boy was out of the man's sight Rufio had taken off at a sprint to escape. He didn't trust the stranger not to follow, and kept his head down and eyes on the ground in front of him to avoid downed trees and branches. When the new man stepped out from behind the tree Rufio wasn't even looking, too distracted and winded to stop himself. He slammed into the man with a yelp, feeling a cold pair of hands wrap around his arms, and immediately acted on instinct while he tried to catch up mentally which what had just happened.
He spun, well used to worming out of the grasp of pirates, indians, and bigger boys. It really didn't have anything to do with strength, just quickness and a willing to turn and move awkwardly, sudden movements that were easy to pull off no matter how confusing the situation.
Aiming a kick at the man's middle (more as a deterrent to try and follow than an actual attack) as he scrabbled away Rufio stumbled, reaching out and managing to pull his sword free just as he tripped. He hit the ground hard on his butt, but swung the sword up to be between him and this new stranger, scrabbling away and looking back the way he'd come, also intent on seeing if the magical person had followed.
"What the hell!" he yelled, heart pounding so hard it almost hurt. He was still gasping for air from the run, and now the fear that had been invoked by the man he'd just fled had erupted into flat out terror. Were these old men just popping up out of the ground to get him?
"Stay away from me!" he spat, looking around just in case anybody else came out to try to grab him. The sword, normally held as steadily and skillfully as any seasoned veteran, wavered as he trembled violently. He grabbed the handle with both hands to steady the blade, backing himself into a tree and pushing into it. He couldn't stop himself from shaking, but he made an enormous effort to hide the movement by pressing into the trunk, feet digging into the soil from the effort. "All of you, stay away, or I'll run you through!"
It was just too much. The storm, the destruction of his home, then the stranger who could trap him with magic, and now this guy coming out of nowhere and grabbing him. It was overwhelming the young Pan's nerves and reverting him back to a scared child, instead of a warrior-emulating preteen he worked so hard to be.