Exactly. "The hand is quicker than the eye" is the phrase, I think. If you're too slow, you can slip up and they can catch on. But, if you do it too fast you can mess up the trick. Like making it "reappear".
[dips his hand forward and before the pebble and roll off it, catches it between two fingers again, holding it up] You can do this like I did with you, near the person to distract them further because people tend to get more distracted the closer you are, or from away from them.