Yes, I've been also considering that. If the powers that brought us here are in any way similar to the energies that allow Kagome-sama to travel between her time and ours, then there would be no reason at all for time to stop in our own world while we are here.