That is right. [There's really no difference in their dress and attire.] The robes are gender neutral, and can fit comfortably for both. [Do you want some Hakuryuu village robes, Kaya.]
The clothing outside of my village is much more diverse I have noticed, too. From town to town, but from economic standpoints as well.