It is not only this world. My own is much the same - even the gods have stories and legends, and those grow more convoluted and exaggerated over time in the interests of making adventures seem far more harrowing than they already were. The monsters become far more vile, and the heroes vastly more heroic.
Unfortunately it is something that is unlikely to be escaped regardless of where you find yourself. I suppose you must simply realize that what they say is not important. You are not a monster, regardless of what fiction they create that may indicate otherwise.