They usually look at you and wonder if it's a good idea to start a chat going? Y'know, it's funny, but back home I actually started getting that. Not saying that you do. Just that... people being scared sucks. FYI, Hawaiian shirts on casual Friday doesn't really convince them you're not a cruel overlord.
...HAHA! Sorry. "If the party is boring, we can go to work." I get it, but woooooo irony.