Is this realistic when it's clearly really useful to have fire mages in your army, and other, not quite friendly cultures have no objection against doing just that?
I would say it depends on what goes along with using this magic. For example, in my writing there's a type of magic that deals with the weather. It's highly dangerous and has been used to win wars before, but consequently completely screwed up the climate in the area where it was used in the long term. Now it's seen as a very bad thing, and the few people who still practice it in a more contained form are pretty much anathema-- which makes sense, because nobody wants their climate (and consequently their food supply) ruined.
So I would ask, is there a reason that they don't use fire (or whatever type) magic? As others have pointed out, cultures do things for illogical reasons all the time-- but having a logical reason just might lend a bit more weight to it, and also perhaps explain how this culture has survived when the neighboring army is using fire magic (if the consequences of using fire magic have harmed that culture and taken away some of their edge, for example).