I track replies to a mod post, too, and I just checked the latest ones I've gotten and, yeah, it doesn't say which journal I tracked it from.
I guess what I'd suggest is, if you can pull up the post and just switch between journals (if you have lj-login, that should be easy) until you see the little track symbol with the check next to it, showing you that you're tracking it from that journal.
If you can't find the particular post you want to untrack, you can go here: http://www.insanejournal.com/manage/subscriptions/ from all of your journals, and uncheck any posts that you no longer wish to track.