This is a tough one, Brits & Aussies pretty much understand each other aside from the odd bit of slang here and there & I think our cousins over the pond get the general drift.
Personally I tend to use a more universal approach, I use for example cell phone instead of mobile, it doesn’t detract from the story, but everyone knows exactly what it is. I always put ‘u’ in words, the Americans after all are the ones who can’t spell.
Pants is an awkward one, especially as they mean totally different items of clothing from one side of the Atlantic to the other. But I think we’re all intelligent enough to realise someone is not going to go to dinner in their underwear!
Now, if you’re talking local dialects & slang then that’s something else altogether. Whilst this would be ok in small doses, if you use too much then you risk alienating your universal readers who would have no clue what’s being said. A prime example of this is a story I found on Nifty some months ago. Based in New York it was littered with so much of it I didn’t bother to go beyond chapter 1, because frankly the flow was constantly interrupted by terms I needed subtitles to understand.
I think most of all; you have to go with what you’re most comfortable with. I’ve got used to my bilingual approach, but even I slip up sometimes... as my editor points out quite frequently. Lol