If they do, they're either not aware of that being the reason, or they don't tell me. Though I have sometimes sensed that certain readers' criticisms of my stories might be based on cultural differences. I think a lot of readers were put off by Firsts, for instance, because of the very distinct sexual experiences I described of a boy who would be, in many readers' opinions, too young to have those experiences, coupled with language that's 'inappropriate' for someone of his age group. I think that attitudes like that are perhaps more common in American readers. It took a really long time before anyone reviewed it at all, which was unusual.
I do set most of my stories in Britain, and I intentionally choose not to use expressions and words that are more international, because I don't like underestimating my readers. I'm pretty sure that most readers are smarter than that, that they can figure it out from context and maybe learn something from it. I also think it adds to authenticity to be uncompromising in this respect. I hope that at least some readers, like you, are drawn to a different cultural setting and find it interesting to read.