I guess it depends on what you mean by "unexpressed." I think there can be a tendency to think of people outside a marriage as this open field of undemanding sexual partners, but they're still people with feelings. If I were to act on a same-sex attraction, I wouldn't just be expressing myself, I'd be in an interaction with a third person, whom I'd then have to consider also. And I'm happy (and, ok, fully extended) in the relationship I have. I don't want to deal with another partner.
Edit: Or to rephrase that: I'm living in a house with three extroverts and two pet rats. I'm not having an affair with a woman who will be ANOTHER person I have to interact with and you can't make me
But again, I'm free to "express myself" by talking about my feelings as much as I like. So have that outlet and it makes life pretty easy.