Listen man, we (as individuals) can’t care for or help everybody. Connect with the ones you care about and don’t harm everyone else. The fact that I don’t care about you, doesn’t mean we can’t coexist or even help each other.

Looking form another angle, why not wanting to socialize in stranger small talk is bad? Why I am expected to accommodate? Why can’t we just enjoy the silence in this hypothetical situation?

I see this sentiment more often than not. Me, as the less social party, is expected to move out of my comfort zone, but the person trying get me into a conversation isn’t expected do the same and just keep to themselves.