You will understand why better when you take a look at who they say this to and who they don’t.

This is not something that generally happens to white people speaking some French in the US. It does not raise the ire of this psychology. On the other hand, they love to target brown people speaking Spanish (almost exclusively, in fact). There is, naturally, spillover where white people speaking Spanish or brown people speaking Hindi would get targeted.

As others noted, and as these examples suggest, this is an instance of xenophobia and racism. Language is being used as a proxy, really, and provides a way for these people to unleash the frustrations they have been taught, societally, to have against them. Generally speaking, these are people that will call any brown person that speaks Spanish a “Mexican” regardless of their actual place of birth, where they were raised, or ethnic heritage.

But this is just a surfacr-level analysis. The next question is why they are taught to target people with xenophobia and racism. Why are there institutions of white supremacy? Why are their institutions of anti-immigrant sentiment? How are they materially reinforced? Who gains and who loses?

At a deeper level, these social systems are maintained because they are effective forms of marginalization. In the United States, racial marginalization was honed in the context of the creation and maintenance of chattel slavery, beginning, more or less, as a reaction to the multi-racial Bacon’s Rebellion. In response, the ruling class introduced racially discriminatory policies so that the rebelling groups were divided by race, with black people receiving the worst treatment and the white people (the label being invented for the purposes of these kinds of policies) being told they would receive a better deal (though it was only marginally so and they were still massively mistreated). This same basic play had been repeated and built upon for hundreds of years in the United States. It was used to maintain chattel slavery, Jim Crow, and modern anti-blackness. It was used to prevent Chinese immigrant laborers from becoming full citizens and becoming a stronger political influence in Western states.

It was and is used to maintain the labor underclass of the United States, which also brings us to xenophobia more specifically. The United States functions by ensuring there is a large pool of exploitable labor in the form of undocumented immigrants. It does this at the behest of the ruling class - the owners of businesses - who have much more power to dictate wages and working conditions when it comes to this labor underclass. They make more money and have more control, basically. But this pissed off and pisses off the labor over class, as they have lost these jobs (or sometimes are merely told they lost them even if they never worked them). To deflect blame away from the ruling class for imposing these working conditions wages, the ruling class instead drives focus against the labor underclass itself, as if working that job for poor pay and bad conditions their fault. This cudgel should remind you of Bacon’s Rebellion again: it divides up workers so that rather than struggle together they fight amongst themselves on the basis of race or national origin. The business owners are pleased, having a docile workforce to exploit.

So while racism and xenophobia are themselves horrific and what is behind the "Speak English!’ crowd, it is really just an expression of the society created by this system that, by its very nature , pits workers against business owners while giving business owners outsized power (they are the ruling class, after all).

Another important element to this is imperialism and how imperialist countries carefully control immigration (it used to be basically open borders not that long ago). But I’ll leave that for any follow-up questions you might have.