Ask HN: Is English in the workplace important?

I just left a contract (in the US) where english was rarely spoken in the office (mostly when dealing with managers). I found this to be odd, and slightly disturbing. It made me uncomfortable, am I wrong to feel that way?