Ask HN: What are the benefits and drawbacks of working remotely?

In reference to the book James Dale Davidson and William Rees-Mogg, what are the upsides and downsides to working remotely for a big tech company in a foreign country? How likely are you to be successful if you start a startup with everyone working remotely today?

I see one of the downsides is that you have the potential to missing out on interacting with smart, world-class people on a daily basis. However, some would argue that this is a downside of not living in the Valley, Cambridge, etc. in general.