Because when the power does cut, how long is it going to take to get those generators up and running? It's not like the whole place can run on a battery backup in the meantime.
Not if you get good generators and the systems on standby. On an oil rig our engines always have their lube oil prelube running and heaters on. We can go from blackout to online in under a minute. Like most other things it's an engineering problem and how much money you're willing to spend to solve the problem.
They're not. This story isn't very good. The point it makes is somewhat valid, but the reporter's lack of understanding on the details means the way the story is written doesn't provide much insight.
Likewise, I'm guessing "utilization" is measured by taking a loadavg, which is... not exactly really a complete measure of utilization.
Well they could be..
Demand Side Response is a mechanism where utility companies pay energy consumers to stop using grid power at peak times. Effectively a company is treated as a power generator but instead of selling power to the grid they just consume less either by turning stuff off or relying on on-site power generation for a while.
It's popular in lots of industries but there seems to be relatively little (publicised) adoption in the data center business yet.
The growing datacenter trend is to build dedicated co-generation facilities (typically, with the power company footing half the bill because you're doing them a favor and taking load off of the local grid).
In the US, generator power is considered dirty, and only meant to be run on during emergencies. Running generators during not-emergency times will typically get you a nice call/visit from the state's environmental agency courtesy of your complaining neighbors. Said agency will ask you to curtail your usage to no more than X hours per month.
In the UK, DSR consists of you entering into a contract with National Grid, one or four years in advance, to produce energy during peak times (not just "reduce usage") at an auctioned rate, but pay a stiff penalty if you fail to deliver. Plus, you're not allowed to participate in back to back years.
They could be doing peak-shaving. Sometimes when your electricity usage heads into a new price tier it becomes cheaper to generate power yourself by burning diesel.
Another possibility is demand-response. Some utilities also offer incentives to customers who can voluntarily take some of their load off the grid during times of high demand.
I'm fairly comfortable with thinking about power consumption, thinking in terms of work, power or energy. This journalist mixes those concepts in confusing contexts. Also with data amounts. Can journalists please just use powers of ten in these contexts? Small explanation up front then use 10^n.
"A server is a sort of bulked-up desktop computer, minus a screen and keyboard, that contains chips to process data."
Yes that is exactly what it is a bulked up desktop PC. Completely ignore these highly specialised coprocessors sitting in the same box. They do nothing but cost a fortune.
Besides the stupid article I can say there are some very old, inefficient servers in the datacenter on my last visit. In our aisle I've walked past dozens of yellowing beige boxes, Netburst era Compaqs, and old Cisco routers. All putting out an incredible amount of heat.
Also this author knows very little about how a modern data centre runs. Just because no one is using it doesn't mean the servers aren't working. I've timed our backup and integrity checks to run during the few hours of the week when usage is light.
Disneyland doesn't shutdown when the last guest leaves. Neither do datacenters.
This is an oversimplification, though I don't doubt that there is a lot of infrastucture crud especially in large companies, which is a problem that /should/ be fixed. But to measure utilization without any context is naive:
- Most applications out there are probably not cpu-bound, so measuring cpu utilization is worthless. Should categorize services as either cpu, disk, network, or memory bound.
- Even world-wide demand will vary a lot throughout daily and weekly cycles, so over-provisioning is necessary for system stability
- Need over-provisioning to handle peaks due to soccer games, earthquakes, Christmas, etc.
All that said, I work at a company that (I get the impression maybe this is unusual these days) has historically pushed the limits of per-machine utilization through:
- service isolation
- minimal (pair-wise) redundancy (which is cheating, I know)
- simple service architecture
- leveraging mobile client processing power wherever possible
among other techniques, while using the latest and greatest bare metal boxes.. and we'd never hit anywhere near 100% cpu utilization. The most we've ever driven even a a cpu-bound app (the front end) is around 60%, which is dangerous, and we've suffered a couple outages as a result.
19 comments
[ 3.0 ms ] story [ 56.6 ms ] threadBackup uninterrupted power is not typically done by keeping generators online 24/7.
So every (serious) datacenter has rows and rows of batteries to cover the lag time between power loss and generator start-of-service.
Likewise, I'm guessing "utilization" is measured by taking a loadavg, which is... not exactly really a complete measure of utilization.
It's popular in lots of industries but there seems to be relatively little (publicised) adoption in the data center business yet.
In the US, generator power is considered dirty, and only meant to be run on during emergencies. Running generators during not-emergency times will typically get you a nice call/visit from the state's environmental agency courtesy of your complaining neighbors. Said agency will ask you to curtail your usage to no more than X hours per month.
In the UK, DSR consists of you entering into a contract with National Grid, one or four years in advance, to produce energy during peak times (not just "reduce usage") at an auctioned rate, but pay a stiff penalty if you fail to deliver. Plus, you're not allowed to participate in back to back years.
Another possibility is demand-response. Some utilities also offer incentives to customers who can voluntarily take some of their load off the grid during times of high demand.
"Online companies typically run their facilities at maximum capacity around the clock, whatever the demand."
"Even running electricity at full throttle has not been enough to satisfy the industry."
Wow, this is bad.
Yes that is exactly what it is a bulked up desktop PC. Completely ignore these highly specialised coprocessors sitting in the same box. They do nothing but cost a fortune.
Besides the stupid article I can say there are some very old, inefficient servers in the datacenter on my last visit. In our aisle I've walked past dozens of yellowing beige boxes, Netburst era Compaqs, and old Cisco routers. All putting out an incredible amount of heat.
Also this author knows very little about how a modern data centre runs. Just because no one is using it doesn't mean the servers aren't working. I've timed our backup and integrity checks to run during the few hours of the week when usage is light.
Disneyland doesn't shutdown when the last guest leaves. Neither do datacenters.
- Most applications out there are probably not cpu-bound, so measuring cpu utilization is worthless. Should categorize services as either cpu, disk, network, or memory bound.
- Even world-wide demand will vary a lot throughout daily and weekly cycles, so over-provisioning is necessary for system stability
- Need over-provisioning to handle peaks due to soccer games, earthquakes, Christmas, etc.
All that said, I work at a company that (I get the impression maybe this is unusual these days) has historically pushed the limits of per-machine utilization through:
- service isolation
- minimal (pair-wise) redundancy (which is cheating, I know)
- simple service architecture
- leveraging mobile client processing power wherever possible
among other techniques, while using the latest and greatest bare metal boxes.. and we'd never hit anywhere near 100% cpu utilization. The most we've ever driven even a a cpu-bound app (the front end) is around 60%, which is dangerous, and we've suffered a couple outages as a result.