I think at a more simple level, why exactly would anyone want to establish a data center in one of the more inhospitable corners of the globe for any reason other than heat exchange?
I came here to say exactly the same thing ... it requires a lot more calories to heat up water one degree Celsius than it does air but the heat still has to go somewhere.
An interesting question would be, how much heat do all of the world datacenters produce and how much energy would it take to raise the average temperature of the Ocean by 1 degree?
The problem I'm concerned about isn't raising the average temperature of the ocean. The problem I'm concerned about is localized heating. Marine life is far more vulnerable to rapid temperature changes than life on land. We've seen dead zones developing in the past downstream of power-plants, as the warm water discharge from the steam turbines raises the temperature of the water and reduces its oxygen carrying capability. My concern is that if we put datacenters in the ocean, they'll create localized hot-spots with much the same result.
Simply put, the ocean is a massive place with so many factors at play (like underwater currents bringing in cold water), that the heat generated by a ton of small "micro" datacenters will be pretty inconsequential.
This is precisely the sort of logic that has since the beginning of time led to us treat the ocean as a huge garbage can, and a massive amount of it has died in the past 100 years.
Seriously, watch Mission Blue. Sylvia Earle has more dive hours than anyone, ever, and thusly personally mapped a great deal of the ocean and has since watched those places die.
The data center generates the same amount of heat no matter what. It doesn't matter where it's generated. So unless you're arguing that we shouldn't have data centers, I don't understand your point.
I agree that the data center generates the same amount of heat no matter where it's placed. But like we've seen with power plants, heating water tends to have far more serious environmental consequences than heating air. If we place a full-scale datacenter in the ocean, will we end up with fish-kills and dead zones like we see downstream of powerplants from time to time?
> 50% OF US LIVE NEAR THE COAST. WHY DOESN'T OUR DATA?
It's corrosive, expensive to get things to and from it for replacement, leaks destroy the hardware, it's not close to power generation, internet access needs cables because RF doesn't penetrate water, everything is going need watercooling which is rather expensive.
It's amazing that the drive to the Northern part of the Olympia Peninsula is so long, Bellingham which looks much farther is around a half hour closer.
I suppose it would depend on what you'd qualify as "the ocean". I lived in Seattle for 4.5 years, and while I don't consider Elliot Bay[1] to be "the ocean" per se, it's pretty close - and you have all the corrosion problems and general exposure to the elements that you'd get, which I think was the original point of the comparison in this thread.
No, but it's a harder proposition to have fibre runs to a server which is in the ocean. You can throw a normal server somewhere silly and connect to it wirelessly, this is just the one remarkable exception.
Imagining they completely solve the problem of sea water, leaks, etc, it still is amazing to think that you would want to do your server maint by pulling a data center out of the ocean on a boat and replacing hard drives and the like.
The only way this makes sense to me is if there is the ability to create something akin to the cargo container as a building block of a data center, where you can have arbitrary compute and storage plug into a greater complex.
Sounds like that's exactly what they want to do: they would only pull them out of the water every 5 years to do computer replacements / maintenance. If some components fail then who cares. They wouldn't do a full rebuild for 20 years.
I worked in large data centers before and I just don't see how this can be done practically. Data centers require quite a bit of physical maintenance.
Every computer design has some element that will render a large part of the design inoperable in case of failure. Either it is a SAN head (even if you have two, the fail over can malfunction), or a switch setup.
Then there are things like failures of simultaneously purchased components (hard drives purchased at the same time, that are worked the same load will roughly fail at the time).
Cloud datacenters are not complex heterogeneous mixes of components. There's no SAN head. It's one thing multiplied + some networking gear. Even if a top of rack switch fails they're still not going to yank the box yet because the TCO will be lowered by too much maintenance at this scale. They wait for their maint interval and fix everything at once (or just upgrade the hardware).
Think of a farm of small data center pods with cloud apps. When failure in a pod exceed useful threshold, apps are migrated out to other pods and the pod is retrieved, serviced and returned to its place.
A custom made barge with dynamic positioning gear and a grabbing/coupling system to detach the pod from the subsea grid, lift it, and then re-attach it would make the servicing relatively efficient.
I could see the roundtrip time for a full hardware replacement of a pod being under an hour, conceivably under 10-15 minutes.
And I guess you wouldn't even need to use sea water as a primary cooler, just as a secondary cooler. I.e. the primary cooler flows through your datacenter and the secondary cooler cools that primary cooler. So fewer pipes are exposed to sea salt.
Yeah. Instead of trying to protect the environment from what's inside (radioactivity), you are trying to protect what's inside from the environment (sea salt)!
> everything is going need watercooling which is rather expensive
Why would everything need water cooling? I'd expect that something using water would be used to keep the air inside the unit cool, and then the cooling for the servers themselves would be ordinary air cooling.
For marine gear you either use a material which won't readily corrode, or you use sacrificial anodes which are galvanically consumed rather than something you care about.
For power you could use a small nuclear reactor, just like submarines and aircraft carriers. I've always thought it would be a fun exercise to take a decommissioned nuclear submarine and turn it into a floating datacenter.
When nuclear vessels are decommissioned they remove the reactors. Operating a naval reactor requires a constant watch by multiple highly-paid experts. They are not cost effective for electrical power generation.
Assuming they have ways around some of those issues this could work out rather well. The important thing is that it's only a research project. Microsoft's research turns out some really awesome stuff but plenty of it failed or is cut. Who knows what'll happen to this but it's a really interesting proposition!
I think they should be able to handle most of those issues, for example they may be able to use wave power for power generation. Furthermore, I don't see how RF opacity is an issue, seeing as anyone running a data center over RF is criminally insane.
You run management connections from separate computers over RF as a backup and for fail-over in event everything else fails. I mentioned that in a recent comment related to Github outage and preventing those:
Can help with certain security situations, too. Not sure how much that applies if it's underwater, though. Most infiltrations would probably turn into a denial-of-service attack effectively haha.
I tried to get it further by proposing a neutrino-based communication system. Can just send the signals straight through the planet itself to a datacenter very far away. I was told there would be both implementation problems and cost overruns with that project. Went back to default recommendations for wireless.
Because 50% of us live near the coast, not past it. Maintaining anything in close association with an ocean is painful. Everything rusts. Even the stuff they say doesn't does. Anything that moves ages at an accelerated rate. As soon as the slightest waves start, little salt crystals appear on every surface.
Of course, the ground can be reasonable cool, if you don't go too deep. The problem is, the heat will mostly stay where you put it---the ground is bad for actually moving the heat away.
I'm still not sure why one would want to place the data center at the bottom of the ocean. I would think that the disadvantage of not being able to perform maintenance for 5 years would be more significant -- can't they just create the data center near a water source and pump the water through pipes that run across a heat exchange on the backside of the servers?
Your idea is what's being done at the largest computing center in Switzerland, the CSCS. They pump water from the relatively deep Lake Lugano [1]. Europe's currently fastest supercomputer (world No. 7) is hosted there.
I assume it is for space reasons. You could put down a data center in a harbor, for example. The amount of metal that would be required to scale this model to that size, however, seems excessive.
Environmentally pumping heat into a river would not be though acceptable I would think, so the water source probably is the ocean, running pipe across beach front property might not be as doable.
> perform maintenance for 5 years would be more significant
They would be built like Google, you don't waste money doing maintenance.
I think the main issue is it's unusual and as such costs of complexity and legal come in.
The main advantage I would think that is not mentioned is mitigating the price of real estate in expensive regions like Hong Kong, New York, San Francisco etc.
I came here to say the same. Cities with expensive real estate are often the cities where latency matters even more. This is essentially a datacenter that can be dropped in the ocean, rent-free.
Regarding real estate - what entity owns the portion of the sea where the data center is located? Is there security? Curious how this would work in terms of property law.
A tsunami would have relatively little effect on a pod suspended/anchored to the sea floor. They only get violent when they reach the shallows and come onto land. It'd have more effect on the shore connection point where the data and power go into the ocean. These could be buried up to the shore though and anything above ground secured in a sealed bunker though that doesn't solve the problem of keeping the pods powered through the power outage after a tsunami.
Yeah the main factor in how much will a large wave or tsunami affect a structure at sea is how deep is the water the object is in. Out in the deep ocean the surface swell is less than a meter and the wavelength is huge (200 km or so long) so the change is relatively gentle and gets subtler as you get deeper. It's really neat.
At first I was concerned that there would be so much humidity inside of the capsule from the condensation caused by the temperature difference between the outside and the inside, but I see they've addressed that by replacing the atmosphere with Nitrogen.
I wonder if they've considered an inert fluid to immerse the computers in? If you can use something like Fluorinert, or even high-grade mineral oil, you might be able to make the vessel not required to withstand crush forces as high since the fluid in the capsule can be at the same pressures.
I do love this idea because they can start putting data centers along of submarine cables. One in the middle of the Atlantic, between London and NYC would be great for HFT traders.
I wouldn't be surprised if the NSA wouldn't love something like this too. A whole data center on cross-oceanic cables would provide a lot of infrastructure they can use to analyze traffic in real-time.
"A whole data center on cross-oceanic cables would provide a lot of infrastructure they can use to analyze traffic in real-time."
And plenty of room for submarine cat-and-mouse. Guess it'd have to be inside US waters.
I'm not sure what nitrogen has to do with humidity. I don't think anything. If humidity was the problem there are other ways to deal with it (the water doesn't come from nowhere, and once it's gone, you won't get any condensation anymore).
Sounds like (pressurized) nitrogen had more to do with increasing the air density for heat transfer. But I still don't know why they chose nitrogen.
Yeah I'm not sure what it has to do with it either, desiccants[1] are nothing new and they are the standard when it comes to humidity reduction in shipping. You might be familiar with those little packets of silica beads in foods and computer components that say "DO NOT EAT" on them.
That said, nitrogen-refrigeration is commonly used for temperature control.
> At first I was concerned that there would be so much humidity inside of the capsule from the condensation caused by the temperature difference between the outside and the inside, but I see they've addressed that by replacing the atmosphere with Nitrogen.
Whatever reason they have for using nitrogen (making the thing fireproof is one reason), avoiding condensation isn't it. You could do that just by using dry air and packing a little silica gel. It's not like you're going to introduce more moisture over time into a sealed system.
I live on Lake cochituate in natick. The fish still contain unsafe levels of toxins from a Natick Labs accident on the bank of the lake in 1984. Interesting choice of project name.
Is the white paper that they mention on the website publicly available? I think it would offer more information than the website so people would understand the rationale for this investigation.
I don't get it. If this is all about heat, I would think that putting the data center beside the ocean and then pumping seawater around to cool things would be far easier than sinking the entire kit. Even if they really really want to be underwater, I'd assume digging an artificial lake and pumping water in and out would be easier than dealing with an actual ocean.
Eh, I think it's indeed more of an intellectual curiosity than anything else, but personally I cannot support enough our encroachment on the ocean. It truly is the last frontier on the surface of the Earth that we have yet to reliably explore.
Autonomous underwater vehicles (AUVs) have a lot more potential to expand our understanding of this planet and I think it is exciting. Just because we can't go there doesn't mean we can't tap into its power.
At the risk of sounding like a BBC documentary, the oceans are the engine of this planet and they deserve to be investigated thoroughly.
This sounds like a great idea, kind of. I'd be seriously concerned about the immediate environmental impact. Otoh, volcanic vents are hotbeds of life, so maybe it'd be ok.
Came in here looking for SOMA reference. I'd imagine a data center would sort of work as its own ARK, though (Not to mention the petabytes of sensitive, personal, possibly secret documents, pictures and videos for your perusal).
Since the current post is more of an original source and is currently ranked higher on the front page, we treated that one as the dupe and merged its comments here.
A clever idea. People are wondering why such a thing might be useful, so let me advance a theory:
Latency.
Suppose you have a bunch of people somewhere, say, the US, and a bunch of other people somewhere else, say, China, and there's an ocean in between. If they need to work collaboratively on something, placing a datacenter in one country or the other yields asymmetric latency; someone has a lot more.
If you can just plop a datacenter exactly at the midpoint, everyone wins. It needn't be the biggest datacenter ever, just one that can handle the latency-sensitive tasks.
They pretty much say why in the project page: renewable energy [tidal, currents?] and cooling. The third, as you mention is latency --they want to be where the people are.
Plus, at depth, storms and typhoons don't affect things all that much, it's rather calm. So, the main threat might be from saboteurs rather than natural disasters [beside the salty environment] because beside a coast guard at the surface [which if contraband coming in is any proxy, it's pretty porous], you don't have a "police presence". So they'd have to rely heavily on monitoring systems.
I think you have the backwards -- sharks aren't attracted to copper cables.. but are attracted to undersea fiber optic cables.. because they carry high voltage power (for the undersea repeaters), which emits an electromagnetic field that attracts the sharks:
I stand corrected, forgot about the power cables I know sharks were attracted to the EM field around the old transatlantic phone and telegraph wires. Well at least it will be a good reason for an outage.
My guess is they are more concerned with latency for real-time services used by millions in big cities, where land is expensive. Think VR servers.
Quick googling yields [1] datacenter land selling for more than $1 million per acre in SV and [2] Google's requirements for datacenter placement. The first four points listed are cheap electricity, carbon neutrality, lots of water, and large parcels of land; the 215 to 1200 acres mentioned in [2] would cost $240 million to $1.5 billion at the price quoted in [1]. Sealed containers anchored to the free sea floor, running on free wave energy and cooled with free sea water would be a very clever way to satisfy those requirements while staying close to the customers.
More likely it's to reconcile latency requirements with national borders. If you want to be close to a country to offer low latency, but political or legal or tax reasons mean you don't want to be in that country, then an ocean datacenter can get you close enough.
That may be a part of it, but I also think a major component is cooling. Cooling account for 30-40% of the running cost of a datacenter, so building with access to enough water, they've essentially cut the running cost with to 2/3 of conventional datacenters.
160 comments
[ 4.7 ms ] story [ 1205 ms ] threadMaybe they already have a 'secret' unannounced private data center there.
The world generates 20,000 TWh of electricity per year, and maybe 10% is for IT. That's 7.2e18 joules of electricity / year in IT.
So it would take 780,000 years to heat by 1 C. Or 780 years to heat by 0.001 C.
Other sources of global warming are far, far worse.
Seriously, watch Mission Blue. Sylvia Earle has more dive hours than anyone, ever, and thusly personally mapped a great deal of the ocean and has since watched those places die.
They take the energy (Ie heat) from waves and power computers.
Energy (heat) in = Energy (heat) out.
If you had some electricity left over and shipped it out you'd be able to cool the oceans.
This reminds me of a book I read recently in which they put the servers on barges. I was an entertaining read, 1/1 would recommend.
It's corrosive, expensive to get things to and from it for replacement, leaks destroy the hardware, it's not close to power generation, internet access needs cables because RF doesn't penetrate water, everything is going need watercooling which is rather expensive.
It's a 3 hour drive from Seattle to the ocean.
[1] https://en.wikipedia.org/wiki/Elliott_Bay
The only way this makes sense to me is if there is the ability to create something akin to the cargo container as a building block of a data center, where you can have arbitrary compute and storage plug into a greater complex.
Every computer design has some element that will render a large part of the design inoperable in case of failure. Either it is a SAN head (even if you have two, the fail over can malfunction), or a switch setup.
Then there are things like failures of simultaneously purchased components (hard drives purchased at the same time, that are worked the same load will roughly fail at the time).
A custom made barge with dynamic positioning gear and a grabbing/coupling system to detach the pod from the subsea grid, lift it, and then re-attach it would make the servicing relatively efficient.
I could see the roundtrip time for a full hardware replacement of a pod being under an hour, conceivably under 10-15 minutes.
Which is something Google already did[1].
1: https://en.wikipedia.org/wiki/Google_Modular_Data_Center
http://www.romanconcrete.com/docs/spillway/spillway.htm
If we want to find a way to build beneath the sea that's a good place to start.
http://www.ancient-origins.net/news-history-archaeology/rese...
Why would everything need water cooling? I'd expect that something using water would be used to keep the air inside the unit cool, and then the cooling for the servers themselves would be ordinary air cooling.
And why "everything is going need watercooling which is rather expensive" ?? If i remember correctly every OVH DC Server are using it.
It might be fun but there are better things you can do with $700M.
Water cooling significantly reduces running costs which is why many DCs are switching to it. Over the long term you save money.
edit: I can't grammar
https://news.ycombinator.com/item?id=10996442
Can help with certain security situations, too. Not sure how much that applies if it's underwater, though. Most infiltrations would probably turn into a denial-of-service attack effectively haha.
Of course, the ground can be reasonable cool, if you don't go too deep. The problem is, the heat will mostly stay where you put it---the ground is bad for actually moving the heat away.
[1] http://www.cscs.ch/cscs/an_innovative_centre/cooling_system/...
[2] http://www.top500.org/lists/2015/11/
> perform maintenance for 5 years would be more significant
They would be built like Google, you don't waste money doing maintenance.
I think the main issue is it's unusual and as such costs of complexity and legal come in.
One question, what happens when there's a tsunami?
This is a particularly relevant question on the US west coast: http://www.newyorker.com/tech/elements/how-to-stay-safe-when...
https://en.wikipedia.org/wiki/Tsunami#Characteristics
I wonder if they've considered an inert fluid to immerse the computers in? If you can use something like Fluorinert, or even high-grade mineral oil, you might be able to make the vessel not required to withstand crush forces as high since the fluid in the capsule can be at the same pressures.
I do love this idea because they can start putting data centers along of submarine cables. One in the middle of the Atlantic, between London and NYC would be great for HFT traders.
I wouldn't be surprised if the NSA wouldn't love something like this too. A whole data center on cross-oceanic cables would provide a lot of infrastructure they can use to analyze traffic in real-time.
Sounds like (pressurized) nitrogen had more to do with increasing the air density for heat transfer. But I still don't know why they chose nitrogen.
That said, nitrogen-refrigeration is commonly used for temperature control.
[1] https://en.wikipedia.org/wiki/Desiccant
Whatever reason they have for using nitrogen (making the thing fireproof is one reason), avoiding condensation isn't it. You could do that just by using dry air and packing a little silica gel. It's not like you're going to introduce more moisture over time into a sealed system.
https://en.wikipedia.org/wiki/United_States_Army_Natick_Sold...
ETA: that's what I get for typing mindlessly instead of cutting and pasting.
Is there any connection or just a weird coincidence?
There is a risk of not just industrial sabotage, but international espionage & terrorism.... and of course... Tom Cruise.
https://en.wikipedia.org/wiki/Operation_Ivy_Bells
I imagine it would be better for maintenance to use the seawater to cool the coolant itself, that way they aren't pumping salt into their datacenter.
Autonomous underwater vehicles (AUVs) have a lot more potential to expand our understanding of this planet and I think it is exciting. Just because we can't go there doesn't mean we can't tap into its power.
At the risk of sounding like a BBC documentary, the oceans are the engine of this planet and they deserve to be investigated thoroughly.
https://en.wikipedia.org/wiki/Thermohaline_circulation
http://halo.wikia.com/wiki/Leona_Philpot
Watch out for that structure gel!
Since the current post is more of an original source and is currently ranked higher on the front page, we treated that one as the dupe and merged its comments here.
Latency.
Suppose you have a bunch of people somewhere, say, the US, and a bunch of other people somewhere else, say, China, and there's an ocean in between. If they need to work collaboratively on something, placing a datacenter in one country or the other yields asymmetric latency; someone has a lot more.
If you can just plop a datacenter exactly at the midpoint, everyone wins. It needn't be the biggest datacenter ever, just one that can handle the latency-sensitive tasks.
Neat project.
Plus, at depth, storms and typhoons don't affect things all that much, it's rather calm. So, the main threat might be from saboteurs rather than natural disasters [beside the salty environment] because beside a coast guard at the surface [which if contraband coming in is any proxy, it's pretty porous], you don't have a "police presence". So they'd have to rely heavily on monitoring systems.
If you need to fix something while the datacentre is in the middle of the ocean, the cost and time will be a multiple vs if it just off the coast.
http://blogs.discovermagazine.com/d-brief/2014/08/15/sharks-...
http://bgr.com/2014/08/13/google-underwater-cable-network/
Quick googling yields [1] datacenter land selling for more than $1 million per acre in SV and [2] Google's requirements for datacenter placement. The first four points listed are cheap electricity, carbon neutrality, lots of water, and large parcels of land; the 215 to 1200 acres mentioned in [2] would cost $240 million to $1.5 billion at the price quoted in [1]. Sealed containers anchored to the free sea floor, running on free wave energy and cooled with free sea water would be a very clever way to satisfy those requirements while staying close to the customers.
[1] http://www.datacenterknowledge.com/archives/2015/07/24/equin...
[2] http://www.datacenterknowledge.com/google-data-center-faq-pa...