Ask HN: Which co-location providers would you recommend in the US?

81 points by devneelpatel ↗ HN
OneUptime (https://oneuptime.com) is looking for a co-location provider in the US for a full rack to begin with.

Havent found good ones so far. Do you use them, if so which one would you recommend?

54 comments

[ 4.2 ms ] story [ 101 ms ] thread
The biggest factor here is location, you can have unlimited choice if you’re in or near a major market (New York, Chicago, LA, etc), which you didn’t mention. If you’re in smaller markets, often times the larger providers still have some sort of presence but prices will be out of control. I would probably find a local managed service provider and see if they have any rack-level offerings, before going with the behemoths like Equinix or Digital Realty because they will bleed you dry if you’re not careful.

I have used Deft, Cyxtera and a handful of smaller providers in the past (and of course Eqx and DLR), but those met a particular project’s needs well and may or may not fit yours.

Where would you prefer to have this set up? What are your growth plans, if any? Will you be looking to grow into other PoPs as your product gains traction? Do you need access to a more diverse mix of transit providers? Are you bringing your own addresses? Are you looking for concierge-/fully-managed level service or are you going to manage all the aspects of the hardware yourself? Lots of questions to get to the bottom to before an ideal provider could be found.

(comment deleted)
Flexential is near us and other major markets. I know our local one just did their monthly UPS battery and generator test. Benefit to Flex is that they have interconnects to their nearby DCs and use Megaport for connecting to Cloud providers. They have a lot of various options and for us their rack pricing was pretty good.
This is the vendor that cloudflare blames for their latest outage, right?

Not saying they are to blame, but it seems like there were electrical issues in the affected data center. Could be something for op to look out for.

If the price is good, having a fallback or a few is much safer than reliance on a single dc. So much so that it matters less if one of dcs burns or whatever. Cloudflare issues are theirs - they can’t blame the dc.
You mean Cloudflare can't at all blame Flexential for:

1) not informing CF of unplanned maintenance on electrical

2) not informing CF that they had failed over to generator due to a ground fault on their PGE transformer

3) not informing CF that ALL their generators failed and were on backup batteries

4) not informing CF that power had completely failed within the DC

5) notifying CF that anything at all had happened for 48 minutes (which was after all the UPS batteries had died)

Yes, this is a one sided analysis. I haven't seen the Flexential one (is there one?).

https://blog.cloudflare.com/post-mortem-on-cloudflare-contro...

It's buck-passing; the responsibility to keep Cloudflare services available to users lies with Cloudflare, not Flexential. A well-designed distributed system is able to withstand an unplanned DC failure without the user noticing. They can certainly note that Flexential's failure was a causal factor in their outage, but if they do, they're also admitting to inadequate engineering practices.
Yep. Basically what I meant as well. Thanks for clarifying it even more.
Flexential can absolutely be blamed on all of that, and if I were their customer I'd be massively pissed off too.

However, CF massively failed at the whole "entire DC goes down" mitigation thing. This time it was Flexential screwing up power, but it could've also been a fire, a flood, a tornado, an earthquake...

Once you get to CFs scale, you simply cannot hide behind your vendors anymore. CF has gotten big enough that we can fully expect them to just deal with it.

It's why I mentioned our maintenance notifications. I know our DC has lost utility power and not been an issue. Those types of outages you need a plan for at any DC. I'm sure if someone took a dump truck to our DC it would shutdown in a different way. Our plan is using our AWS direct connect to roll everything over to AWS.
Don't fall for the HE $400/mo rack deal is my advice.
I'm curious about this. Can you elaborate on what went wrong?
Not the above commenter, but when we got a quote from them (over a year ago), the pricing seemed really good for a full rack.

But looking through the fine details, it turned out to only offer 7 amps (power) for the entire rack.

While that's not totally useless, it's pretty close to being so.

They seemed to be looking for suckers to offload useless facility space onto or something.

i see they're getting less generous. it used to be 15A for that price. not awesome but slightly less useless.
7A doesn't sound right for a full rack. Each rack has its own breaker and those are 15A at the smallest. I suspect the 7A was for a half-rack deal, though those are extremely rare for HE.
It could be 7A per leg with two power legs so when one trips you are not at 30A on a 15A breaker.
Hmmm, it was a while ago. Maybe it was a half rack thing.

That being said, it turned out to be in one of their oldest data centres that they hadn't been able to upgrade.

So the available power and cooling for each rack wasn't even close to meeting modern server needs.

I'd also like to hear about this. But it's worth pointing out, that's just the rack space, bandwidth is priced separately. But, HE seems to me to offer very competitive and high quality bandwidth.

Also, I think most situations don't need to go from 0-servers to 44U of servers. (Although VC backed bet-the-farm situations may like that route)

the deal used to include a gigabit unmetered drop as well.

but as others have noted, the power is where they get you.

i don't think it's a completely terrible deal actually.

What are your goals?

If budget is a factor, there are local/regional providers you can find good deals with. E.g. I've had good experiences with https://www.opticfusion.com/ in Seattle.

If low-latency across the US is a factor then...doing that with one will be sub-optimal, however I'd either start in a specific market you want to chase (e.g. LA or NYC), or go more central like Chicago.

If connectivity flexibility and peering are key for you then you might consider locating in a "carrier hotel". The best locations for those vary by geography (e.g. in Chicago that would be at Equinix, but in LA it's Coresite). You'll pay more, but can readily access the peering exchange.

Figuring out your goals is key here. There are lots of providers all over the spectrum, so narrowing the options is key.

Location is clearly important. Where do you want your datacenter(s) and do you want all locations managed by the same company or different companies.

Electrical reliability: some sites are fed from multiple substations, have a well maintained battery room, redundant generators, well tested transfer switches... and some don't. Automatic transfer switches tend to be a SPoF though, and a major provider, doing everything right, is likely to have an outage at one of their facilities due to a transfer switch failure once every 3-10 years, and afterwards, there will be an unavoidable scheduled maintenance to replace it (unless it had to be replaced during the outage). Maybe there's a new way to install transfer switches redundantly though?

HVAC reliability: HVAC needs electricity too, so see the previous one, but servers don't like to be hot, what are their plans for when HVAC equipment fails.

Network reliability: if you're at a carrier neutral site, you'll get fiber to your rack from your upstreams and then it's all up to you or them. If your provider is your transit, do they deliver redundant connections (lacp), do they run redundant connections all the way to their border router, redundant connections to their upstreams, do they have multiple upstreams, will they pull fiber from an upstream carrier to your rack. Do they participate in the local peering exchange? Also, have a capacity in mind.

Remote hands / physical access is important. If you have a lockable rack, do all the racks have the same key? IPMI can avoid a lot of physical time on the machines, but not all of it.

In addition to all of these, there's a what they say, and what's actually the case, and if they'll let you inspect / audit.

There's no wrong answers on these, I have my personal hosting at a facility with probably wishy washy answers to everything and I have low trust, but it's cheap and I don't need nine nines. Other people need all the reliability. Or something in between.

How good is the pricing at Optic Fusion? I’m always wary of “contact us for a quote”.
I don't recall exactly (it's been a while since I was handling the colo contracts directly). I just remember that it was really good value.

You're probably going to have to contact folks for quotes for most colo providers. They're doing "enterprise sales", so they want to talk to you, see what the needs and opportunities are, then quote you based on your needs.

Do you have any location preferences? What are your requirements for power, network, and remote hands support?
Thank you for all your recommendations. Latency is not a concern because we will use that rack to store logs, metics and other telemetry data for our customers.

I believe major concern is power + bandwidth and good support because we wont be able to drive / fly there at moments notice.

Is there a reason you want to run your own server? It seems like a lot of headache for this use case.
Not the OP, but the reason I've looked at co-loing my own server is to host off-site backups in the 400-2000TB range. With the right colo price, it can easily be 1-2 orders of magnitude cheaper than cloud storage over a 5-year timeframe.
That sounds about right. My back-of-the-envelope is $2000/mo to colo (not including up-front hardware cost) and $12,000/mo for 2000TB from rsync.net (although you might be able to get a bulk discount).
Without knowing where you are, it's hard to say. But things to look out for: What is included in the monthly fee? Is power extra? What about a second power strip for the backup power supplies? Is network included? How much? Often they charge per network port connection, in which case how fat is each connection? Is it billed by 95th percentile or raw bits (you want 95th percentile)? Are there access fees (some places charge you every time you enter the facility)?

What is their connectivity? Which Tier 1 providers are they directly connected to? What is the total bandwidth connected to the datacenter? Do they have direct connects to other providers? Direct connects to AWS/Azure/GCP?

I'd build a spreadsheet with the answers to all of these questions. Color the cells green/yellow/red based on how you feel about the answers.

You won't be able to compare directly between them but it should give you a good idea.

> Without knowing where you are, it's hard to say.

This is certainly a relevant question for some businesses (e.g. US-based ones); but I would think not for the majority.

Say that I have a SaaS company that serves many US customers, but which currently has no physical presence in the US. (This isn't a hypothetical; my company is headquartered in Canada, and most of the employees are Canadian — some are remote, but we currently don't employ any Americans.)

And say that I want to run a low-latency base-load "POP" to serve US customers, ideally just starting with one DC that's placed as centrally (in Internet latency terms) to as much of the contiguous US as possible.

Say that I currently do this by just using IaaS — AWS and GCP obviously have great connectivity — but for base load, the egress is getting too expensive, and they also don't offer machines that are vertically resourced nearly as highly as modern technology makes possible. (We do realtime OLAP stuff; the ability to have 100+ cores, multiple TBs of memory, and — critically! — more than 9TB of NVMe per machine, is all very useful to us.)

Say that I don't expect our base-load needs to grow very quickly, so we don't expect to have high needs for physical access. And that we won't own very much hardware at first — maybe 20 (very expensive) machines.

My thinking, here, is that in such cases, it would actually be a waste of money to have even a single full-time ops staffer; that in such a case, it would be lower OpEx to just have some kind of retainer contract with an "ops as a service" company willing to send someone out to the DC on our behalf for installations, removals, upgrades, and troubleshooting. For anything that doesn't require putting hands on the hardware, we can fix it with BMC access (or, if we can't reach the BMC, then a phone call to the DC with strong suspicion that one of their switches is faulting upstream of us.)

Or, equivalently, there's no reason that we wouldn't work with a DC that's willing to do a "managed bare metal" offering with customer-supplied hardware — that's just an "ops as a service" offering provided by the DC staff, after all.

Or hey, why do we want to get into the business of eventually having to resell old server parts if we don't have to? We don't actually own the hardware yet; the plan to purchase the hardware is part of the greater plan to decrease base-load TCO. If we can find a provider who will lease us managed bare metal, in exactly the configurations we want, at reasonable prices, then that would make even more sense, wouldn't it? They'd also maintain the pool of spares for us, so we don't have to buy and sit on those. (Of course, this is starting to creep closer to having a middle-man who can extract rents by charging on a value-add basis, like IaaS providers do. Some managed-bare-metal leasing providers are like this; but I've found that there are others that operate on a cost-plus basis — probably because they're overgrown DC + managed-ops-staff offerings that haven't yet realized they've repositioned themselves to be middle-men.)

This was the line of thinking we went down, and in the end it actually lead us back out of the US entirely. Since we're Canadian, we found some financial advantages in going with a managed bare-metal offering from a company with a Canadian presence that we could pay in CAD. So we went with OVH Canada (i.e. their Montreal DC.) AFAICT, the network map from Montreal looks like "a fat 40ms-long pipe over to N. Virginia, and then peering equivalent to hosting in US-East-1 from there." Works great for peering with GCP US-Central-1, where we still have our elastic load.

For a company like mine, would you do something different?

I interpreted co-location provider as "I will provide hardware and ops". But you're right, OP wasn't clear enough on the ask. They could mean managed bare metal, they could me "I send you my servers and you do the hands on part".

The ask was too light in details, so I made assumptions.

Missing from your list of questions, a really important one:

- does the colo state think you having servers there is a business presence, and you owe them tax

It's pretty settled precedent that leasing or owning tangible property in a state or county creates a tax nexus. But what form those taxes take—and the tax rate and available credits—does vary.

Expect to pay property taxes (usually annually) on the value of the equipment in all but 11 states[0].

Independent of property tax, most also expect you to either pay sales tax when you buy the equipment in-state, or pay use tax when you bring it into the state (possibly with a credit for sales tax already paid elsewhere). Some states have passed exemptions specifically for data center equipment, e.g. Pennsylvania[1].

If your home state is one of ~23 that's a member of "streamlined sales tax"[2], you don't have to fill out a separate sales tax return for each member state in which you have a nexus. (But thanks to South Dakota v. Wayfair[3], you may be required to collect sales tax in other states anyways, regardless of the nexus created by your data center presence.)

[0]: See Table 1 here for a list: https://taxfoundation.org/research/all/state/states-moving-a...

[1]: https://www.revenue.pa.gov/IncentivesCreditsPrograms/Compute...

[2]: https://www.streamlinedsalestax.org/

[3]: https://en.wikipedia.org/wiki/South_Dakota_v._Wayfair%2C_Inc...

I would think you would be first thinking about location (it's a big country).

We use https://www.edgeconnex.com/ (originally meer.net, then SVColo, before the EdgeConnex acquisition) with transit from Wave (was Layer42, was meer.net). Looking at their current web site though I'm not sure if they are marketing single rack colo now. Possibly we're grandfathered in.

Well, piggy backing on this, if anyone has recommendations of providers in Los Angeles, CA I'm all ears.

Assuming I'll need to go with a reseller because I need a half rack. Redundant 20A circuits would be nice, but not required.

I was very happy with Quadranet when I lived there. We just migrated to OpenColo in Santa Clara. We were with Quadranet from 2005-2022. They're not shiny, but they're competent and we had great latency and very few issues.
A shortcut is to pick one from this list: https://aws.amazon.com/directconnect/locations/

If AWS is a customer, you can trust that it's a good well-connected facility, and you have the option of a cheap low-latency direct fiber connection to AWS within the same data center should you ever need it. You'll have your pick of Internet carriers in any of these carrier-neutral facilities. Ask the colocation facility to recommend a local contractor for your installation work and use their Remote Hands service for small maintenance work.

Wholesaleinternet.com may have a good deal, but I would ask about whatever remote hands services they offer.
I highly just recommending renting machines. Hetzner or OVH are low cost providers. There is very little reason to have to buy your own machine and do the co-location thing.

If you need more machines, put them in any office that has a fiber connection, like it is cheap to get a 3Gbps connection these days. Do more than one location if you need some of them online. In different cities if needed. Basically you want to avoid being in the same weather location/powergrid connection.

I've done this before and it works really well to cut costs.

(comment deleted)
There is a very deep rabbit hole you can go down with the webhostingtalk.com forums, which have a lot of options and deep history and conversation on this subject.
It's tough. I had an excellent colo provider that gave a nice discount because of all the open source stuff my servers do, that didn't have me go through layers of trained monkeys to communicate with someone technical, that communicated well through email rather than through billing. Unfortunately, they closed very recently and I'm just now looking for a new home for that collection of servers.

Another colo I wish I could still recommend is Turnkey Internet (http://www.turnkeyinternet.com). They were a smaller facility - quite literally - where everyone was friendly and personable, with no fuss, no issues, and very good pricing. They've now been bought by ColoHouse, and I have no idea how that will affect things there.

If the acquisition hasn't changed the people and practices in their Latham, NY facility, then I'd recommend them.

I'm interested to check out some of the recommendations here for my own machines :)

On another note, it is interesting to see how many people go out of their way to try to discourage colocation in favor of renting / using service providers. I've always believed that movement to be successful marketing, both to make money for the large providers, and if you're the conspiratorial type, perhaps to make as much data as possible accessible to government TLA agencies. But why do people think it's OK to answer someone's question with, "don't do that, because I have an opinion about something I've never done"? Hmmm...