Launch HN: Fly.io (YC W20) – Deploy app servers close to your users

It's closer to Google's Cloud Run, Heroku, or Fargate since we don't constrain you to a framework and the restrictions that come with it.

There is some overlap with CloudFlare Workers. The big, practical difference is that you can put apps on fly.io that abuse CPUs, write to disk, accept TCP traffic, etc.

Most people we've worked with want to run run apps they've already written (or open source like https://github.com/h2non/imaginary).

I think we fit that category. We definitely have features/philosophies that match the Wikipedia article!

We don't (yet) because we really loathe slow response times and booting peoples' apps is absurdly slow sometimes. Our #1 feature request for Firecracker is suspend / snapshot functionality so we can do that.

Instead, we give everyone $10/mo of credits and have a really tiny VM that you can run full time for $2.67/mo.

There's one in the Newark area:

`ewr Parsippany, NJ (US)`

from https://fly.io/docs/regions/

The city list does look random, but it's actually the simplest cities to build out with physical servers + anycast.

We _tend_ to do better than AWS on latency to your apps, and from upstate New York you'd probably be connecting to New Jersey. I would be Virginia is quicker than Ohio for you most of the time too.

We're not AWS, but: 1) we own physical servers and 2) we're profitable 3) have big public companies as customers.

"I won't use anything from smaller or new companies or Google in case it goes away". That's silly, how is this the top comment? Are people that conservative in tech of all places? This is a docker runtime. If it goes away you'll be up and running again without delay on AWS or anywhere else, you just won't have the edge performance characteristics.

Depending on the region your infrastructure is located in, AWS doesn't own the datacenter.

For example in the Paris (eu-west-3) region, their availability zones are operated on hardware owned and managed by Telehouse, Interxion and Equinix.

We got someone else to rack them. Customer apps are VMs on top of the physical servers we lease. At scale we'll build our own datacenters, but at scale we can get not-me to do that.

No timeline yet but we're eager to get into India and South America when we can

We actually noticed people asking about gRPC on Hacker News, especially on Cloud Run posts:

https://news.ycombinator.com/item?id=19612577

It's usually small teams, individual devs who want gRPC. Even if it's within a large company, it's almost always one technical person.

Right now we're best suited for app servers, databases won't (yet) run very well on fly.io. We are trying really hard to focus on what we have because it's so valuable but we love DBs so much we might end up trying to "solve" them soon.

I’m experimenting with this, they have Redis at every edge, with a way (SELECT 2) to send commands to all edges with eventual consistency. No RDBMS yet, said they’re looking at CockroachDB.

I’m running a single central Postgres server on Heroku and planning to use the Redis edges to cache.

Do you mean it would get expensive to run multiple applications that listen on a single TCP port?

It _might_, if you need a bunch of ipv4 addresses it'll add up fast. But you could always put your own router app in place to accept that port on one IP, find the right ipv6, and forward connections along.

That's awesome, your use case is perfect for region scoped persistent storage.

We're testing persistent storage privately with a few customers now and the results are exciting. My favorite is using minio as a private global s3 for caching.

What are you using for the index storage engine?

We attempted C++ about a year ago, but I was never confident in our ability to clean up memory allocations (we had leaks) or avoid undefined behavior (we had segfaults).

I definitely feel more confident about our Rust code. It's no silver bullet, but it prevents a lot of unsoundness with its compile-time guarantees.

I can't really compare to C++, but it's easy to write new code or refactor old code. It took some time to get there, though.

All in all, I would recommend Rust wholeheartedly. The ecosystem is growing and getting more mature every week. The community is very helpful in general, especially the tokio folks.

How do you currently host your Docker container at edge nodes?

(It's a facetious question: unless you're a $100B company you're not doing anything of the sort.)

> Takes "the cloud is expensive" to a whole other level.

What do you mean by this? It seems pretty much on par with the expensive cloud pricing of the big players.

> Does it mean the backhaul is private and not tunneling through the public internet?

Backhaul runs only through the encrypted tunnel. The Wireguard connection itself _can_ go over the public internet, but the data within the tunnel is encrypted and never exposed.

> I use Cloudflare Workers and I find that at times they load-balance the traffic away from the nearest location [0][1] to some location half-way around the world adding up to 8x to the usual latency we'd rather not have. I understand the point of not running an app in all locations esp for low traffic or cold apps, but do you also "load-balance" away the traffic to data-centers with higher capacity?

This is actually a few different problems. Anycast can be confusing and sometimes you'll see weird internet routes, we've seen people from Michigan get routed to Tokyo for some reason. This is especially bad when you have hundreds of locations announcing an IP block.

Server capacity is a slightly different issue. We put apps where we see the most "users" (based on connection volumes). If we get a spike that fills up a region and can't put your app there, we'll put it in the next nearest region, which I think is what you want!

CDNs are notorious for forcing traffic to their cheapest locations, which they can do because they're pretty opaque. We probably couldn't get away with that even if we wanted to.

> Frequently? Are these server-routers running in more locations than data centers that run apps?

We run routers + apps in all the regions we're in, but it's somewhat common to see apps with VMs in, say, 3 regions. This happens when they don't get enough traffic to run in every region (based on the scaling settings), or occasionally when they have _so much_ traffic in a few regions all their VMs get migrated there.

> Interesting, and if you're okay sharing more-- is it that the anycast setup and routing that took time, or figuring out networking wrt the app/containers?

Anycast was a giant pain to get going right, then Wireguard + backhaul were tricky (we use a tool called autowire to maintain wireguard settings across all the servers). The actual container networking was pretty simple since we started with ipv6. When you have more IP addresses than atoms in the universe you can be a little inefficient with them. :)

(Also I owe you an email, I will absolutely respond to you and I'm sorry it's taken so long)

Hey, I'm the tech lead of Workers. I don't want to intrude too much on this thread, but just wanted to say: we don't do any special load-balancing for Workers requests; they are treated the same as any other Cloudflare request. We use Anycast routing (where all our datacenters advertise the same IP addresses), which has a lot of benefits, but occasionally produces weird routes. Often this relates to specific ISPs having unusual routing logic that, for whatever reason, doesn't choose the shortest route. We put a lot of effort into tracking these down and fixing them (if the ISP is willing to cooperate). We do sometimes re-route a fraction of traffic away from an overloaded datacenter by having it stop advertising some IPs, but if the internet is working as it should, that traffic should end up going to the next-closest datacenter, not around the world. When you see requests going around the world, feel free to file a support request and tell us about your ISP so we can try to track down the problem and fix it.

Predominantly, middleman, markdown and a splendid web designer.

We actually have servers in North America, Europe (including Frankfurt), and Asia Pacific. The complete list is here: https://fly.io/docs/regions/#discovering-your-applications-r...

Building an anycast network is expensive. That's part of what we want to make accessible to devs. There are a couple of companies (like Packet, and possibly Vultr) you can lease servers from that will handle anycast. These tend to get you into the same ~16 regions, expanding past those can be difficult and even more expensive. That's what we're working on now.

This is my favorite Hacker News comment of all time.

This basically bumps the amount of behavior you can stuff into the edge by several orders of magnitude. Previously, Lambda@Edge will like, maybe validate a JWT. Now, you can put like half your app in there. It sounds like a tiny incremental change but it's big enough that it simply redefines what's even possible in a POP.

They originally had (and seem to still have) the same JS runtime at the edge to give you a smart CDN/reverse proxy.

They just updated it from being JS only to being able to run any Docker image. Cloudflare gives you a persistent key/value store and Fly provides a non-persistent Redis cache.

You don’t have to move your entire app but there are plenty of use-cases where you can move more logic to the edge.

There's a whole bunch of use cases people have today that don't require a database. Here's a few that I'm excited about:

  - image, video, audio processing near consumers
  - game or video chat servers running near the centroid of  people in a session
  - server side rendering of single page js apps
  - route users to regional data centers for compliance
  - graphql stitching / caching (we do this!)
  - pass through cache for s3, or just minio as a global s3 (we do this!)
  - buildkite / GitHub Action agents (we do this!)
  - tensorflow prediction / DDoS & bot detection
  - load balance between spot instances on AWS/GCP
  - TLS termination for custom domains
  - authentication proxy / api gateway (we do this!)
  - IoT stream processing near devices

CloudFlare Workers is a fantastic serverless function runtime, but we're a layer lower than that. You can actually build it on top of fly.

edit: formatting

That's always a concern, but not something we focus on. DX is a big deal. That's why Heroku and CF Workers are so popular despite everything the big guys offer.

It's a bit of a pivot but still in the same space. Here's a few reasons why we landed on this:

- Our JS apps require customers to write new code to solve problems. That’s a tough sell for companies with existing code they need to make fast.

- The more people used JS apps the more unexpected things they wanted to do. Like tensorflow at the edge, audio and video encoding, game servers, etc. No way we could support any of that without moving down the stack.

- Reimplementing the service worker api was a slog we didn’t want to continue. Deno is fantastic and we’d rather just run those apps than compete with it

That's something we're certainly thinking about, but we need to get compute right first!

> That way we could build entire application on edge without buying into the FaaS movement

So you want to be able to upload your code and have someone manage the infrastructure and datastore.

Isn't that the definition of FaaS?

The "simplest" gains come from adding an in memory cache, we include Redis for this and some apps work really well just leaving the DB where it is, caching aggressively, and running close to users: https://fly.io/docs/redis/

Read only replicas are a great first step for most applications. I'd probably do caching first, then replicas (which are kind of like caching).

Region specific persistence is one way to improve write latency, and I think the simplest for most apps. We've experimented with CockroachDB for this (it keeps ranges of rows where they're most busy), and you can actually deploy MongoDB this way.

I don't think I like the failure mode of your app getting slower (losing your ApplicationDN) right at the time it's really popular. I don't have a better solution though.