Tomcat/Jakarta EE/JSP is a surprisingly solid stack. I only tried it once. Everything mostly just worked, and worked pretty well. You get to write pages PHP-style (interspersed HTML and code) but with the full power of Java instead of a hack language like PHP. Of course that paradigm may not suit everyone, but you also don't have to handle requests that way as you can also install pure Java routes. It supports websockets. You can share data between requests since it's a single-process multi-threaded model, so you can write something with real-time communication. You can also not do that; JSP code (and of course local variables) is scoped to a request. Deployment is very easy: drop the new webapp (a single file) in the webapps directory, by any method you like e.g. scp, and when Tomcat notices the new file is there, it transparently loads the new app and unloads the old one. You do have to watch out for classloader leaks that would prevent the old app being garbage-collected, though - downside of a single-process model.
Surprised with the choice of Apache. There are better choices for serving CGI nowadays. The only reason for still running Apache is you have legacy cruft that requires Apache (like .htaccess).
Apache has a lot of cool modules for auth handling (like open connect), so this makes it easier if you want to write your business logic in a more obscure language while still have auth setup for you.
I got my start in the CGI era, and it baked into me an extremely strong bias against running short-lived subprocesses for things.
We invented PHP and FastCGI mainly to get away from the performance hit of starting a new process just to handle a web request!
It was only a few years ago that I realized that modern hardware means that it really isn't prohibitively expensive to do that any more - this benchmark gets to 2,000/requests a second, and if you can even get to a few hundred requests a second it's easy enough to scale across multiple instances these days.
I have seen AWS Lambda described as the CGI model reborn and that's a pretty fair analogy.
Even back in the 1990s, CGI programs written in C were lightning fast. It just was (is) an error prone environment. Any safer modern alternative like the article's Go program or Nim or whatever not making database connections will be very fast & low latency to localhost - really similar to a CLI utility where you fork & exec. It's not free, but it's not that expensive compared to network latencies then or now.
People/orgs do tend to get kind of addicted to certain technologies that can interact poorly with the one-shot model, though. E.g., high start up cost Python interpreters with a lot of imports are still pretty slow, and people get addicted to that ecosystem and so need multi-shot/persistent alternatives.
The one-shot model in early HTTP was itself a pendulum swing from other concerns, e.g. ftp servers not having enough RAM for 100s of long-lived, often mostly idle logins.
The nice thing about CGI is that you don't have to reinvent isolation primitives for multi-tenant use cases. A bug in one request doesn't corrupt another request, due to process isolation. An infinite loop in one request doesn't DoS other requests, due to preemptive scheduling. You can kill long-running requests with rlimit. You can use per-tenant cgroups to fairly allocate resources like memory, CPU and disk/network I/O. You can use namespaces/jails and privilege separation to restrict what a request has access to.
> These days, we have servers with 384 CPU threads. Even a small VM can have 16 CPUs. The CPUs and memory are much faster as well.
With this hardware, if you reach for Kestrel you can easily do a few trillion requests per day. The development experience would be nearly identical - You can leverage the string interpolation operator for a PHP-like experience. LINQ and String.Join() open the door to some very terse HTML template syntax for tables and other nested elements.
The hard part is knowing how to avoid certain landmines in the ecosystem (MVC/Blazor/EF/etc.). The whole thing can live in one top-level program file that is ran on the CLI, but you need to know the magic keywords - "Minimal APIs" - or you will find yourself in the middle of the wrong documentation.
Does anyone know if the benchmarking tool the author uses, plow, avoids coordinated omission (https://www.scylladb.com/2021/04/22/on-coordinated-omission/)? I didn’t see any mention in the docs, and haven’t been able to peruse the source code yet.
This was something that I've been suspecting for some time. We're moving towards complicated architecture while having possibility to use good ol' tech with newest CPUs.
I've been asked about architecture of a stocks ticker that would serve millions of clients to show them on their phone the current stock price. First thought was streams, Kafka, pubsub etc but then I came up with static files on a server.
I've created a visualizer for apache requests with the workers, queues and whatnot [0]. You can load the demo to view real traffic comic from HN earlier this year.
2400 requests is somewhat decent. Having seen companies like amazon, where their website backend can only handle a few hundred RPS per box, 2400 ain't that bad.
Yeah, instead of relooking at this, we went off and built a new paradigm “serverless functions.” Obviously, serverless functions, like those via Lambda, have some other safety mechanisms in place (eg micro vms), but you could probably get pretty far with CGI and adjusting capabilities and such, with far less complexity.
> I ran these benchmarks on an older 16-thread AMD 3700X
The 3700X is an 8 Core Zen 2 CPU. Or about 150 RPS per vCPU. By EOY we will have 256 Core Zen 6c EPYC, on a Dual Socket that is 512 Core or 1024 Thread. 153,600 RPS.
And in the old days a RPS is a single pageview, which isn't the case in the modern world.
CPU Core still have a healthy 5 - 10 years cost reduction roadmap. I wonder if CGI will make a comeback someday.
Why would it be limited to ~ 100 connections on a 1-4 GB RAM server? Out of curiosity if we fork() httpd and exec() the cgi handler, it doesn't take the same RAM as the parent process and it could just take a few KB or MB, is that right? So I guess 1000+ concurrent connections even on a small server is possible.
Over a decade ago I was starting a java process up with local MySQL and getting 45k rps read, request per thread with load wasn't hard to achieve. Not sure why this is an accomplishment.
28 comments
[ 4.0 ms ] story [ 79.7 ms ] threadOne shared JVM for maximum performance!
It can also share db connection pools, caches, etc. among those applications!
Wow!
We invented PHP and FastCGI mainly to get away from the performance hit of starting a new process just to handle a web request!
It was only a few years ago that I realized that modern hardware means that it really isn't prohibitively expensive to do that any more - this benchmark gets to 2,000/requests a second, and if you can even get to a few hundred requests a second it's easy enough to scale across multiple instances these days.
I have seen AWS Lambda described as the CGI model reborn and that's a pretty fair analogy.
People/orgs do tend to get kind of addicted to certain technologies that can interact poorly with the one-shot model, though. E.g., high start up cost Python interpreters with a lot of imports are still pretty slow, and people get addicted to that ecosystem and so need multi-shot/persistent alternatives.
The one-shot model in early HTTP was itself a pendulum swing from other concerns, e.g. ftp servers not having enough RAM for 100s of long-lived, often mostly idle logins.
With this hardware, if you reach for Kestrel you can easily do a few trillion requests per day. The development experience would be nearly identical - You can leverage the string interpolation operator for a PHP-like experience. LINQ and String.Join() open the door to some very terse HTML template syntax for tables and other nested elements.
The hard part is knowing how to avoid certain landmines in the ecosystem (MVC/Blazor/EF/etc.). The whole thing can live in one top-level program file that is ran on the CLI, but you need to know the magic keywords - "Minimal APIs" - or you will find yourself in the middle of the wrong documentation.
I've been asked about architecture of a stocks ticker that would serve millions of clients to show them on their phone the current stock price. First thought was streams, Kafka, pubsub etc but then I came up with static files on a server.
I wonder how much would it cost though
[0]: https://www.ibrahimdiallo.com/reqvis
Note: works best on desktop browsers for now.
At the very least go for FastCGI, for christ’s sake…
If this refers to https://github.com/six-ddc/plow then -- oops! lots of issues in that repo, no tests, etc. etc.
The results in the README are also pretty clearly unsound! In both scenarios, writes were faster than reads?
_edit_: I guess because the writes all returned 3xx, oops again!
Probably don't take this article's claims at face value...
I just did a `time perl -e ''` (starting perl, executing an empty program), it took 5ms. 33ms with python3, 77ms with ruby.
The 3700X is an 8 Core Zen 2 CPU. Or about 150 RPS per vCPU. By EOY we will have 256 Core Zen 6c EPYC, on a Dual Socket that is 512 Core or 1024 Thread. 153,600 RPS.
And in the old days a RPS is a single pageview, which isn't the case in the modern world.
CPU Core still have a healthy 5 - 10 years cost reduction roadmap. I wonder if CGI will make a comeback someday.
Could someone please explain what the better options are?
- Keep-alive/pooled connections to remote services can significantly reduce average latency for making those calls.
- In-memory caches that allow amortizing repeated lookups across requests.
Just those two alone mean that a serious high performance server probably couldn’t get away with just CGI even ignoring startup time.