Caddy is great! But am I the only one who gets frustrated seeing breaking changes in a minor version release? I know semver isn't "the one true way", but it sure does solve a problem for me. I've had too many applications FTBFS or start up on minor (or even patch!) version bumps because they included breaking changes.
It's not clear to me that trying to assign semver semantics to a command line interface is going to work well; trying to define the "interface" for command line tools seems too challenging. Perhaps one might suggest that caddy should have used a different versioning pattern, but I have to admit that the x.y.z pattern is so prevalent these days, I have a hard time faulting someone for using it.
Interesting timing for me since I was just reading the semver.org site just last week. I know that their proposal is to only include breaking changes in the major version, but in my experience lots of products use major versions for marketing purposes, and minor versions for functional changes including both breaking and non-breaking changes.
The important thing is to be clear in your docs about your versioning strategy. And from a quick search of the Caddy website, I couldn't find anything that explains this. Their install guide doesn't really mention versions at all, giving people no clues that a minor version change could break their sites.
> in my experience lots of products use major versions for marketing purposes
This is what people are desperately hoping to change. The industry is tired of updating from 4.7.1 to 4.7.2 and having everything break. Please bring sanity to versioning, so people can have a reasonable expectation that x.y to x.z isn’t going to require days of rewriting stuff.
We could improve that for sure. I am hoping to redo the website docs later this year... and will try to include some information about our development and release process.
Basically we try to put the bigger changes in the "minor" version bump (because bumping the major version introduces a lot of friction in the Go ecosystem) and encourage people to read release notes. We are open to suggestions though!
> I know that their proposal is to only include breaking changes in the major version, but in my experience lots of products use major versions for marketing purposes, and minor versions for functional changes including both breaking and non-breaking changes.
The solution is one more number: Marketing.Breaking.Feature.Bugfix
Semver is great for libraries. Caddy is a project which has so many dimensions of "surface area" that it's hard to boil down the implications of changes into a single decimal-separated number. Presumably this is why a lot of larger projects use year-month versioning or just bump the major version every release. I'm actually considering doing the latter, except ... Go modules are very opinionated about major version bumps because it was designed with libraries, not commands (main funcs) in mind... and bumping the major version each release would be also be a major inconvenience to dozens of plugin authors.
Caddy has a Go API, a configuration surface (two built-in: JSON and Caddyfile); a config API; HTTP, TLS, TCP, and UDP behavior; a command-line interface; etc... the list goes on and on in ways that can break a deployment. Which of those does the one and only semver version apply to? Go has opinionated tooling around semver for the Go package stuff, so we kind of have to cater to that, but end users don't really care about that. We could split the project into multiple smaller sub-projects, each with their own versions, but then it gets confusing and tedious to build and maintain. It's also inconvenient for people to contribute to that. And we'd have to ship Caddy with several versions, one for each "surface area dimension."
We've settled on mediocrity with our current version scheming which I admit is not my favorite, but I haven't really found anything better yet. Year-month versions are nice except that it implies either a regular release cadence (which we don't have) or that a larger span between two releases is more significant than more frequent releases (maybe true, but maybe not); it doesn't really tell you anything about the build... just approximately when it was made, but not even exactly when it was made (a month is a big window!) - I guess if you do multiple releases per month you just tack another number on the end? Maybe it should just be a timestamp. Or we could invent an N-dimension decimal number or some sort of string that has to be split and parsed...
Anyway. We do try to be gentle with breaking changes. Most of these have been documented as deprecated for years as well as printing warnings in logs. But we try to minimize the number of these, for sure.
Your versioning system honestly doesn't sound that bad. If you took the major version out of the picture, you could consider the minor and patch versions to line up with the major and minor versioning scheme of Compatible Versioning (ComVer) [1]. I think if you were to explain your versioning system around ComVer, including the inconveniences which currently come from major version updates of Go modules, it would be quite clear. If Go were to improve upon the major version inconveniences down the road, you could then migrate your major / minor ComVer versions from the minor / patch SemVer versions to the major / minor SemVer versions. Great work, by the way.
The release notes are voluminous which is heartening because I use Caddy in several places and I think it’s great software. Long may the fixes and features continue!
Love caddy. Only minor downside is having to build your own container to get route53 DNS support.
Would be awesome if they provided a pre-built container for each module, but understand it'd create a very large number of builds for them, and then people would want module+module which would create an exponentional number of possibilities.
Maybe an all modules build and the ability to toggle them via ENV?
> Would be awesome if they provided a pre-built container for each module,
I created a project that does this, it automatically builds images on new releases of caddy or any of the plug-ins.
It builds images for almost all caddy-dns plug-ins, thanks to unlimited github actions minutes :)
Thanks for noticing! I try to enforce that on all the PRs, it makes it a lot easier to categorize and sort things in the release notes. I didn't come up with the idea though, it was a team effort (shoutout to Mohammed who helped set up a lot of our release processes)
I'm hoping to find a simple local http server that for development and basic local use cases. I have been using the http-server package from nodejs but I'd rather install a binary without dependencies.
I have been trying to use Caddy locally, haven't been successful with replacing http-server yet but it's a good start
Honestly, this has the potential to change the World Wide Web (I use that term very deliberately, given that PHP powers the vast majority of the Internet globally, including some of the biggest sites). This could be as huge of a change as when single core processors went multi-core. It is not uncommon to nearly double your site's performance and halve your work as a developer/devops person.
Not to be too pedantic but while it powers the majority of websites, it doesn't power much of anyone's day-to-day internet usage (even counting Facebook's usage of Hack).
The other biggest example of PHP usage I can think off is Wikipedia.
Whilst PHP is considered dated now and the language has many faults, you can't critic how easy is to get started (drag drop files in a directory and you're done)
go.sum is not imports. It's a checksums file, it's used in determining which versions of things to pin to. It doesn't necessarily mean all of those things end up in the final binary. See https://go.dev/ref/mod#go-sum-files. You can run `caddy build-info` to see everything that is a dependency in the build. From my count, a vanilla build of Caddy v2.8.0 has 122 packages as dependencies (many of which are indirect).
I think you underestimate how much a general purpose webserver needs to be able to do. Do you have any specific criticism about the list of dependencies? What do you think is actually unnecessary?
Not the original poster, but at a glance, oauth, sso, and a whole bunch of PostgreSQL(?!) packages stand out as surprising things to see in a web server.
If you actually run `caddy build-info`, you'll see that none of those are actually in the build. You can't just look at go.sum, like I explained above.
(Dividing line count by 2) It looks like we currently rely on about 322 direct or transitive dependencies. It's worth noting many of those are test-only dependencies.
A lot of the /x/ packages are repeated, just different versions; and I believe -- if my understanding is right -- a lot of these are only used for checksum verification, not necessarily a manifest of what is compiled into the binary.
But, the point remains, dependencies can bloat quickly, which is why we went with such a modular architecture for Caddy 2.
This is a bit tangential... I recently noticed that a module that is split into packages that have distinct subsets of dependencies will only add transitives to your own go.mod based on the packages that you actually use.
This tripped me up trying to minimize dependencies for a library by splitting it into a separate repo from the rest of the application, but this is totally unnecessary (and annoying to manage).
Is it just me, or is this a monumentally bad idea? Which is more likely - LE having to revoke certificates, or hacker figures out how to trigger the revocation mechanism? Or Congress passes a bill outlawing encryption that doesn't use a backdoor key, and LE is ordered to revoke hundreds of millions of keys?
This is a hammer in search of a nail that doesn't really exist.
Now I'm warming up to it. But I still think I would have done it differently... ARI is essentially a "hint" to clients, but at the same time CAs can provide benefits (e.g. rate limit exemptions) if abiding by ARI windows. It feels weird to have scheduling split between the CA and the client. I would have liked a solution that either keeps the scheduling authority with the client, or transfers it to the CA, rather than straddling both. (I think my ideal scenario would be to have a single endpoint that returns a list of all certificates on the account that should be renewed presently. That way clients just have to poll it and renew any certs on that list. It would be much simpler than what I ended up implementing.)
I am not actually concerned too much with ARI with respect to revocation or hackers or government coercion. Mainly complexity.
Anyway... I could probably talk about the pros and cons of ARI for a while. It is a net good, I think, and it does solve a problem (mostly a problem CAs have, and it kind of puts the burden on clients). I just want the ACME/TLS ecosystem to settle down a little now. :)
Caddy is the greatest thing since sliced bread. It is such a good reverse proxy and a paradigm-shifter for its auto-certificates and HTTP/3 support. It's a great example of how high quality Go software can be. (Thank you Matt Holt)
> I still have nightmares about trying to set up SSL with nginx and my own self-managed certificates.
For anyone who needs to run their own CA (which I'm now doing for my homelab), I've found that using GUI software like KeyStore explorer is a sufficiently easy and lazy way of doing that, which actually works well, both for securing regular sites, as well as doing mTLS: https://keystore-explorer.org/
For what it's worth, using OpenSSL directly and automating that for more frequently rotated certificates wouldn't be quite as pleasant, yet doable.
> Shoutout to Let’s Encrypt as well for making this so much easier!
For ACME stuff, Caddy will be excellent and honestly is probably the best option out there right now!
Nginx (and certbot) or Apache (and mod_md or certbot) will get you most of the way there as well, though the route will be a bit longer.
Caddy is amazingly simple to setup. Automatic HTTPS is a killer feature.
I have to use Envoy at work for gRPC services and I want to quit the industry every time I have to edit their YAML/protobuf monstrosity of a config system.
My websites run on https because how easy Caddy makes it. Caddy made it possible for me. Cannot thank Matt Holt enough for creating Caddy and making it available to all of us.
I haven't used Caddy and I'm sure it's great, but you could have used nginx or anything else as well also. Offering https is honestly pretty easy these days.
I've been using nginx for years and switched to Caddy just because I was so fed up with configuring nginx to automatically renew TLS certs issued by Let's Encrypt. This is so much easier and reliable with Caddy.
I know about certbot and have considered it, but our customers can use their own custom domain name, which means we need to be able to select the certificate depending on the SNI hostname, which is a bit tricky in nginx. It's possible to use the $ssl_server_name variable in the ssl_certificate and ssl_certificate_key directives, but then the certificate will be loaded for each TLS handshake. And also need to be careful with race conditions when refreshing the certificate, to ensure that the certificate and they key are matching. Overall, it's doable, and people do it, but it's not as straightforward as just using Caddy.
Envoy config surely is complex, but it's also the most flexible and robust way of managing config on a large scale I have come across.
The way envoy lets you create clusters of envoys, then just setup their config to come from a centralized config source through a grpc connection is honestly the most sane way of managing thousands of proxies at scale I have found. Trying to push nginx (or any other config as a file proxy) updates at scale is a nightmare of its own.
We manage a large number of envoy clusters, where the state of how proxying should happen is all contained within a SQL database where the rules and records change dozens or hundreds times a minute. There is one service that's responsible for monitoring the DB and translating it to envoy configs, then pushing them out to 1,000s of envoy processes. It has been extremely reliable and consistent. For a given input, always produce the same output. It's very easy to unit test, validate and verify, then push the update.
Nginx, and Caddy I'd imagine, are great at set-it-and-forget-it configs or use cases. But envoy is a programmable proxy where you can have dozens of clusters with different configs that get updated dozens of times a minute. I don't know of any other proxy that offers something like that.
Caddy does (some of) that too actually. It has a live config API and support for clusters and synchronized configs and TLS cert management. It can also get the proxy upstreams dynamically at request-time through various modules. Some of the biggest deployments program/automate Caddy configs using APIs and multi-region infrastructure.
Envoy is definitely a powerful & useful tool, we use external auth to centralize our authentication, I just dislike editing large yaml documents with 10 levels of indentation.
Reverse proxying is just one task that a web server can perform. Caddy also has a file server (directly serving files from disk or from some virtual filesystem), can write static responses, can directly execute code via plugins like FrankenPHP, can manipulate/rewrite/filter/route the request and/or response, etc. Just look at this list https://caddyserver.com/features
In simple terms, you can think of a reverse proxy as an http server that is a middle man. For a simple case why you might use one. SSL/TLS can be a pain to set up in your web application code. So what you can do is write your web application and not worry about SSL/TLS certs. Then you can place a reverse proxy infront of it and configure the reverse proxy for SSL/TLS. This way your not dealing with that complexity in your code and someone else is managing it. From there, the reverse proxy takes requests and reroute them to your web application. My reverse proxy is exposed to the internet on port 443, when a packet hits it, it knows to rereoute the traffic to server running on my machine at localhost port 8080. You can also have a reverse proxy to have one singular ingress point for many web applications. The reverse proxy will know that requests for http://MyCoolWebApp.com go to localhost:8080 and http://MyOtherCoolWebApp.com go to localhost:8081.
It's really opinionated about it though. I still don't know how to stop it from trying to get certificates for specific hostnames. It seems to work with everything auto, or nothing at all.
Define the host as http://hostname in the config instead of just hostname and it will do only http for that config. You can have a separate https config that's is different as well.
That's its value really. It has the defaults you usually want with minimal boilerplate. If you need/want something more complex it's not necessarily the right tool any more.
I say this not as any kind of dig against Caddy but I feel like the entire value proposition is that its default configuration covers the 90% case so well. Sometimes being easy to use with good defaults goes a really long way.
caddy is 1000x better than nginx. It’s like comparing a jet engine to a steam engine. Sure, the steam engine was good for awhile, but the whole world moved on.
I would love to have a version of Caddy written in Rust. I believe that this is going to happen some point in the future. Nginx has the best performance but not a good UX compared to caddy. Rewriting to Rust or C++ will make it the ultimate reverse proxy.
78 comments
[ 4.1 ms ] story [ 149 ms ] threadThe important thing is to be clear in your docs about your versioning strategy. And from a quick search of the Caddy website, I couldn't find anything that explains this. Their install guide doesn't really mention versions at all, giving people no clues that a minor version change could break their sites.
This is what people are desperately hoping to change. The industry is tired of updating from 4.7.1 to 4.7.2 and having everything break. Please bring sanity to versioning, so people can have a reasonable expectation that x.y to x.z isn’t going to require days of rewriting stuff.
Basically we try to put the bigger changes in the "minor" version bump (because bumping the major version introduces a lot of friction in the Go ecosystem) and encourage people to read release notes. We are open to suggestions though!
The solution is one more number: Marketing.Breaking.Feature.Bugfix
Semver is great for libraries. Caddy is a project which has so many dimensions of "surface area" that it's hard to boil down the implications of changes into a single decimal-separated number. Presumably this is why a lot of larger projects use year-month versioning or just bump the major version every release. I'm actually considering doing the latter, except ... Go modules are very opinionated about major version bumps because it was designed with libraries, not commands (main funcs) in mind... and bumping the major version each release would be also be a major inconvenience to dozens of plugin authors.
Caddy has a Go API, a configuration surface (two built-in: JSON and Caddyfile); a config API; HTTP, TLS, TCP, and UDP behavior; a command-line interface; etc... the list goes on and on in ways that can break a deployment. Which of those does the one and only semver version apply to? Go has opinionated tooling around semver for the Go package stuff, so we kind of have to cater to that, but end users don't really care about that. We could split the project into multiple smaller sub-projects, each with their own versions, but then it gets confusing and tedious to build and maintain. It's also inconvenient for people to contribute to that. And we'd have to ship Caddy with several versions, one for each "surface area dimension."
We've settled on mediocrity with our current version scheming which I admit is not my favorite, but I haven't really found anything better yet. Year-month versions are nice except that it implies either a regular release cadence (which we don't have) or that a larger span between two releases is more significant than more frequent releases (maybe true, but maybe not); it doesn't really tell you anything about the build... just approximately when it was made, but not even exactly when it was made (a month is a big window!) - I guess if you do multiple releases per month you just tack another number on the end? Maybe it should just be a timestamp. Or we could invent an N-dimension decimal number or some sort of string that has to be split and parsed...
Anyway. We do try to be gentle with breaking changes. Most of these have been documented as deprecated for years as well as printing warnings in logs. But we try to minimize the number of these, for sure.
[1]: https://gitlab.com/staltz/comver
Would be awesome if they provided a pre-built container for each module, but understand it'd create a very large number of builds for them, and then people would want module+module which would create an exponentional number of possibilities.
Maybe an all modules build and the ability to toggle them via ENV?
I created a project that does this, it automatically builds images on new releases of caddy or any of the plug-ins. It builds images for almost all caddy-dns plug-ins, thanks to unlimited github actions minutes :)
https://github.com/FarisZR/caddy-dns-OCI
It takes a lot of discipline to have all tickets & commits titled this way. But it makes reading a change log so much easier to parse as a human.
I have been trying to use Caddy locally, haven't been successful with replacing http-server yet but it's a good start
Whilst PHP is considered dated now and the language has many faults, you can't critic how easy is to get started (drag drop files in a directory and you're done)
https://github.com/caddyserver/caddy/blob/master/go.sum
it actually doesn’t seem like that long of a list if you only consider one version per library
Or just go version -m
> 122 packages
That's still a lot
> many of which are indirect)
That means literally nothing. Indirect still gets built, indirect still in the exe.
A lot of the /x/ packages are repeated, just different versions; and I believe -- if my understanding is right -- a lot of these are only used for checksum verification, not necessarily a manifest of what is compiled into the binary.
But, the point remains, dependencies can bloat quickly, which is why we went with such a modular architecture for Caddy 2.
This tripped me up trying to minimize dependencies for a library by splitting it into a separate repo from the rest of the application, but this is totally unnecessary (and annoying to manage).
Go modules are pretty good.
Is it just me, or is this a monumentally bad idea? Which is more likely - LE having to revoke certificates, or hacker figures out how to trigger the revocation mechanism? Or Congress passes a bill outlawing encryption that doesn't use a backdoor key, and LE is ordered to revoke hundreds of millions of keys?
This is a hammer in search of a nail that doesn't really exist.
Now I'm warming up to it. But I still think I would have done it differently... ARI is essentially a "hint" to clients, but at the same time CAs can provide benefits (e.g. rate limit exemptions) if abiding by ARI windows. It feels weird to have scheduling split between the CA and the client. I would have liked a solution that either keeps the scheduling authority with the client, or transfers it to the CA, rather than straddling both. (I think my ideal scenario would be to have a single endpoint that returns a list of all certificates on the account that should be renewed presently. That way clients just have to poll it and renew any certs on that list. It would be much simpler than what I ended up implementing.)
I am not actually concerned too much with ARI with respect to revocation or hackers or government coercion. Mainly complexity.
Anyway... I could probably talk about the pros and cons of ARI for a while. It is a net good, I think, and it does solve a problem (mostly a problem CAs have, and it kind of puts the burden on clients). I just want the ACME/TLS ecosystem to settle down a little now. :)
I set the same thing up with Caddy in minutes and it’s been running flawlessly for years.
Shoutout to Let’s Encrypt as well for making this so much easier!
For anyone who needs to run their own CA (which I'm now doing for my homelab), I've found that using GUI software like KeyStore explorer is a sufficiently easy and lazy way of doing that, which actually works well, both for securing regular sites, as well as doing mTLS: https://keystore-explorer.org/
For what it's worth, using OpenSSL directly and automating that for more frequently rotated certificates wouldn't be quite as pleasant, yet doable.
> Shoutout to Let’s Encrypt as well for making this so much easier!
For ACME stuff, Caddy will be excellent and honestly is probably the best option out there right now!
Nginx (and certbot) or Apache (and mod_md or certbot) will get you most of the way there as well, though the route will be a bit longer.
I have to use Envoy at work for gRPC services and I want to quit the industry every time I have to edit their YAML/protobuf monstrosity of a config system.
The way envoy lets you create clusters of envoys, then just setup their config to come from a centralized config source through a grpc connection is honestly the most sane way of managing thousands of proxies at scale I have found. Trying to push nginx (or any other config as a file proxy) updates at scale is a nightmare of its own.
We manage a large number of envoy clusters, where the state of how proxying should happen is all contained within a SQL database where the rules and records change dozens or hundreds times a minute. There is one service that's responsible for monitoring the DB and translating it to envoy configs, then pushing them out to 1,000s of envoy processes. It has been extremely reliable and consistent. For a given input, always produce the same output. It's very easy to unit test, validate and verify, then push the update.
Nginx, and Caddy I'd imagine, are great at set-it-and-forget-it configs or use cases. But envoy is a programmable proxy where you can have dozens of clusters with different configs that get updated dozens of times a minute. I don't know of any other proxy that offers something like that.
But where Envoy shines, it really shines.
That's its value really. It has the defaults you usually want with minimal boilerplate. If you need/want something more complex it's not necessarily the right tool any more.
I say this not as any kind of dig against Caddy but I feel like the entire value proposition is that its default configuration covers the 90% case so well. Sometimes being easy to use with good defaults goes a really long way.
I was hoping I could use it to forward "pure" UDP ports but I think it's considered out-of-scope by the devs. They do support HTTP/3, however.
I recent wrote a forward_auth server to use with Caddy's forward_auth functionality:
https://github.com/crowdwave/checkpoint401
https://caddyserver.com/docs/caddyfile/directives/forward_au...