Same here! We are huge fans of Caddy at Narration Box (https://narrationbox.com) and we are considering replacing NGINX completely in our Django production stack. (Currently we only use Caddy for development and staging) The author of Caddy, Matt Holt, is a really great person and is extremely responsive on Twitter and GitHub. Highly recommend checking them out.
Nginx requires too much configuration. It is very brittle if you do not have a full time DevOps person monitoring things. E.g. for production web apps, take a look at the amount of tooling that has been developed to manage Nginx:
This is not a criticism of Nginx per se, it comes from a time when Apache is the only other alternative and end-user UX is hardly priority. An analogy would be systemd and pm2, pm2 have few advantages over systemd on paper but in practice it has a much more pleasant UX and saner defaults even though it requires a separate installation on most servers while systemd is available by default. It is the difference between Heroku and AWS. For the 99th percentile of efficiency Nginx still beats Traefik and Caddy, but sometimes scaling servers is cheaper than hiring DevOps staff.
I see Caddy as your general use case production web server prior to hitting large scaling needs. I've never gotten it to perform as leanly or scale as well as nginx, but most people will probably not need the headroom nginx provides vs. caddy.
Caddy is significantly easier to manage and get running to a "good enough" stage for people that aren't familiar with nginx and apache, even for most medium sized production deployments.
But, if you've got a team that's comfortable with nginx or apache, and doesn't find themselves spending an undue amount of time fiddling around with managing them, I don't personally believe there's any reason to switch from them to caddy.
I really like Caddy and use it for general side projects. Super simple setup, handles all the LetsEncrypt certificates for me, can be running different apps on different paths.
We have several companies using it for tens of thousands of their customer sites. It's the only server we know of that can handle hundreds of thousands of certificates efficiently!
I somewhat remember the Caddy team having a history of shady behavior: advertisements in headers, hostility towards contributors, telemetry you have to opt-out of. Has any of this changed or has it become worse?
The telemetry server was shut down months ago and v2 has no telemetry client. Although, the counts we gathered were informative. We learned a lot about MITM activity on the Internet, the health and maturity of TLS clients, and how many connections Caddy has secured (trillions, if you're wondering).
As for whether things have gotten worse, I suppose that's up to you!
I wish there was a better way manage telemetry for open source. We have this problem at smallstep, too. We don’t have any telemetry — nothing is instrumented / nothing phones home — but that means it’s really hard to know which features are popular. Which makes it hard to prioritize. We only find out when something breaks and we can look at the volume of issues, heh.
We ask people to let us know if / how they’re using our stuff, but very few people do. It’s a tough problem.
Why did you do it? What have you sought to achieve with it? It's always a head scratcher for me why people building open source tools would put time and effort towards telemetry. Advertisments in HTTP headers? That's a new one. Advertise to whom, devs looking at a browser debug console?
It was for a class during my grad program. It was an academic interest in understanding client behavior on the Internet that wasn't limited to proprietary networks like Cloudflare or Google.
We collected some good measurement data -- anonymous, technical, non-personal, etc -- but the terabytes added up and the academic community didn't seem particularly responsive to it in the end. The "opt-in" / "opt-out" depended on how you built/obtained Caddy, but essentially we made it a compile-time decision so that we could reduce biases from the data. By deciding on the download page whether telemetry was enabled, we could also know how representative the data actually was: otherwise it'd be meaningless.
We also wanted to know how widely Caddy was being used. Telemetry was mostly just counts of things, so that's how we know that Caddy has secured trillions of HTTPS requests and managed millions of certs. But it was expensive to run.
The "ads" in headers were intended to be a friendly nod to our sponsors who made it possible. It was a novel idea. I thought it was a good balance of non-intrusive and perfectly visible at the same time: developers who were peering into HTTP requests would see the headers and our sponsors would get some benign recognition from their target audience, while nobody else would see them. It also was supposed to encourage purchasing commercial licenses, which didn't have that header. The licenses were necessary to continue funding a desperately underfunded project.
Needless to say that didn't work out and the only reason the Caddy project didn't shut down entirely is because corporate sponsors (Ardan Labs and one other to be announced probably next week) believed in the project enough to pick it up.
So anyway, I also got college credit for implementing an Internet measurement system, which was really fun and interesting. And as mmalone said adjacently, open source projects really need to know what kind of usage they're getting. With no way to engage customers except at their voluntary discretion, it's impossible to know how to improve the project. Open source is, by definition, an open feedback loop. It only closes if users come back and provide information.
A lot of other major open source projects or free software ship telemetry, even on by default sometimes -- see Windows, Chrome, Ubuntu, Firefox, VS Code, macOS, and countless others. Yet nobody cares.
But having that information was critical in shaping the development of Caddy 2, FWIW.
It's not a stretch to say that other projects didn't get as much hate for it. It really hurt the project's momentum when there was such a strong reaction against it.
It is absolutely a stretch. Mozilla is still catching hell for it. People have switched operating systems over this. Caddy is no different in that regard... except that its creator did the right thing and got rid of it. Mozilla and Microsoft haven't.
> Windows, Chrome, Ubuntu, Firefox, VS Code, macOS
Those are projects funded by 800-pound gorilla companies/foundations. It's either their way or the highway and people complain all the time, some have even forked the code and ripped out telemetry (Vscodium). Homebrew got a response comparable to Caddy but they screwed up even worse because they use Google Analytics.
>It's always a head scratcher for me why people building open source tools would put time and effort towards telemetry.
Because you want to know what people are actually using in your projects. I've never put telemetry in anything I use, but I 100% understand why. It's hard to prioritize feature development if you don't know how people are using what you're making, especially if the users aren't very vocal. Without people talking to you and without telemetry on what people are doing, you might spend months worth of man-hours on something no one cares about.
For projects where all you do is build out the things you specifically need this isn't an issue - all you care about is making the stuff you're doing work, and people getting benefit out of things is a bonus but not necessarily your goal. People will submit pull requests or feature requests, or they won't. No skin off your nose. Not all projects are developed this way, though.
> It's always a head scratcher for me why people building open source tools would put time and effort towards telemetry.
It's not completely the same as telemetry, but as an administrator of a free open source web service, I have to say that web page analytics (1st party for us) are a huge motivator, when you see how many people from all around the world are using it.
Matt's already spoken to the ads/telemetry (which are ancient history).
My experience as a contributor has been nothing but positive. But if anyone's had a bad experience, I'm sure the Caddy team is more than willing to talk it out and reconcile.
Hi HN -- this is what I've been working on for the last 14 months, with the help of many contributors and the backing of several sponsors. (Thank You!)
Caddy 2 is a fresh new server experience. Some things might take getting used to, like having every site served over HTTPS unless you specify http:// explicitly in your config. But in general, it will feel familiar to v1 in a lot of ways. If you've used v1 before, I recommend going into v2 with a clean-slate using our Getting Started guide: https://caddyserver.com/docs/getting-started
I'm excited to finally get this out there. Let me know what questions you have!
Hey Matt - thanks for creating and maintaining Caddy all these years! Like others have said in this thread, it is so easy to set up and maintain that it really does feel like "magic".
In terms of speeding up adoption of Caddy 2, it may be useful to have a list somewhere of the concrete improvements between the two (as I'm sure there are many). A (very) brief look and search around only yielded this article[1] which referenced another link of improvements that now 404s[2].
Another piece of feedback: It’s scary to consider using such a crucial piece of software in production when documentation is referred to as “sort of deprecated” and “slightly outdated.”
Thanks, we'll work on it! The nav/flow of content could probably be improved.
Not gonna lie though, there's more to learn. V2 is a powerful machine -- so do expect that there will be some reading. Once you know how it works, it's easy to use. Very simple configs are possible, etc.
It's often used as a reverse proxy and static file server, but oh, so much more is possible. Today, if you're using HTTPS, you should almost certainly be using Caddy. Maybe tomorrow if you need to set up a memory-safe SSH server, you could be using Caddy. (Just an example.)
Not a stupid question, it's not a phrase you hear very often because there isn't... really... one... at all. (Not in mainstream use AFAIK?)
Memory safety is a class of guarantees certain software offers you against certain vulnerabilities. Software written in C is generally considered "memory unsafe" since it's hard to write correct C code when managing with memory, so it's easier to find exploits in them that cause them to reveal secrets.
Go software has stronger memory safety guarantees than C programs like OpenSSH and Nginx. So that's one big benefit of using Caddy.
As it happens, someone in the audience here is writing a SSH app for Caddy, so you have a pure Go SSH server that is less vulnerable to those class of attacks.
But what is the value add for Caddy here? Like how the out of box and auto SSL defaults with HTTP.
Apart from memory safety, can the SSH version of caddy impose better defaults that OpenSSH doesn’t right now? Maybe TLS certs, security key support, etc?
Absolutely. And Caddy adds its on-line config API and simpler configuration experience for an all-around more secure, easier-to-maintain, harder-to-get-wrong system.
Right, I suppose I'm asking what an elegant SSH config would look like, having almost only ever touched my sshd config less than 10 times in the 15+ years I've managed servers.
I know I'd ideally like easier SSO integration, for example. Or provisioning of users.
Yeah, my hope is that someone will write a sort of scheduler / supervisor app for it so that I don't have to keep re-learning systemd every time I stand up a new service...
Thank you for Caddy, it has saved me a lot of time and it's a joy to work with. Running it on about 8 boxes.
Is there an admin UI for v2? Seeing the configuration changes, it was the first idea that popped, would be great for the selfhosted community - maybe someone picks it up as a side project if it's not in scope.
Thank you for your work on Caddy! After a false start during the beta (mainly because of the missing documentation) I upgraded my personal websites from v1 to v2 RC1, and since then I have been a fan of the new version. Caddy 2 makes somewhat complex configuration more consistent and easier to express in the Caddyfile. The only thing I miss from v1 is the default handling of when static files are not found.
I am very interested in the prospects of Starlark in Caddy 2. With an integrated scripting language Caddy could on its own be a replacement for OpenResty or Apache with mod_mruby. The preliminary implementation was removed in the beta phase with a note saying it would have to wait for when the project was financially stable [1]. Do you still plan to integrate Starlark if the project is a financial success?
Just tried it by replacing NGinX on my personal servers... I don't have anything complex (python backends, some static files...) but so far the user experience is stellar :)
CommonName has been deprecated for years, so all of Caddy's certificates (and all of Let's Encrypt's) are SAN certificates. Caddy manages single-SAN certs, in accordance with recommended best practices: https://docs.https.dev/acme-ops#use-one-name-per-certificate
(We learned from experience that single SAN scales better and is less prone to troubles. For example, Caddy sites were not affected by the recent Let's Encrypt revocation incident because it manages only single-SAN certificates. And frankly, that shouldn't matter since you don't have to manage them.)
The nginx autoconfiguration plugin has no influence on how many domains the certificate you're requesting will be valid for.
Example: We run a SaaS on our client's domains, so for every client, we have to run `certbot -d subdomain.client1.com,client1.ourdomain.com,staging.client1.ourdomain.com,...`
That means we run certbot with ~4 domains (we include staging and other subdomains) for each client we have. This is highly automatable.
Whether you use nginx autoconfiguration or not is up to you.
No no, you should totally use separate certificates (for our SaaS that's crucial, otherwise we would let our competitors know who our entire client base is, if all the domains would be included in one certificate)
I don't know too much about certbot's nginx plugin, I don't use it and don't see the benefit to be honest, we ran into problems with it (it didn't work for us because our clients have to set up a CNAME DNS entry, and that domain has to be included in the certificate)
We simply run certbot without the nginx plugin, and then have a config template for new virtual servers in nginx, based on a template. Certbot's nginx plugin would mess with our config constantly, changing from version to version, leaving artifacts, and we didn't like that.
Good luck! I'd also recommend splitting the config into multiple files, like it has been best practice for the last decades [1] (link goes to Arch (which I also recommend using) but the advice is not Arch specific)
Some parameters might have one or two more words to type, but the growth in flexibility is enormous, and we felt it was worth the tradeoff. You can see a comparison here: https://caddyserver.com/docs/v2-upgrade#caddyfile
We also think slightly more verbosity in the v2 Caddyfile is actually a good thing. Aside from more flexibility, it makes it a little clearer what's going on, and concerns are better separated.
For example, the reverse proxy is no longer in the business of rewriting requests: there are other directives for that.
In fact, we completely got rid of "rewrite hacks" that involved appending path components to the URI so they could be routed, then needing to strip those before proxying.
Request matching is very powerful in Caddy 2: you can still easily match on paths in-line with your handler directives, but you can also match on any other part of the request (client IP, query string, regular expressions, etc) with an intuitive @-block syntax: https://caddyserver.com/docs/caddyfile/matchers
Anyway, I think you'll feel right at home with it after a while. I've been using it for months now and it's so simple overall.
(I also use the JSON config directly on a couple of my servers, where the behavior I need to express is way more elegant and concise than in the Caddyfile.)
Thanks for the response! Looking forward to upgrade my stuff. Is there a reason why you decided to introduce an entirely new configuration method via JSON?
Basically, it is highly interoperable and allows us to build config adapters on top of it, so you don't have to use JSON if you really don't want to. Most people use the Caddyfile. You can use YAML or TOML or CUE if you prefer those. Or heck, you can use your nginx config: https://github.com/caddyserver/nginx-adapter
I started using caddy v1 thanks to lightning-fast and simple TLS setup and extremely simple configuration in general, thanks to the excellent docs and handy plugins for common stuff like cors, rewrites and fastcgi.
Shortly after upgrading to caddy v2, I switched to nginx, after spending way too much time trying to make sense of the new Caddyfile and docs (the thing I miss the most of the docs is the plugin list, along with the cors plugin, and in general the non-existant entry curve of Caddy v1).
Tbh, I also miss the one-liner install script <3
I'm genuinely sorry for the project; I didn't mind at all any of the previous so-called shady behavior (I personally don't object at all against a developer trying to make a living out of his own open-source project); I've been using caddy for four years now, and I really hope to switch back to it, one day.
If you need any help when coming back to Caddy v2, come ask for help on the community forums: https://caddy.community/
We'll be glad to help you get set up!
The docs are still a work in progress, we'll eventually have a plugin registry like in v1 but that requires a bunch of work behind the scenes to make it happen.
As francislavoie said, the website with its nice one-click custom download page and list of registered plugins is a lot of work. It'll come, but it's not ready yet. Much like the v1 release 5 years ago where the interactive download page didn't come until about a year later, it'll take some time. Hopefully not a full year though. :)
I've already got the new build server mostly written -- should be way faster.
Maybe this has changed since I looked at things yesterday, but I couldn't find a list of default plugins anywhere on the website, nor how to configure those default plugins. I think this is what the person you're replying to is talking about.
On the old site, there was a plugin documentation page that made it trivial to see what the plugins were called and what options they took.
I eventually asked my local copy of Caddy what plugins were available, which answered one of the two questions, but am I missing some kind of documentation page?
The interactive, custom build page is an unrelated, but also interesting topic. About a year ago, I was using abiosoft's Caddy Builder image to build a Caddy docker image with the plugins I wanted, but this no longer seems to work even for Caddy V1, and I don't think it has been updated for V2. It would be awesome if there were an official Caddy Builder docker image. Even if the website existed, I would still have to wrap that binary in a docker image somehow for my own use cases.
But, these are all just my personal opinions -- I'm sure you have your own priorities for the project, and I'm excited to see V2 has officially released!
The new website does not yet have the ability to register plugins. I'm writing it -- I already have the new build server mostly written -- but like with v1, it may take a few months.
It's not worth delaying the release for that, especially when we have tools like xcaddy that make it easy to build Caddy with plugins, no need to touch any code: https://github.com/caddyserver/xcaddy
Right now, the process is: find a plugin you like, then run:
xcaddy build --with <module-path>
But yeah, it will get easier with time. We'll get there.
If you're navigating from the homepage, click on "Documentation" at the top, then click on "modules" on the left navbar just above "JSON Config Structure". That list is generated directly from the source code, so it is the definite list of what's included by default and supported by the native JSON config.
The other way is, as you found out, to run `caddy list-modules` to get the definitive list of what's packed into the binary.
I have been using Caddy v1 for more than a year now as a web server for some of my production sites. I'm really happy with the simplicity of it and especially out of the box SSL support with LetsEncrypt. I believe with v2 I will be moving more of my sites to Caddy. Thank you for all the hard work :)
Thank you for sharing your experience. Glad to hear it. Feel free to join our forums and be a part of the community! https://caddy.community - could always use more people with experience helping out. :)
I liked v1 a whole lot. I'm interested in v2, though I'm extremely skeptical of the additional complexity of the config adapter layer (https://caddyserver.com/docs/config-adapters) - I'm more or less sold on having an internal JSON structure with a DSL on top of it (allowing dynamic/programatic config is really interesting), but the idea of having _multiple_ different DSLs or config syntax bindings for it seems a bit much.
The biggest feature in Caddy 2, from what I can tell, is the new HTTP config API, and I'd love to see a practical example of what could be done with that.
Caddyfile is the primary supported DSL. The rest are nearly 1:1 mappings to the internal JSON with some sugar, except for the nginx config adapter which is a bit special! For example jsonc provides JSON, but with comments.
A few of those are third party maintained, and are plugins. They're listed on that page to point out what's possible.
Don't worry about the "additional complexity" -- config adapters are modular, and only the Caddyfile adapter is baked into the standard modules. Just plug in the adapters you need. Caddy functionality can grow rather infinitely without bloat because of this.
True, the Caddyfile adapter is a bit of a beast because it is extensible itself -- but it's a lot cleaner and more flexible than the v1 implementation, where the Caddyfile was its native config syntax.
Believe me, this is a big improvement.
Anyway, you don't have to use the adapters if you don't want to. That's the beauty of it.
More info on the architecture and config adapters:
There's no wsgi transport built-in, we haven't found any golang libraries that let us do this yet. For now, until we find a solution, we recommend using something like gunicorn and use the `reverse_proxy` directive in Caddy to proxy to it.
I'm no *SGI expert, but ASGI might be easier to bridge with the Golang-style concurrency-model than WSGI. Not sure if there are any libraries for go<->ASGI yet though.
Thank you so much for investing the time to do it, this was the only thing holding me back from using Caddy for everything over the last ~2 years. I know it's a painful process for Go packages in particular, but it's definitely worth it to be more easily installable for diehard apt users like myself.
That sidebar doesn't show on major landing pages, so it's not as silly a mistake as it might seem. I missed it for several minutes while looking around myself, as I started by following the prominent links on those landing pages. I only found the separate installation information page when someone else here helpfully linked to it.
Honestly, I hadn't even noticed the search box. It appears in a slightly unusual place and styled sufficiently subtly that I totally overlooked it throughout my entire browsing session. (FWIW, this is on a very large and colour-calibrated monitor on my PC, so my experience here may not be typical.)
In case it's of interest, as a new visitor, I started browsing from your main v2 landing page, scanning down most of that information. Then I followed the prominent download and get started links near the top. I think I next went to the documentation area, and started browsing the links on the left, though I totally missed the "Install" link just under "Welcome" because my eye was drawn first to the Tutorials section and its getting started link, and then I went exploring from there on down.
I suppose that was an unfortunate combination of two things to miss. :-)
You may already know this, but it's very tricky to package Golang projects in official debian repos because they require that every dependency be a separate package. This is a ton work both upfront and to maintain in the long-term, so we haven't made much progress on that front.
We'd love to get some help from people who are more comfortable with the debian ecosystem though! It's definitely something we want to achieve, but for the short term we had to make the decision to simply release the .deb via https://gemfury.com which provides us with free APT repo hosting for open source projects.
Unless I'm missing something, that only says you need to package all dependencies if it is a library. Which means you could make a binary-only statically compiled package without adding a bunch of dependencies right?
Of course, that isn't helpful if you need to compile with extra plugins, but in that case you are probably using standard go tools and not apt packages anyway.
We were specifically told that we would need to have all our dependencies packaged. Debian needs to be able to validate that it was built entirely from source code from library packages, not from a binary built externally.
Be aware that the [trusted=yes] config disables GPG signature verification for the repository, which does slightly break the security model of Apt (where repositories can be untrusted/HTTP-only as long as all packages are signed by trusted keys).
Unless I'm mistaken, a malicious package with the same name as a critical system package could be added to that untrusted repository by an attacker who had compromised it, and if the version number is higher than that of the 'official' package and your Apt priority config prioritises it, your system would download and install it without any verification.
This is a fairly niche attack vector that may not be considered a significant risk in some environments, but it's one to consider when establishing your threat model and risk appetite.
As a side note, it's worth noting that GPG signing packages is not a silver bullet either, especially if the signing keys and administrative access to the repository fall within the same security boundary (e.g. developer doing both from their PC without any segregation/sandboxing). However, it has proven to be a robust method so far, and definitely beats explicit [trusted=yes].
Yeah, we're aware. Unfortunately Gemfury doesn't support GPG signing yet (see https://gemfury.com/help/apt-repository#apt-setup) but we will set that up as soon as they do. Their service made it the easiest to get set up on short notice.
Hi mholt!,
I am building an ecommerce SaaS.Obviously,i will have each domain pointed to my server.Is Caddy a good choice for automatically getting a certificate for each website?Are there any limitations?
Thanks!
Yes, it is! Ask Jack Ellis at Fathom Analytics, they're doing this for their customers: https://usefathom.com/blog/bypass-adblockers -- and I think they have a new blog post and video coming out soon about it.
We've had great fun with Caddy for Version 1 of our custom domains feature.
For Version 2, we've done a lot of work with Matt and the solution we've now reached is:
1. DynamoDB as storage for certificates, allowing sharing between servers without regenerating during an issue with an availability zone
2. Multiple Caddy reverse proxy servers in different regions
3. AWS Global Accelerator to route the user to the closet server to them
It's so great because we can also proxy our CDN through Caddy (yes, it needs to go through the user's custom domain), and we've got insanely fast response times on that.
We've load tested the proxy servers and they can handle an incredible amount, we're very pleased with the solution.
I will be publishing and sharing an article soon, detailing my journey and our final solution.
I've started using caddy 2 couple of days ago, I am using Caddyfile because json based configuration looks really awful and is hard to maintain (quick changes take a lot of time). The only problem I've encountered is that you can't turn off TLS for a route using Caddyfile. Why is that?
* Citation needed? (I mean, yes, of the thousands of dimensions of a web server, Caddy is not the fastest or leanest in all of them, but what's important to you?)
I just upgraded a server to Caddy 2 this past weekend. It's running a single WordPress site.
There were some gotchas in converting the Caddyfile to the new format, but I eventually sorted it out. Caddy 2's method of updating the config made the iteration process a lot faster.
I'll admit I got bit by a few differences between the v1 Caddyfile and the v2 Caddyfile too. Namely forgetting * at the end of a path _prefix_ matcher. (In Caddy 2, path matches are exact, unless you suffix with a *.)
I see a mention of using it as an ingress controller, but not much in the docs. Is there more details on this or is this better suited to run in a container as opposed to using nginx for that and leave the ingress to traefik/etc?
Hi @mholt, thanks a lot for Caddy. It is really awesome and we are considering using it for www.mailerlite.com landing pages. The problem that we have right now is that there is no Cloudflare plugin built-in for V2 so it is a show stopper for us.
"Latest", as in "1 hour ago" latest, or "15 minutes ago" latest? Because the 2.0 image (not RCs) is needed and I think it only got published a few minutes ago.
I have just tried again, used caddy:latest and got the same error. Didn't mean to take much of your time, I can open up a GitHub issue if it's the better way to resolve this.
EDIT: I have just seen your edit. Will try and report back.
Thanks for the nice piece of software. I like Caddy a lot. I would like to better integration with docker in terms of automated exposure of docker services based on labels for example. Similar how traefik does. That would be really nice to have.
I've always felt like that was kind of a hack, but with Caddy's API it's doable. I don't really use Docker myself, so... want to contribute the feature? I'll show you how to integrate it.
246 comments
[ 3.1 ms ] story [ 254 ms ] threadI haven't actually heard of Narration Box before now, but it looks fantastic!
I'm also a huge fan of Sourcegraph (@sqs if you're reading this!) -- which has really helped Caddy get to this point.
https://www.digitalocean.com/community/tools/nginx
https://github.com/yandex/gixy
This is not a criticism of Nginx per se, it comes from a time when Apache is the only other alternative and end-user UX is hardly priority. An analogy would be systemd and pm2, pm2 have few advantages over systemd on paper but in practice it has a much more pleasant UX and saner defaults even though it requires a separate installation on most servers while systemd is available by default. It is the difference between Heroku and AWS. For the 99th percentile of efficiency Nginx still beats Traefik and Caddy, but sometimes scaling servers is cheaper than hiring DevOps staff.
Caddy is significantly easier to manage and get running to a "good enough" stage for people that aren't familiar with nginx and apache, even for most medium sized production deployments.
But, if you've got a team that's comfortable with nginx or apache, and doesn't find themselves spending an undue amount of time fiddling around with managing them, I don't personally believe there's any reason to switch from them to caddy.
Have been running Caddy in production for a year with no issues and much rejoice over my 3 loc config which I can literally memorize.
All praise to Caddy's devs gj
The telemetry server was shut down months ago and v2 has no telemetry client. Although, the counts we gathered were informative. We learned a lot about MITM activity on the Internet, the health and maturity of TLS clients, and how many connections Caddy has secured (trillions, if you're wondering).
As for whether things have gotten worse, I suppose that's up to you!
We ask people to let us know if / how they’re using our stuff, but very few people do. It’s a tough problem.
We collected some good measurement data -- anonymous, technical, non-personal, etc -- but the terabytes added up and the academic community didn't seem particularly responsive to it in the end. The "opt-in" / "opt-out" depended on how you built/obtained Caddy, but essentially we made it a compile-time decision so that we could reduce biases from the data. By deciding on the download page whether telemetry was enabled, we could also know how representative the data actually was: otherwise it'd be meaningless.
We also wanted to know how widely Caddy was being used. Telemetry was mostly just counts of things, so that's how we know that Caddy has secured trillions of HTTPS requests and managed millions of certs. But it was expensive to run.
The "ads" in headers were intended to be a friendly nod to our sponsors who made it possible. It was a novel idea. I thought it was a good balance of non-intrusive and perfectly visible at the same time: developers who were peering into HTTP requests would see the headers and our sponsors would get some benign recognition from their target audience, while nobody else would see them. It also was supposed to encourage purchasing commercial licenses, which didn't have that header. The licenses were necessary to continue funding a desperately underfunded project.
Needless to say that didn't work out and the only reason the Caddy project didn't shut down entirely is because corporate sponsors (Ardan Labs and one other to be announced probably next week) believed in the project enough to pick it up.
So anyway, I also got college credit for implementing an Internet measurement system, which was really fun and interesting. And as mmalone said adjacently, open source projects really need to know what kind of usage they're getting. With no way to engage customers except at their voluntary discretion, it's impossible to know how to improve the project. Open source is, by definition, an open feedback loop. It only closes if users come back and provide information.
A lot of other major open source projects or free software ship telemetry, even on by default sometimes -- see Windows, Chrome, Ubuntu, Firefox, VS Code, macOS, and countless others. Yet nobody cares.
But having that information was critical in shaping the development of Caddy 2, FWIW.
People care about each and every one of those. It's disingenuous to pretend that Caddy was singled out.
It's not a stretch to say that other projects didn't get as much hate for it. It really hurt the project's momentum when there was such a strong reaction against it.
Those are projects funded by 800-pound gorilla companies/foundations. It's either their way or the highway and people complain all the time, some have even forked the code and ripped out telemetry (Vscodium). Homebrew got a response comparable to Caddy but they screwed up even worse because they use Google Analytics.
Because you want to know what people are actually using in your projects. I've never put telemetry in anything I use, but I 100% understand why. It's hard to prioritize feature development if you don't know how people are using what you're making, especially if the users aren't very vocal. Without people talking to you and without telemetry on what people are doing, you might spend months worth of man-hours on something no one cares about.
For projects where all you do is build out the things you specifically need this isn't an issue - all you care about is making the stuff you're doing work, and people getting benefit out of things is a bonus but not necessarily your goal. People will submit pull requests or feature requests, or they won't. No skin off your nose. Not all projects are developed this way, though.
It's not completely the same as telemetry, but as an administrator of a free open source web service, I have to say that web page analytics (1st party for us) are a huge motivator, when you see how many people from all around the world are using it.
My experience as a contributor has been nothing but positive. But if anyone's had a bad experience, I'm sure the Caddy team is more than willing to talk it out and reconcile.
Caddy 2 is a fresh new server experience. Some things might take getting used to, like having every site served over HTTPS unless you specify http:// explicitly in your config. But in general, it will feel familiar to v1 in a lot of ways. If you've used v1 before, I recommend going into v2 with a clean-slate using our Getting Started guide: https://caddyserver.com/docs/getting-started
I'm excited to finally get this out there. Let me know what questions you have!
In terms of speeding up adoption of Caddy 2, it may be useful to have a list somewhere of the concrete improvements between the two (as I'm sure there are many). A (very) brief look and search around only yielded this article[1] which referenced another link of improvements that now 404s[2].
Cheers,
[1] https://caddyserver.com/v1/blog/caddy-2-ready-for-developers
[2] https://github.com/caddyserver/caddy/tree/v2#list-of-improve...
That blog post is pretty old. We do have a current upgrade guide here in our docs: https://caddyserver.com/docs/v2-upgrade
We also have a (sort of deprecated) wiki with some (slightly outdated) high-level improvements listed: https://github.com/caddyserver/caddy/wiki/v2:-Improvements
The official documentation at https://caddyserver.com/docs/ is up to date, and not deprecated.
Help? :D
Deb files will be just fine.
When you can make dpkg files, making an apt repo is trivial (using reprepro shipped with Debian)
And when it’s your repo, you make the rules about packaging what dependencies into which file(s)
But by having an apt repo (that users would subscribe to), you make it much easier for users to keep up to date
https://caddyserver.com/docs/install#debian-ubuntu-raspbian
One thing I stumbled upon the other day was that I (and google apparently, too) have trouble navigating the new website.
I find it much simpler to find relevant information on the old site.
Anyway, keep up the good work and stay healthy!
Not gonna lie though, there's more to learn. V2 is a powerful machine -- so do expect that there will be some reading. Once you know how it works, it's easy to use. Very simple configs are possible, etc.
Memory safety is a class of guarantees certain software offers you against certain vulnerabilities. Software written in C is generally considered "memory unsafe" since it's hard to write correct C code when managing with memory, so it's easier to find exploits in them that cause them to reveal secrets.
Go software has stronger memory safety guarantees than C programs like OpenSSH and Nginx. So that's one big benefit of using Caddy.
As it happens, someone in the audience here is writing a SSH app for Caddy, so you have a pure Go SSH server that is less vulnerable to those class of attacks.
Apart from memory safety, can the SSH version of caddy impose better defaults that OpenSSH doesn’t right now? Maybe TLS certs, security key support, etc?
I know I'd ideally like easier SSO integration, for example. Or provisioning of users.
Is there an admin UI for v2? Seeing the configuration changes, it was the first idea that popped, would be great for the selfhosted community - maybe someone picks it up as a side project if it's not in scope.
I am very interested in the prospects of Starlark in Caddy 2. With an integrated scripting language Caddy could on its own be a replacement for OpenResty or Apache with mod_mruby. The preliminary implementation was removed in the beta phase with a note saying it would have to wait for when the project was financially stable [1]. Do you still plan to integrate Starlark if the project is a financial success?
[1] https://github.com/caddyserver/caddy/commit/8b28c36d480070c8...
Am curious what tools you use for testing Caddy: generating traffic for load testing, web fuzzing, net simulation, etc?
CommonName has been deprecated for years, so all of Caddy's certificates (and all of Let's Encrypt's) are SAN certificates. Caddy manages single-SAN certs, in accordance with recommended best practices: https://docs.https.dev/acme-ops#use-one-name-per-certificate
(We learned from experience that single SAN scales better and is less prone to troubles. For example, Caddy sites were not affected by the recent Let's Encrypt revocation incident because it manages only single-SAN certificates. And frankly, that shouldn't matter since you don't have to manage them.)
I was asking because right now my nginx/certbot setup requests a single certificate for all my domains, and I would like to have them separate.
(I'm sure you can do that with certbot, I just haven't tweaked my setup, yet)
So yeah, Caddy uses one name per certificate by default, and there's no way to change that.
Look at certbot's -d parameter, this does not have anything to do with nginx, apache, caddy or any other web server.
Example: We run a SaaS on our client's domains, so for every client, we have to run `certbot -d subdomain.client1.com,client1.ourdomain.com,staging.client1.ourdomain.com,...`
That means we run certbot with ~4 domains (we include staging and other subdomains) for each client we have. This is highly automatable.
Whether you use nginx autoconfiguration or not is up to you.
Does the nginx autoconfiguration allow separate certificates, or do you simply say "don't use that"?
I don't know too much about certbot's nginx plugin, I don't use it and don't see the benefit to be honest, we ran into problems with it (it didn't work for us because our clients have to set up a CNAME DNS entry, and that domain has to be included in the certificate)
We simply run certbot without the nginx plugin, and then have a config template for new virtual servers in nginx, based on a template. Certbot's nginx plugin would mess with our config constantly, changing from version to version, leaving artifacts, and we didn't like that.
[1] https://wiki.archlinux.org/index.php/Nginx#Managing_server_e...
We also think slightly more verbosity in the v2 Caddyfile is actually a good thing. Aside from more flexibility, it makes it a little clearer what's going on, and concerns are better separated.
For example, the reverse proxy is no longer in the business of rewriting requests: there are other directives for that.
In fact, we completely got rid of "rewrite hacks" that involved appending path components to the URI so they could be routed, then needing to strip those before proxying.
Request matching is very powerful in Caddy 2: you can still easily match on paths in-line with your handler directives, but you can also match on any other part of the request (client IP, query string, regular expressions, etc) with an intuitive @-block syntax: https://caddyserver.com/docs/caddyfile/matchers
Anyway, I think you'll feel right at home with it after a while. I've been using it for months now and it's so simple overall.
(I also use the JSON config directly on a couple of my servers, where the behavior I need to express is way more elegant and concise than in the Caddyfile.)
Basically, it is highly interoperable and allows us to build config adapters on top of it, so you don't have to use JSON if you really don't want to. Most people use the Caddyfile. You can use YAML or TOML or CUE if you prefer those. Or heck, you can use your nginx config: https://github.com/caddyserver/nginx-adapter
Shortly after upgrading to caddy v2, I switched to nginx, after spending way too much time trying to make sense of the new Caddyfile and docs (the thing I miss the most of the docs is the plugin list, along with the cors plugin, and in general the non-existant entry curve of Caddy v1). Tbh, I also miss the one-liner install script <3
I'm genuinely sorry for the project; I didn't mind at all any of the previous so-called shady behavior (I personally don't object at all against a developer trying to make a living out of his own open-source project); I've been using caddy for four years now, and I really hope to switch back to it, one day.
Thank you for your work, mholt! <3
We'll be glad to help you get set up!
The docs are still a work in progress, we'll eventually have a plugin registry like in v1 but that requires a bunch of work behind the scenes to make it happen.
I've already got the new build server mostly written -- should be way faster.
On the old site, there was a plugin documentation page that made it trivial to see what the plugins were called and what options they took.
I eventually asked my local copy of Caddy what plugins were available, which answered one of the two questions, but am I missing some kind of documentation page?
The interactive, custom build page is an unrelated, but also interesting topic. About a year ago, I was using abiosoft's Caddy Builder image to build a Caddy docker image with the plugins I wanted, but this no longer seems to work even for Caddy V1, and I don't think it has been updated for V2. It would be awesome if there were an official Caddy Builder docker image. Even if the website existed, I would still have to wrap that binary in a docker image somehow for my own use cases.
But, these are all just my personal opinions -- I'm sure you have your own priorities for the project, and I'm excited to see V2 has officially released!
You're in luck! :D
https://hub.docker.com/_/caddy
Note that the v2 images were _just_ pushed a few minutes ago, so the description might still point to rc3.
See "Adding custom Caddy modules" on that page for instructions on how to use the `caddy:builder` image.
It's not worth delaying the release for that, especially when we have tools like xcaddy that make it easy to build Caddy with plugins, no need to touch any code: https://github.com/caddyserver/xcaddy
Right now, the process is: find a plugin you like, then run:
But yeah, it will get easier with time. We'll get there.If you're navigating from the homepage, click on "Documentation" at the top, then click on "modules" on the left navbar just above "JSON Config Structure". That list is generated directly from the source code, so it is the definite list of what's included by default and supported by the native JSON config.
The other way is, as you found out, to run `caddy list-modules` to get the definitive list of what's packed into the binary.
The biggest feature in Caddy 2, from what I can tell, is the new HTTP config API, and I'd love to see a practical example of what could be done with that.
A few of those are third party maintained, and are plugins. They're listed on that page to point out what's possible.
There's a tutorial for the API in the docs that should give you a good idea of what you can do: https://caddyserver.com/docs/api-tutorial
Some adapters are 1:1 translations so they are just a few lines of code: https://github.com/caddyserver/json5-adapter/blob/master/jso...
True, the Caddyfile adapter is a bit of a beast because it is extensible itself -- but it's a lot cleaner and more flexible than the v1 implementation, where the Caddyfile was its native config syntax.
Believe me, this is a big improvement.
Anyway, you don't have to use the adapters if you don't want to. That's the beauty of it.
More info on the architecture and config adapters:
- https://caddyserver.com/docs/architecture
- https://caddyserver.com/docs/config-adapters
https://www.google.com/search?q=golang+%22asgi%22
That was my pet project for the past couple weeks, I'm so glad to see people appreciate it!
There's ongoing effort to package Caddy into Debian. Until then, this is the alternative as stopgap.
Thanks for pointing out
In case it's of interest, as a new visitor, I started browsing from your main v2 landing page, scanning down most of that information. Then I followed the prominent download and get started links near the top. I think I next went to the documentation area, and started browsing the links on the left, though I totally missed the "Install" link just under "Welcome" because my eye was drawn first to the Tutorials section and its getting started link, and then I went exploring from there on down.
I suppose that was an unfortunate combination of two things to miss. :-)
We'd love to get some help from people who are more comfortable with the debian ecosystem though! It's definitely something we want to achieve, but for the short term we had to make the decision to simply release the .deb via https://gemfury.com which provides us with free APT repo hosting for open source projects.
I understand why, but it’s indeed quite a pain.
Of course, that isn't helpful if you need to compile with extra plugins, but in that case you are probably using standard go tools and not apt packages anyway.
We were specifically told that we would need to have all our dependencies packaged. Debian needs to be able to validate that it was built entirely from source code from library packages, not from a binary built externally.
Unless I'm mistaken, a malicious package with the same name as a critical system package could be added to that untrusted repository by an attacker who had compromised it, and if the version number is higher than that of the 'official' package and your Apt priority config prioritises it, your system would download and install it without any verification.
This is a fairly niche attack vector that may not be considered a significant risk in some environments, but it's one to consider when establishing your threat model and risk appetite.
As a side note, it's worth noting that GPG signing packages is not a silver bullet either, especially if the signing keys and administrative access to the repository fall within the same security boundary (e.g. developer doing both from their PC without any segregation/sandboxing). However, it has proven to be a robust method so far, and definitely beats explicit [trusted=yes].
(Thank you, Let's Encrypt)
Feel free to post in our forums with questions! https://caddy.community
For Version 2, we've done a lot of work with Matt and the solution we've now reached is:
1. DynamoDB as storage for certificates, allowing sharing between servers without regenerating during an issue with an availability zone
2. Multiple Caddy reverse proxy servers in different regions
3. AWS Global Accelerator to route the user to the closet server to them
It's so great because we can also proxy our CDN through Caddy (yes, it needs to go through the user's custom domain), and we've got insanely fast response times on that.
We've load tested the proxy servers and they can handle an incredible amount, we're very pleased with the solution.
I will be publishing and sharing an article soon, detailing my journey and our final solution.
We’ve done this exact same thing and have a technical writeup here: https://ohdear.app/blog/how-we-used-caddy-and-laravels-subdo...
Tl:dr: very easy to do, Caddy takes care of 90% of the complexity.
More info:
- https://caddyserver.com/docs/caddyfile/concepts#addresses
- https://caddyserver.com/docs/automatic-https#overview
Counterpoints:
- https://twitter.com/mattiasgeniar/status/1249751608721911814
- https://twitter.com/JackEllis/status/1257396020662673414
There were some gotchas in converting the Caddyfile to the new format, but I eventually sorted it out. Caddy 2's method of updating the config made the iteration process a lot faster.
I'll admit I got bit by a few differences between the v1 Caddyfile and the v2 Caddyfile too. Namely forgetting * at the end of a path _prefix_ matcher. (In Caddy 2, path matches are exact, unless you suffix with a *.)
[0] https://caddy.community/
It's still very much a work-in-progress and could use help to complete.
> If the latest OCSP staple indicates that a revocation, Caddy automatically replaces the certificate with a new one.
Thousand line nginx config files reduced to nothing. You can literally run it "configured" from the command line.
Congrats on the release!
You mean like this one? https://github.com/caddy-dns/cloudflare
(I'm using it today, works great. Assuming that's what you meant.)
Error during parsing: getting DNS provider module named 'cloudflare': module not registered: tls.dns.cloudflare
Using latest docker image.
Edit: You also need to plug it in.
EDIT: I have just seen your edit. Will try and report back.
Thanks for your time!
Also, see the docs on docker hub: https://hub.docker.com/_/caddy/
You'll need to build Caddy with the builder image to add the DNS plugin, see the section titled "Adding custom Caddy modules".
Thanks for the nice piece of software. I like Caddy a lot. I would like to better integration with docker in terms of automated exposure of docker services based on labels for example. Similar how traefik does. That would be really nice to have.
You might be interested in this discussion: https://github.com/lucaslorentz/caddy-docker-proxy/issues/13...