HN Explorer

Show HN: Open-Source Webhooks Gateway for Platform Engineers (github.com)

119 points by rtukpe ↗ HN
Hey Friends,

Convoy is an open-source webhooks gateway. Webhooks continue to be hard at scale, and large teams require consistent tooling for inbound & outbound webhooks. Convoy enables developers to securely send, receive and manage millions of webhooks reliably with features like retries, rate limiting, circuit breaking, customer-facing webhook logs, zero downtime secrets rotation and more.

Since our initial launch [0], we've learned a lot about our users and made several important improvements we are excited to share:

- We are now a webhooks gateway! Akin to API Gateways that sit at the edge of your network to receive all API traffic and route them to the respective microservice, webhooks gateways sit at the edge of your network to receive webhooks from any backend service and route to client endpoints as well as ingest events from multiple providers and route them to the required backend services.

- We now have first-class integration with Pub/Sub Systems. Our users wanted increased deliverability guarantees. Your backend services write events to a queue/topic etc. Convoy consumes the queues, creates webhook events and dispatches those reliably to client endpoints. We currently support Amazon SQS and Google PubSub. On our roadmap, we have the following planned - Kafka, RabbitMQ & Nats (In that order)

- We switched our backend store to PostgreSQL. This improves the self-hosted experience tremendously. MongoDB was great for storing schemaless data, but was severely lacking in some important features for our users, e.g. transactions don't work on a single node; you need to bootstrap a replica set; also, we wanted to ship updates frequently, but the lack of migrations for schema & data changes slowed us down.

- We decided to go the open-core route of OSS monetisation because it offered us a good balance to serve our community and make enough money to run the company. Like GitLab, we hope to be good stewards of our community edition. Since our enterprise edition is simply a wrapper around the community edition with enterprise features like RBAC, Audit Logs etc., we are properly incentivised to continue making it excellent.

- Our Cloud platform is in the private alpha stage. Please contact us at founders@getconvoy.io to gain access!

Our mission is to serve hobbyist developers all the way to the most ambitious teams on the planet with a consistent and easy-to-use webhooks gateway for asynchronous communication on the internet.

We welcome you to try it out using our getting started at https://github.com/frain-dev/convoy#installation-getting-sta.... Share with us your webhook horror stories and give us feedback.

[0]: https://news.ycombinator.com/item?id=30469078

16 comments

[ 2.2 ms ] story [ 44.4 ms ] thread
toomuchtodo
Hello! Love it (webhooks are traditionally an unloved but crucial system component)! How would you say you compare to and differentiate from https://svix.com?

Edit: Thank you for the reply! Wishing you much success.

subomi
Thanks!

While we are pretty similar, several differences still exist today.

- Convoy is fully open-source, including our UI, adding multiple users, creating multiple projects, debugging & managing events. Svix has only its dispatcher open-source without a UI.

- Convoy directly integrates with PubSub systems like Amazon SQS & Google PubSub to ingest webhook events. Svix can only ingest events via a REST API.

- Convoy has features to send and receive webhooks. Svix has features to send webhooks only.

- Convoy is written in Golang, Svix is written in Rust. :)

avinassh
This looks great! I will give it a spin

> e.g. transactions don't work on a single node

any Mongo has this restriction

subomi
Yup, it sucked! Users had to spin up a replica set all the time. Wasn't fun.
MaBeuLux88
If you are in a development environment, you can set up a single node Replica Set. Some features like transactions and change streams rely on the oplog collection which only exists in Replica Set. A Replica Set can work with a single primary node. You won't have fail over of course, but it's not the point in a dev environment.
pojzon
Can this work as a replacement for zapier?
subomi
No, it can't. But it's a better replacement to receive the webhook before pushing it to zapier. Zapier doesn't give you good logs of webhooks to retry so you can re-trigger a workflow. Convoy does. :)
toomuchtodo
Consider Activepieces, Trigger.dev, or Automatisch for Zapier alternatives (if Zapier doesn’t fit your use case for whatever reason). All have recent HN threads if you use search at the bottom.

Convoy might still be a useful tool to front webhooks to the above mentioned no|low code alternatives, depending on your needs.

ashrafsam
Thank you for the mention. Founder of Activepieces here ready to help/support with whatever needed - we hang out on Discord all the time.
djbusby
Excellent! I was working on taking an internal project to FOSS that does exactly this. Thank you!
TeeWEE
What’s wrong with a autoscaling container behind an load balancer that has a http handler with optionally a pub sub in between? Security, retry etc are not hard. You could use a service mesh. Why not use k8s
subomi
It's the usual how difficult can this be to build and operate. But what we've found is it's a pain at scale.

- How do you ensure that one or a set of bad endpoints do not clog the queueing system and delay delivery of other working endpoints?

- How do you provide easy tools for developers across projects to debug and fix webhook delivery problems?

- How do you ensure this container is multi-tenant to serve multiple teams and no two teams can view each other data?

- How do you ensure flexibility if a new team decides to use RabbitMQ instead of Kafka?

Once you start to factor in all this requirements & more, you eventually build your own Convoy. :)

TLDR: Convoy is that container that is flexible and powerful enough to manage all your webhooks needs.