HN Explorer

Show HN: Cptn.io – open-source integration platform (cptn.io)

139 points by kc10 ↗ HN
Hi, I am Krishna Thota. I am building an open source integration and data platform(https://cptn.io). The product is MIT licensed and the repo is at https://github.com/cptn-io/el-cptn.

I have started on my startup journey an year ago and launched a monitoring platform called DevRaven. Unfortunately the product did not takeoff as expected. That story is for another day.

But during the course of building the product, I have built several integrations leveraging MQs and Cloud Functions. While building and deploying Cloud Functions for happy paths is easy, I had to monitor logs for failures, build retry mechanisms or manually process failed events, keep instances running to prevent cold start timeouts. It can also get expensive with charges for MQs, server time for running cloud functions etc and costs can be unpredictable.

I thought of building a platform where I can build integrations quickly, have the ability to look at incoming/outgoing events, look at logs, retry any failed events etc. And finally, predictable costs for running the infrastructure. cptn.io provides all these capabilities and more. You can build pipelines to integrate with any cloud services, send data from your backend to data warehouses, listen to web hook events etc. The platform can be integrated into any stack by sending events to HTTP end points.

Instead of trying to build a business first or launch an open source product under restrictive licenses, the platform will be available under MIT license so any user or customer can use it. There is no ee folder or complex dual licensing and I am also committing to releasing SSO under MIT. The plan is to offer a managed service in the cloud at a later time, accept sponsors for prioritizing features for enterprise customers and charge for enterprise support.

It should take less than 5 minutes to get the platform running on your machine. Welcome any feedback, feature requests, PRs and bug reports.

41 comments

[ 3.5 ms ] story [ 80.9 ms ] thread
dgwight
Is this an open source Segment Analytics competitor? I liked the idea of segment, but the pricing jumped way up after the free tier and I had to remove it
kc10
Technically the product can be used as CDP platform if a client library is capable of sending tracking events to the Source URL and you just want to pipe them to your buckets/data warehouses.
lagniappe
This is pretty cool! The UI example reminds me of Yahoo Pipes (rip).
codegeek
So is this like an Open Source Zapier or at least with similar ambitions if not quite there yet ?
kc10
Yes, over time with contributions from community the app library will grow and pipelines can be built with ready to use apps.

For complex cases or specialized integrations, you can use any npm library to build your flow.

You can integrate into your stack with HTTP API and we can build SDKs for popular languages so it's just a function call to send the events.

tootie
Seems more like open source fivetran. For loading data into a warehouse.
robertlagrant
I worked with Software AG webMethods for quite a while and always wondered about building something like this. Nice job.
badrequest
How does this compare to, say, RudderStack?
kc10
It can work similar to RudderStack if we build a JS library that can send tracking events into the platform. But I think Rudderstack has more features around ETL and Reverse ETL which do not exist today in cptn.

RudderStack is AGPL while cptn is MIT.

nkmnz
What differentiates this from n8n?
kc10
I haven't used n8n before, but n8n looks much more mature, feature rich and no code capabilities. You can build any workflow in cptn as well, but most of the workflow is built via scripting.

Licensing "seems" to be restrictive if you want to use for commercial purposes (I am not quite sure about Sustainable Use License).

sneak
n8n is not open source.
BionicPimp
I like the UI. seems pretty straight forward. Best of luck.
henricourdent
Hi! In what does this differentiate from Windmill.dev? The license seems to be more permissive (AGPL vs MIT) but there isn't any Python support, and less integrations (for now ?) :)

Wishing you the best of luck!

https://github.com/windmill-labs/windmill

kc10
Thanks.

I think there are a number of platforms for building workflows and integrations today. Some of the platforms serving specific niche (e.g. RudderStack chose to serve CDP, airplane for building internal dev tools). But in the end all of them act on a trigger to transform inbound event data and send it to a destination end point.

Windmill looks cool with support for multiple languages, no-code editor and all the OOTB integrations.

cptn is still evolving - it needs to have more integrations, UX flows need to be improved, it has to get better from Ops perspective.

born-jre
Hmm.. Gives me early vibes. Which reminds me, maybe I should start promoting my thing[0] too

best of luck

[0]: https://github.com/temphia/temphia

kc10
Thanks. It's not v1 yet.

It's about 3 months in the making and needs improvements around ops, tests. I am hoping to make progress with adding tests in the coming days and provide monitoring capabilities.

jamesmcintyre
Your project looks really interesting. I've been looking into many different "plugin" architecture options (mainly with the goal to run untrusted js/ts code) and your extended readme sounds interesting.

But i can tell it's early so i'm not fully seeing how this would be used (like an example folder).

- I too have tried to think of how plugins could be "full-stack" with both sandboxing on BE & FE via iframe but i'm not seeing in your repo the code that handles the FE sandboxing? (I could be totally looking in the wrong place)

- I looked into using wasm "containers" as well- does your architecture have a scalable way to optimize "cold boot" of a container for near-instant run/render?

Again, cool project and good luck! I think there's so much room for innovation in this area of making micro-functionality truly portable & composable!

born-jre
thanks

yeah making core framework in fast low level language and making scripting/embed language on top is such underexplored area i think.

i have removed suborigin bashed frontend sandbox for now, inline iframe is just faster to iterate, i will bring back suborigin later. iframe related code would here.

https://github.com/temphia/temphia/blob/main/code/frontend/l... https://github.com/temphia/temphia/blob/main/code/frontend/e...

suborigin code is also same but template part would be in go template and you will need to setup (<wildcard>.myapp.com) domain name.

no i donot do use any wasm runtime snapshoting, i do cache the instance once they are run though so yeah first one will have latency.

i will try to post some demo/examples of app running later on, maybe. ¯\_(ツ)_/¯

gigatexal
is this a sort of heroku? this gives me pause because i am dubious about building on a platform's PaaS as it is now I have this as an option but they (AWS, GCP, etc) are likely to be around, but you're just you.

So is this something we can self host or... is there a hostable solution you want to push folks to?

kc10
It is self-hostable today. Just clone the repository, update the .env file and run "docker compose up" to get the instance running.
nico
Can you buffer requests to Twilio dynamically based on feedback?

ie. Can the system be fed back number of undelivered messages and adjust throttle accordingly?

kc10
There is no throttling today based on outbound failures. But I think it is a pretty common use case for handling destination failures.

Something like - if x or x% events are failing in a pipeline, pause the pipeline for y minutes or until some one unpauses it.

I can look at adding it.

nico
That would be amazing
replwoacause
No PowerShell support?
kc10
Only javascript is supported for building transformations and destinations. But you can use any npm library in your script, platform will automatically download it at runtime and executes your script.
culopatin
We use a platform called Actions Express that I’m not most fond of. Any ideas of integrators with SeviceNow plugins?

I’m basically doing web calls and parsing data, but the ServiceNow plugin is nice to have.

kc10
Not sure I understood the use case. Are you looking into integrate into SN platform? You can add a pipeline to integrate with SN platform using OOTB rest api or add scripted rest end points for custom integrations.

A ServiceNow integration is on my radar. But if you can provide more details I can definitely try to address it when I build the app. Please feel free to log an issue in github if you can provide any details or send an email to me (email in my profile).

deevus
Can this work for building pipelines for specific users? E.g. our customers can connect their CRM to our platform.
kc10
There are a couple of ways to address this:

1. Create a Source for each customer and send customer specific data to each source. Configured pipelines for each source will receive the event data and processes it.

2. Create one source with multiple pipelines connected to it. In each pipeline, you can drop processing the data based on any conditions.

#1 is preferred so you are sure each customer's data is only reaching the pipelines configured for that customer data.

deevus
Can this be done by API?
kc10
The entire backend is available via REST interface. I am looking to support finer API permissions via auth tokens.

But the APIs are accessible today with basic auth.

esafak
Krishna: From the comments, you can tell everybody is trying to position your product, so I suggest explicitly naming your competitors in your landing page, and explaining how you are better or different. Don't worry about driving prospective customers away; everybody already knows about the competitors.
kc10
Thank you.

Fair feedback. I definitely need to anchor myself to specific niche instead of saying it can work for anything and everything.

joshxyz
yes pls

i would love to know how it compares with n8n.io, segment.io, products like those

novoreorx
Cptn.io is ideal for quickly prototyping a custom backend for specific tasks. While n8n also offers this capability through its webhooks, Cptn is more lightweight and developer-oriented.

To illustrate, suppose I want to create an API that saves a Pinboard link and sends it to a Telegram channel when passed notes. In such cases, Cptn is a better choice than n8n.