Show HN: CocoIndex – Open-Source Data Engine for Dynamic Context Engineering (github.com)

Hi guys, I'm back with a new version of CocoIndex (v0.3.1), with significant updates since last one. CocoIndex is ultra performant data transformation for AI & Dynamic Context Engineering - Simple to connect to source, and keep the target always fresh for all the heavy AI transformations (and any transformations).

Adaptive Batching Supports automatic, knob-free batching across all functions. In our benchmarks with MiniLM, batching delivered ~5× higher throughput and ~80% lower runtime by amortizing GPU overhead with no manual tuning. It you use remote embedding models, this will really help your workloads.

Custom Sources/Target With custom source/target connector, you can now use it to any external system — APIs, DBs, cloud storage, file systems, and more. CocoIndex handles incremental ingestion, change tracking, and schema alignment.

Runtime & Reliability Safer async execution and correct cancellation, Centralized HTTP utility with retries + clear errors, and many others.

Btw, we are also on Github trending in Rust today :) it has Python SDK.

We have been growing so much with feedbacks from this community, thank you so much!

We have lots of releases since last time i shared the project, a few recent ones (each contains 20+ releases) : - https://cocoindex.io/blogs/changelog-0310 - https://cocoindex.io/blogs/cocoindex-changelog-2025-10-19 - https://cocoindex.io/blogs/cocoindex-changelog-2025-08-18