Show HN: An ETL engine with hard memory guarantees (github.com)

2 points by logannyeMD ↗ HN
This is an external-memory ETL engine that enforces hard memory bounds (as opposed to systems that can only try to stay within limits). It's the result of playing around with a recent idea from complexity theory, tinkering to find practical applications of where efficient time-space simulation could be useful.

It enforces a strict memory cap via RAII tracking and scheduling, so you can process arbitrarily large datasets with a fixed, small memory footprint. Operators automatically spill to disk when needed. Might be helpful for serverless and edge environments where you need predictable resource usage.

0 comments

[ 3.0 ms ] story [ 12.6 ms ] thread

No comments yet.