Show HN: Ktx – Open-source executable context layer for data agents (github.com)
We built it after going through the experience of building production-grade data agents for dozens of companies. If you’ve also tried building them, or simply tried using Claude Code or Codex on your data warehouse, you’ll know that accuracy is the #1 issue. Agents are great at generating valid SQL, but it’s not always correct SQL.
To cite a few examples of “agents gone wrong”:
- Stale column + hidden business rule: when preparing a board report, a finance analyst asks Claude Code for “ARR by customer segment”, it derives ARR from multiple tables (subscriptions, plans, accounts), then groups by accounts.industry. But CC doesn’t know that this industry column was deprecated a few months prior, or that past board reports excluded paused subscriptions from the ARR calculation
- Join fanout: a data analyst at a retailer uses their company’s internal agent to prep a product revenue deck for a QBR. The agent joins orders to order_items, then sums orders.total_amount_cents grouped by order_items.product_id. The SQL runs fine, but each order’s revenue is repeated once per line item, which most people will miss if most orders only have 1 item
- Missing attribution logic: a marketing analyst asks Codex “Which campaigns drove the most revenue?” Codex joins marketing_touches to users to orders and groups by utm_campaign. But since each order can have multiple touches before purchase, the same order can be credited to first touch, last touch, every touch, or every campaign the user clicked before buying. If the agent chooses the method that doesn’t match the team’s attribution logic, they’ll make suboptimal decisions
To solve this at first we gave the agent more context through skills + a wiki-style knowledge base. That gives it some useful extra context but still relies on it writing the SQL without incorrect assumptions.
The next solution we explored was implementing a classic semantic layer. That solves the executable part, but they’re such a pain to build and maintain since they were made for legacy BI tools. Plus as a standalone tool, they lack all the useful context from unstructured data sources like internal docs.
So we built ktx and split it into 2 parts:
1. Business context goes in Markdown wiki pages that are auto-ingested and auto-populated
2. Queryable definitions go into YAML files that define tables, row grain, joins, measures, dimensions, filters, and filter groups
That way, when an agent needs a metric, it asks ktx for a measure, dimensions, filters, and filter groups instead of writing the whole query itself. ktx’s planner chooses the join path, uses grain and relationship metadata, catches issues like join fanout and chasm joins, and compiles the warehouse SQL, while utilizing the extra unstructured knowledge it has access to.
ktx is Apache 2.0. It can ingest from most warehouses (BigQuery, Snowflake, Postgres & others), modeling tools (dbt, MetricFlow, LookML), BI tools (Looker, Metabase), doc tools like Notion, and corrections from user interactions.
Install manually:
npm install -g @kaelio/ktx
ktx setup
Or give this prompt to your agent:
Run npx skills add Kaelio/ktx --skill ktx and use ktx skill to install and configure ktx
We’d especially like feedback from people who’ve tried using Claude Code, Codex, or building custom agents on analytics warehouses. Where did they fail? And what did you try to make the answers more reliable?
19 comments
[ 36.8 ms ] story [ 194 ms ] threadI see this covers a file based approach, was there ever a consideration for a graph based approach?
For business context, how do you handle context that evolves over time?
It’s just that making such docs had next to no ROI 10 years ago. But today they are the difference between success and failure.
It’s fascinating - thank you
(and who writes the wiki / business rules ? Can they be reverse engineered from existing query stack? )
Sounds great - all the best
Edit: don’t take the above as criticism - just trying to fit new ideas into an old dog.
Does `ktx setup` need Claude specifically?
> LLM - picks a Claude backend. The default uses your local Claude Code session, so no API key is required. You can also use an Anthropic API key or Vertex AI.
I'm currently on Copilot at work
some observations: setup (without openai embeddings) was pretty easy. Nice! Not sure why it took ~19 minutes for a postgres database with 20sh tables. Also the data from pg_desc did not get into the wiki
I am going to try to embed it in our agentic flow and see how it fits. Very cool project though.
As for the descriptions: `pg_description` should be used already, we store raw pg descriptions in the semantic-layer/<connection>/_schema/*.yaml files under descriptions.db. there's also an AI generated description next to it. AI is the combination of the raw DB description + bunch of metadata we manage to collect about a table/column.
could you run `ktx sl` and see if the descriptions appear in the CLI ?
also feel free to join our community Slack - ktx.sh/slack We'll be happy to help with other questions