The introduction seems human written, but the slightly technical bits after that are clearly AI written, which makes sense, since the system that's being described was written by AI as well.
It would be been a more thorough analysis if they had accounted for the engineering teams’ salaries in time devoted to recreating this tool and maintaining it, rather than just tokens.
In the end, it would likely work out to more like 50% cost savings. Still good, but certainly more expensive than this article ultimately claims
You should also factor in that you have way more flexibility in the dashboards you create and everybody can build them easily, just by describing what they need.
> We already run everything on Kubernetes, so deploying a service was mostly a matter of adding a Terraform module and a Helm chart.
Haha "we run everything on kubernetes, so deployment is a matter of using the least logical complement to it, and the lowest budget, flakiest and overbuilt alternative to plan manifests with none of the benefits of an operator"
Enterprise friends: what is snowflake and why did it have fine grained permissions for a whole company? I didn’t understand it from the snowflake website.
ex: you don't want all users to be able to access finance's tables, else you might have folks trading stock on information they're not supposed to be privy to.
I built something similar in my previous job. The part that we were never able to figure out is how to guarantee that everybody is looking at the right numbers. Even if you have a comprehensive semantic layer, there are still a lot of things that can go wrong in putting metrics on a dashboard. This is why analysts are still needed, IMHO.
Over a decade ago i worked at a company with a BI tool that was exactly this, minus the AI of course. it was just PHP pages backed by embedded SQL built by a non engineer. anytime someone wanted a different slice of data, the person just copied an existing page and swapped out the sql and the column names. URL query string was used for the different parameters of the SQL statement, i forget if it was open to sql injection but it was an internal tool only. even after implementing a BI tool everyone still used the PHP pages because it was faster and easier to just bookmark the specific page you want with the data you wanted.
much like low code tools, most people dont want to mess with self serve BI. AI might help with that but i think the tool in the article opens the door for people creating slightly different versions of the same thing. also, claude is awful at visualizations. every one i see looks the same, crammed full of bad charts and too much text, poor color choices, and no real value in terms of story. still, this is an interesting implementation of that concept.
if you want to operate this 'bucket of HTML files' over time, my startup https://valet.dev gives you the tools to host, govern, enforce policies, and secure what vibe coders create at your company.
The world of who can create software is growing (hello citizen developers!) but the challenges of operating and maintaining software is just as hard.
Building is easy, supporting and sustaining is harder especially in light of things like all the CVEs. My startup valet.dev is a platform for skills, websites, apps, and agents that is solving for this by baking governance and sustainability into the system. All engineering teams need this, but as we see the rise of citizen developers bring productivity gains to their companies the need for collaboration, security, and sustainability builtin to the environment they work in is even more important than ever.
> A BI tool is mostly a way to turn a query into a picture, and a picture now costs about $0.50 of tokens. It gets hard to justify a $150k licence for that part
That is an awfully simplified view of BI. In my experience, most companies struggle to define what is important, what needs to be measured, and how those measurements should be used to improve business.
While businesses try to work these details out, they try getting their analysts to in the author's words print out good pictures. But the BI tools really exist to make it easy for analysts to navigate hairy internal situations.
Producing a picture or HTML more easily is not going to solve the decisioning problem. If anything it might make it worse.
In the end, it's going to cost a lot more than 50 cents a picture.
Yeah, I personally draw a distinction between data viz and BI. AI has definitely changed the former, it's pretty cool to be able to design visuals with claude design: you're not bound by what libs can do, html + svg lets you visualize data exactly how you want (and you can even get fancy with 3d stuff, with the pitfall of just showing eye candy with no real insights). I've built some fun stuff like that myself (also with duckdb-wasm).
But BI is more than just visualization: you need stuff like solid data models and semantic layers, tools that help enforce data governance (so that the same metric doesn't get re-defined in 12 different places, possibly with slight variations each time), proper access control (with RLS/CLS). Sure you can probably vibe-code your own BI tool, but it's gonna take quite a bit more work than just the viz part.
I have loved D3 for more than a decade but I'd never recommend it to a client because maintenance requires a large learning curve. Not any more, ${YourAIAgent} + D3 is awesome.
For ages the data industry has obsessed over tooling when really the biggest impact comes from the organisational/people aspects - achieving consensus on metrics, dimensions, key entities. In short, governance. You can't buy that you like can a SaaS product.
You are making an argument (that I can faintly see the outlines of) but eliding over the meat of it. I think you need to define “hairy internal situations.”
It reads to me like their first version (sharing the output html) essentially skipped their permission layer. That seems like a dangerous decision if someone can share the sales data analysis openly with users not allowed to see such data?
For a related use case, I have been using claude to add or change features in my own tensorboard log reader. The flexibility of being able to change anything at any time so I can see what I want to see is really nice.
I've always preferred local monitoring tools, but tensorboard is seemingly just on life support at this point.
I work at Retool, and probably the majority of our business today is "replacing SaaS with citizen-developed apps". I think that a lot of these CRUD-y SaaS products (which is, really, what BI is) will disappear over the next few years. The bottlenecks I see:
* Data quality — we've seen Databricks / Snowflake schemas with 10^5 tables. Just putting a LLM on that doesn't work - you do need some sort of semantic layer. This is especially bad since LLMs are like slop cannons — if you ask it to prove X with the data, it will build you an app / visualization that proves X.
* Deduplication — it turns out many people want moderately different views of the same data. How can you host everything centrally and DRY everything out such that you can guarantee correctness?
* Writing data back — this is where it gets really hairy. Dashboards that only read data from Databricks is dangerous insofar that data can leak. But once you start needing to write data back, you need access to APIs / MCPs... and then you want some sort of authorization layer (since many SaaS apps don't have great authorization within themselves).
Obviously we work on quite a bit of this at Retool, but we are always looking for ideas on how to make this better. We believe that many things should be in a platform (e.g. centralized data connectivity, authentication, authorization, etc.) setup by engineering, and then non-developers can go crazy generating safe apps.
That is a lot of tables for one schema. Maintaining so many tables seems like a security vulnerability in and of itself from risk PoV.
I think your prediction makes sense. There needs to be a lot of guardrails and you need a fully integrated platform/stack on the backend to protect the user from doing dangerous things because the AI ain't gonna tell them when something is dangerous or doesn't perform at the scale they need
On write-back: at Kairos Agéntica, where I work, we ended up where your last
paragraph is. The ERPs our customers run mostly don't have authorization
fine-grained enough to hand to an agent, so we decided it has to live in the
platform. What runs in production today:
Reads and writes are separate. Agents read through a semantic layer; writes
only exist as narrow MCP tools ("send this payment reminder", not "call the
ERP API").
A run that calls a write tool pauses until a person approves that specific
action, unless the tool is explicitly marked auto-approve, and what executes
is exactly what was approved, arguments included.
Who can approve is a per-agent rule over the user's groups, and it can
differ by tool, so the approval step is the authorization layer the source
system doesn't have.
What it doesn't fix yet: the MCP server still holds a credential broader
than any single approval, so tool scoping carries a lot of weight.
The next thing we're looking at is approval fatigue: an LLM judge that
clears routine actions. Has anyone run something like this? What approaches
worked for you?
We were paying $40K/year for both salesforce and salesloft. Both got built into a simple app for the sales and SDR team. The new app is way clean and faster than Salesforce. We even hired a consulting agency to setup/manage Salesforce and its reports. That consulting agency is also gone.
For some use cases, and when everyone has access to all the systems, and data volumes allow reasonable perf, I could see this working well.
For things like customer-facing analytics, there needs to be a deployment layer for AI-native artifacts that supports ongoing operations & maintenance. If you want a conversational data agent to answer questions and reduce custom analytics requests, that's more to manage and maintain.
If you watched the Claude Dashboards announcement, they said legacy bi will be the deployment layer for AI artifacts. I don't buy it. Legacy BI was terrible at deployment in the first place.
We've built a performant, browser-native alternative to deploy and scale AI artifacts at Ridge AI (ridgedata.ai).
Fascinating to watch all of this evolve in real time.
Ayeee! We're actively doing this right now at my company. One of the more clever engineers has tackled replacing a $300k/yr Looker subscription, which we've paid for a decade, with a homegrown implementation sponsored by Claude.
And it seems like we're succeeding! To be fair, we weren't making good use of Looker's features and so the problem was easier than expected... but the core idea was validated in a couple of weeks, we've had just one person on the project part-time for a couple of months, and we told Looker (well, Google) we won't be renewing yesterday.
We're just doing a 1:1 conversion to start with the hope that our customers barely notice the change. Afterward, we'll re-envision our dashboards with the new found flexibility of not serving them via iframe.
It seems like this is a relatively simpler case, where everyone sees the same content. Oftentimes, people want dashboards to query as the current user, some user group, or the people below the current user in the org chart. That means that you need to run queries dynamically, and generate the chart at runtime, rather than just hosting HTML.
71 comments
[ 3.9 ms ] story [ 44.5 ms ] threadIn the end, it would likely work out to more like 50% cost savings. Still good, but certainly more expensive than this article ultimately claims
Haha "we run everything on kubernetes, so deployment is a matter of using the least logical complement to it, and the lowest budget, flakiest and overbuilt alternative to plan manifests with none of the benefits of an operator"
I guess I'm not reading this...
“Looker is Google Cloud agentic business intelligence and embedded analytics platform.”
Something of the past according to the author.
It doesn't have to be intentional, it just needs to be a sufficiently complex, messy real-world data.
https://xkcd.com/1138/
much like low code tools, most people dont want to mess with self serve BI. AI might help with that but i think the tool in the article opens the door for people creating slightly different versions of the same thing. also, claude is awful at visualizations. every one i see looks the same, crammed full of bad charts and too much text, poor color choices, and no real value in terms of story. still, this is an interesting implementation of that concept.
The world of who can create software is growing (hello citizen developers!) but the challenges of operating and maintaining software is just as hard.
Now it's just a git repo.
That is an awfully simplified view of BI. In my experience, most companies struggle to define what is important, what needs to be measured, and how those measurements should be used to improve business.
While businesses try to work these details out, they try getting their analysts to in the author's words print out good pictures. But the BI tools really exist to make it easy for analysts to navigate hairy internal situations.
Producing a picture or HTML more easily is not going to solve the decisioning problem. If anything it might make it worse.
In the end, it's going to cost a lot more than 50 cents a picture.
But BI is more than just visualization: you need stuff like solid data models and semantic layers, tools that help enforce data governance (so that the same metric doesn't get re-defined in 12 different places, possibly with slight variations each time), proper access control (with RLS/CLS). Sure you can probably vibe-code your own BI tool, but it's gonna take quite a bit more work than just the viz part.
Today that’s just a chat session, see https://reportr.com
For ages the data industry has obsessed over tooling when really the biggest impact comes from the organisational/people aspects - achieving consensus on metrics, dimensions, key entities. In short, governance. You can't buy that you like can a SaaS product.
I've always preferred local monitoring tools, but tensorboard is seemingly just on life support at this point.
https://github.com/dleemiller/BlernsBoard
" look how i vibecoded this app and save $1.99"
* Data quality — we've seen Databricks / Snowflake schemas with 10^5 tables. Just putting a LLM on that doesn't work - you do need some sort of semantic layer. This is especially bad since LLMs are like slop cannons — if you ask it to prove X with the data, it will build you an app / visualization that proves X.
* Deduplication — it turns out many people want moderately different views of the same data. How can you host everything centrally and DRY everything out such that you can guarantee correctness?
* Writing data back — this is where it gets really hairy. Dashboards that only read data from Databricks is dangerous insofar that data can leak. But once you start needing to write data back, you need access to APIs / MCPs... and then you want some sort of authorization layer (since many SaaS apps don't have great authorization within themselves).
Obviously we work on quite a bit of this at Retool, but we are always looking for ideas on how to make this better. We believe that many things should be in a platform (e.g. centralized data connectivity, authentication, authorization, etc.) setup by engineering, and then non-developers can go crazy generating safe apps.
I think your prediction makes sense. There needs to be a lot of guardrails and you need a fully integrated platform/stack on the backend to protect the user from doing dangerous things because the AI ain't gonna tell them when something is dangerous or doesn't perform at the scale they need
Reads and writes are separate. Agents read through a semantic layer; writes only exist as narrow MCP tools ("send this payment reminder", not "call the ERP API").
A run that calls a write tool pauses until a person approves that specific action, unless the tool is explicitly marked auto-approve, and what executes is exactly what was approved, arguments included.
Who can approve is a per-agent rule over the user's groups, and it can differ by tool, so the approval step is the authorization layer the source system doesn't have.
What it doesn't fix yet: the MCP server still holds a credential broader than any single approval, so tool scoping carries a lot of weight.
The next thing we're looking at is approval fatigue: an LLM judge that clears routine actions. Has anyone run something like this? What approaches worked for you?
For things like customer-facing analytics, there needs to be a deployment layer for AI-native artifacts that supports ongoing operations & maintenance. If you want a conversational data agent to answer questions and reduce custom analytics requests, that's more to manage and maintain.
If you watched the Claude Dashboards announcement, they said legacy bi will be the deployment layer for AI artifacts. I don't buy it. Legacy BI was terrible at deployment in the first place.
We've built a performant, browser-native alternative to deploy and scale AI artifacts at Ridge AI (ridgedata.ai).
Fascinating to watch all of this evolve in real time.
[0]: https://news.ycombinator.com/item?id=50200997
And it seems like we're succeeding! To be fair, we weren't making good use of Looker's features and so the problem was easier than expected... but the core idea was validated in a couple of weeks, we've had just one person on the project part-time for a couple of months, and we told Looker (well, Google) we won't be renewing yesterday.
We're just doing a 1:1 conversion to start with the hope that our customers barely notice the change. Afterward, we'll re-envision our dashboards with the new found flexibility of not serving them via iframe.
Crazy times!