HN Explorer

24 comments

[ 3.1 ms ] story [ 37.0 ms ] thread
nwellinghoff
Nice. I like how you made it an easy to drop in proxy. Will definitely use this when debugging issues!
buremba
This is very neat! IMO inspecting the queries the agents run on the database is a better approach to understand how the code works, even more than reviewing the code.

I just tried and it works smoothly. For those who doesn't want to plug in the agents to their database directly, I built a similar tool https://dbfor.dev for the exact purpose, it just embeds PGLite and implements PG wire protocol to spin up quick PG databases with a traffic viewer included.

debarshri
We do something similar in adaptive [1].

What you can also do is add frontend and backend user to the proxy and then agents won't ever get the actual db user and password. You can make it throwaway too as well as just in time if you want.

Traditionally it was database activity monitoring which kind of fell out of fashion, but i think it is going to be back with advent of agents.

[1] https://adaptive.live

CodeWriter23
Really been wanting something like this. Thanks!
stephenr
Can you explain how this is a better option than just enabling the general log for MySQL as needed?
sneak
Was AI used to build this? It looks a lot like the kind of scratch-an-itch projects I have been grinding out with AI lately, in size, timeline, code, and function. If not, you are a very very productive programmer.

If so, would you mind sharing which model(s) you used and what tooling?

qxxx
Just tried it out. Works fine. Love it! I tried it with a wordpress site. It is showing hundreds of sql queries in one request (thats probably why that wordpress site is so slow lol)

What I would love to see here is:

- some kind of sorting: eg. by excecution time or order. So I can see the slowest queries.

- search/filter feature.

- faster scrolling with pgup/pgdown keys.

- maybe how often the same query was executed. I could check the code and maybe optimize the queries.

ahachete
IMO transparent proxies for observability are not the best pattern. And I speak from experience, we developed the Postgres plugin for Envoy [1], [2] and we use it in StackGres [3], among others, for this very same reason, observability.

There's two main problems with said proxies:

* Latency. Yes, yes, yes, they add "microseconds" vs "milliseconds for queries", and that's true, but just part of the story. There's an extra hop. There's two extra sets of TCP layers being traversed. If the hop is local (say a sidecar, as we do in StackGres) it adds complexity in its deployment and management (something we solved by automation, but was an extra problem to solve) and consumes resources. If it's a network hop, then adds milliseconds, and not microseconds.

* Performance. It's not that hard to write a functioning PG wire proxy (it's not trivial either). But it is extremely hard to make it perform well under high load scenarios. Most of the proxies I have seen crack down under moderate to high performance.

What's the solution then? The Postgres extension model to capture the metrics (we also experimented with eBPF, but it causes too many kernel-user space context switches when you can do the same in an extension without them), and a small sidecar to push the metrics out via a standardized protocol like OTEL.

[1]: https://www.envoyproxy.io/docs/envoy/latest/configuration/li...

[2]: https://www.cncf.io/blog/2020/08/13/envoy-1-15-introduces-a-...

[3]: https://stackgres.io

Edit: formatting

DavidKarlas
Could I put this into default docker-compose for developers, so when they work on project with micro services they can quickly inspect SQL queries if something weird is happening? How would this UI work with that scenario, feels like React frontend would better serve that purpose.
lasgawe
I run two production RDS instances by myself. I never thought I really needed something like this. What I usually did was just check the details on the AWS dashboard. Thanks for building this. I love it.
hankchinaski
what are the latency implications of going through this proxy? looks good