HN Explorer

Ask HN: Kibana like data explorer UI for PostgreSQL?

44 points by reacharavindh ↗ HN
I am exploring an idea of using Postgres + Timescale for storing very small volume of log data and having a "search based" web UI like Kibana's to let me and other users explore the log data.

Anyone here hacked around with such idea? have any open source projects to recommend?

Notes: ELK will work, but it is an overkill for my needs, plus, I want to build this for fun with PostgreSQL.

I'm not good at Javascript and UI development in general (systems background), and would prefer to hack around some existing project rather than building something from scratch.

27 comments

[ 3.3 ms ] story [ 66.2 ms ] thread
brett-za
Maybe something like Metabase (https://metabase.com/) would work for you?
reacharavindh
Thanks the suggestion. I played with Metabase a bit today. It is great for custom dashboards, searchable query links etc. but, I could not find a interface element that would let me "full text search" a column in Postgres table..

Do you have ideas?

vajrapani666
You can use custom queries in metabase and parameterize them, for any parameter you can expose the parameter with a text field available when you embed the "question" anywhere.

https://hevodata.com/blog/postgresql-full-text-search-setup/

reacharavindh
The link was very helpful to learn about making use of FTS in Postgres. Thanks!

I still could not figure out how to expose a filter as a user modifiable dynamic search box/text field with Metabase. I could not find such a facility in their documentation, nor in their forums. I'm giving up on Metabase for this need.

bosky101
+1 for metabase. had to turn off autoscanning though because it starting slowing down other queries when scanning a large table frequently.

we also use pgweb linked elsewhere here for engineers.

btw, we built a sql statement to spreadsheet + charts interface. DM if interested. intuitively - i think others may find it useful.

MightySCollins
We started with redash and moved to Metabase. It's really quite powerful and and teams great too.
seektable
Try our free BI tool https://www.seektable.com It has PostgreSql connector; with 'flat table' report type you can easily filter/sort records without need to write SQL, and pivot tables/charts are good for summary reports (under the hood GROUP BY queries are generated)

BTW, SeekTable can connect to ElasticSearch so it may be used with this data source too.

reacharavindh
Thanks for the suggestion. But, I'm looking for an open source tool that I can tinker with, contribute to and modify to my needs if needed now or in the future.

Seektable looks cool for enterprises that wants a hands-off reporting tool. Good luck!

LogicX
We use http://redash.io to look at our timescale data. Works our quite well for charting, and searching... they let you put variables into your sql query, then they become input fields, making search easy.
reacharavindh
I think this is exactly what I was looking for. I'm out now trying to figure out how to get redash running on my Mac to see if it is what I want.

:-( Sadly, their documentation shows how to quickly setup redash on AWS, GCP, and Docker. But, no instruction about some guy just trying to get it running on a Mac without Docker.

https://redash.io/help/open-source/setup

I will figure it out eventually, I think.

arikfr
We’re a small team and have to prioritize. At some point maintaining the Mac setup was too much, and Docker covers most use cases.

But setup for Postgres should be relatively simple as the extra dependencies required are minimal.

If you want you’re welcome to post a question on our forum (discuss.redash.io) and I’ll follow up with pointers on how to set it up on your Mac.

Update: just noticed that our developer setup guide without Docker is still online: https://redash.io/help/open-source/dev-guide/setup.

reacharavindh
Thanks for the link to dev-guide. I missed it because I was looking for instructions for standalone installation and skipped "developer guide" thinking it might be about dev docs about source code org and stuff.

Sorry if my response sounded like a snarky comment or a rant. That was not my intention. I can only thank you for an open source project and do not feel entitled for anything.

I ended up using Docker compose to get started with redash on my Mac. But, I cringe on the inside every time I am forced to use Docker (a daemon with higher privileges) to run stuff from the Internet, and it runs Postgres, Redis, a linux kernel, and so much more for essentially running a Flask application.

Now that I know redash fits my needs, I will spend the time needed to read the docker compose file and trace those steps and assemble the pieces necessary to run redash directly on my Linux server without Docker.

Thanks for redash!

arikfr
> Sorry if my response sounded like a snarky comment or a rant.

Actually it didn't. All good :)

> Now that I know redash fits my needs, I will spend the time needed to read the docker compose file and trace those steps and assemble the pieces necessary to run redash directly on my Linux server without Docker.

Glad to hear it fits! Note that if all you need is Postgres support, don't bother with installing requirements_all_ds.txt and you can skip some of the system packages we install.

Maybe this forum thread will be helpful: https://discuss.redash.io/t/bring-back-bootstrap-sh/2436/12.

jason_slack
I’m using timescale with 400gb of data. I don’t use web tools but let me throw out DBeaver if you need a solid database management tool: https://dbeaver.io
reacharavindh
Just curious. What do you do with your 400GB of TS data on Timescale? As in how do you query it? Are they only metrics or do you have textual data in there as well?
jason_slack
I bring in stock data. I query it. I make tables to store models for future reference or historical purposes. I have tables for order tracking, metrics, statistics, etc. it’s all automated.
sosedoff
If you need a lightweight web UI for Postgres - have a look at https://sosedoff.github.io/pgweb/
reacharavindh
Thanks for this recommendation. I'm using pgweb as a general purpose SQL web interface to Postgres. But, I cannot easily adapt pgweb to bring out a search interface that only searches a certain column of table in postgres.
sosedoff
Not sure what kind of issues you're having with search, but it's pretty straightforward to browse table data with a filter on a single column (be it numerical or text data). Im the author - shoot me an email (in profile) if you have any questions.
pytyper2
Is there a SQL server management studio like application for postgres?
h1d
TablePlus