Ask HN: Advice for processing large quantities of data locally?

2 points by notchatgpt ↗ HN
I have a Postgres database with a few terabytes of data. I need to process a large quantity of data (terabytes) which will involve queries/upsertions into the database.

How do you best do this and finish in a reasonable amount of time (1-2 weeks)?

For reference, I have 64GB of RAM and working with several 5TB drives (a mix of NVMe, SSD and spinning discs).

0 comments

[ 2.1 ms ] story [ 7.3 ms ] thread

No comments yet.