Show HN: Pytablewriter – A python library to write a table with various formats (github.com) 6 points by thombashi 10y ago ↗ HN
[–] jastr 10y ago ↗ This is pretty cool!What functionality does this provide that Pandas doesn't?Can it handle tables too big for memory? For example, could I use it to read 10 gigs from PostgreSQL to a CSV. [–] thombashi 10y ago ↗ I would recommend that divide one SELECT query for 10GB data into for-loop that smaller SELECT query (like 1MB-data/query) execution and writing table by using LIMIT and OFFSET clauses. In that way, you don't need a large amount of memory.
[–] thombashi 10y ago ↗ I would recommend that divide one SELECT query for 10GB data into for-loop that smaller SELECT query (like 1MB-data/query) execution and writing table by using LIMIT and OFFSET clauses. In that way, you don't need a large amount of memory.
2 comments
[ 2.6 ms ] story [ 13.1 ms ] threadWhat functionality does this provide that Pandas doesn't?
Can it handle tables too big for memory? For example, could I use it to read 10 gigs from PostgreSQL to a CSV.