Ask HN: What are the best tools for DB ingestion?

I have a database (MySQL) that is unique id, name, address, and data appended.

I get updates to the DB. I am looking for the best tool to update / ingest new / delete old data quarterly.

It is about 6 M records each update.

looking for front end tools, if they exist. Thank you ahead of time.

1 comment

[ 3.0 ms ] story [ 9.5 ms ] thread

I use SQL Workbench for bulk loading data from a legacy database into postgres, inserting or updating about 600,000 rows nightly. (It'll work with any database via JDBC.) A good starting point is to run some imports interactively using the "data pumper" section of the GUI, and then there's a button which shows you the equivalent script. Support is via a google group, and is excellent.

http://www.sql-workbench.net/