HN Explorer

Ask HN: What API should I build a realtime data viz on top of this week?

3 points by zevaverbach ↗ HN
I’m about to have a rare childless week of vacation in a boring town with decent internet access, and I’d like your advice for what to work on!

I’m a senior-ish full stack dev (Python + React) and I’m interested in getting some experience with real-time data viz using Highcharts or AG Charts.

I don’t want to get bogged down in scraping but other than that I’m open to any suggestions for the project as far as:

    - data source
    - what protocol to use
    - whether to do JAM or SSR 
    - aesthetics/styling
Please don’t tell me to “take it easy”, I already knocked that out in the first part of vacay. :-)

11 comments

[ 4.0 ms ] story [ 37.2 ms ] thread
jzombie
Maybe build something to do stock (or crypto, or some other market) analysis of some sort?
tycoon177
I'm not sure if you're in the US, but there are a lot of data sources at https://data.gov/ that are pretty interesting! A similar service may be available for other countries.

If you want to really get out of your comfort zone, I would suggest a new language/paradigm. I've recently been loving learning Elixir with Phoenix liveview. It fills a lot of the full-stack needs and lets you focus on your backend data and your frontend display.

zevaverbach
Thanks so much for the suggestions! (Yes, I’m in the US)

Do you know offhand if any of those data sources update frequently enough for “real-time” visualizations?

tycoon177
I kind of doubt it, but you could fake a realtime api by streaming results from your backend of choice as if you were getting it from an api. Some of the data is large enough that you could download it async on your backend and stream results as you parse it! I'm actually downloading the liquor sales for my state since 2012 and so far it's over 2gb of data, so definitely stream-worthy.

I did also find this just now. https://github.com/bytewax/awesome-public-real-time-datasets

tycoon177
Oh, and if you do stick with react, I'd suggest this library! It's super powerful. I'm using it for some visualizations at work and it's pretty awesome what can be done. https://recharts.org/en-US/
zevaverbach
Thanks again, I’ll have a look at this.
jbernsteiniv
The USDA has an awesome API called Food Data Central. The data is public domain in terms of licensing and the rate limit is about 1000 calls per hour per IP address (which is extremely reasonable especially for early testing).

https://fdc.nal.usda.gov/api-guide.html

zevaverbach
Thanks so much for the suggestion, I will have a look although I’m hoping for something which updates at least a few dozen times per minute.