HN Explorer

Show HN: SingleAPI – Convert the Internet into your own API (singleapi.co)

7 points by semanser ↗ HN
Hi! I created a universal data API that uses headless browsers and GPT to extract any data from the web in JSON format. I started this project because I needed some API to do data enrichment to get company data (headcount, investment rounds, etc.). Once I did the first version, I quickly realized that there can be many use cases for such a tool: data enrichment, web scraping, data validation, etc.

7 comments

[ 4.3 ms ] story [ 13.2 ms ] thread
semanser
I also published a simplified version on GitHub, so you can try to self-host it. I'm really excited to see all the possible use cases for such a tool besides web scraping or data enrichment.

https://github.com/semanser/JsonGenius

KomoD
What role does GPT play in this?
semanser
After retrieving all the text data from the webpage (using a headless browser), GPT is used to filter out all the noise and extract the actual information requested in the request schema. Let's say you request for {"product_name": "string"}. GPT will retrieve that product name from the webpage and return the correctly formatted JSON with the fields you requested.

It works pretty similarly to GraphQL when you define a schema that you want, and the backend returns the exact data that you requested. But in this case, the data is received from the webpage that you provided.

KomoD
Can the results really be trusted then? Isn't it possible for GPT to make something up that doesn't exist on the site
[deleted]
(comment deleted)
lpellis
This is pretty cool, it is able to parse data out of a random pricing table somewhere in the page. It does seem to just make up data it if is not found in the page (probably expected with LLM's), I wonder if you can reduce that with some prompting, or maybe verify the data is actually present? Your schema page docs is broken https://singleapi.co/docs/schema