HN Explorer

13 comments

[ 9.7 ms ] story [ 33.2 ms ] thread
jaygray0919
Does this work if an HTML page is on a Cloudflare network? My experiment returns the message: An unknown error occurred while rendering the page.
Woshiwuja
Cant get it to work, tried even with google but its always a cloudflare error...
solidgoldmagik
tried it on a bunch of random sites (news articles, github repos, company home pages, etc) and seems to work for me.
jaygray0919
Please show a successful query. For example, here is a test query: curl -d POST https://api.siteview.app {"url": "https://lite.cnn.com"}
solidgoldmagik
{ "url": "https://wework.com" }

{ "url": "https://www.cnn.com/2023/11/08/politics/takeaways-republican..." }

{ "url": "https://pytorch.org" }

{ "url": "https://www.natesilver.net/p/160-million-americans-are-eligi..." }

jaygray0919
Here is another part of the response: <span class="cf-footer-item sm:block sm:mb-1">Cloudflare Ray ID: <strong class="font-semibold">8236b959ba542980</strong></span>
jaygray0919
Note: i was running a VPN but turned it off for the experiment - thinking that something is causing a network redirect
Woshiwuja
I dont think it was the vpn. Tried again with the same command and it worked. Im guessing it was just a bit overloaded because of this post
jaygray0919
When testing the wework link i get this message: `<title>Worker threw exception | api.siteview.app | Cloudflare</title> (more ...)`. Does curl need additional information to escape this type of response? Do you use a curl alternative?
solidgoldmagik
are you passing the `content-type: application/json` header? I just now ran the wework and it worked as expected