Unfortunately it doesn't work anymore, but the titles it generated for the papers were all plausible but also ridiculous CS paper titles, e.g. Rooter: A Methodology for the Typical Unification of Access Points and Redundancy
Super easy, just took 10k titles/comments/points from the Angolia API, formatted them as JSON Lines like the following with jq, and fed them to the very well built and documented openai CLI.
{"prompt": "A plausible Hacker News title:", "completion": " The Feynman Lectures on Physics (1964) (280 points, 62 comments) END"}
The space at the beginning of the completion is for tokenizing, and the END token is for use as a stop token in the generations.
I absolutely want to read some of these. There needs to be a way to for people to post and rank the best matching articles that actually exist. I dream of the day that generative AI is repurposed to act as a more effective search engine.
It averages around 0.0005$ per request according to the footer. Could this end up costing the author quite a bit due to HN traffic? Also, whats stopping bad actors from writing a script to continuously fetch the page?
I wonder if some sort of caching might help lower costs.
Nothing. And in fact that’s exactly what happened to me. Some fella from HN spawned like 45 simultaneous wget’s in a loop to cause maximum financial damage. All of a sudden we see Firebase’s cost graph go vertical.
It happened after I mentioned “just be kind, please! Theoretically this could cost a lot of money.”
So there’s at least one person who will do exactly this just for fun.
Firebase customer support was super cool about it, but it still knocked us off the paid tier.
There's intentionally no caching, every batch is warm from the AI oven.
HN usually drives around 10k visits, so organically it's going to be well within my Saturday night budget. If someone decides to hammer it, well, the OpenAI account has a hard limit of $20/month. It will live until it's killed I guess.
Is there a dataset of HN titles? This made me want to fiddle with this, but step one is to get the data, and I don't want to crawl HN if the data has already been collected.
There's an API[0] but it's frustratingly limited in capabilities (albeit not rate-limited.) You'll have to iterate all post IDs, download each post as JSON and get the titles that way.
There's also a Google dataset but I don't know the URL for it or if it's up to date.
What's missing precisely? Seems to be good enough for every use I could think of.
One time I even downloaded every single item from it, with a threaded fetcher of I think 16 threads, iterating from 1 up to latest ID and it was done in some like 2 hours I think.
No ability to directly download threads with a single request, for one, or query it like a database to sort or filter results, exclude unwanted fields, etc.
Those things should be trivial to achieve with most general purpose languages, as the API is so simple. No need for pagination or other things, just request things by ID recursively and you get the full thing, then after than filter/select whatever you want.
Pseudo-code to show how simple it would be:
function get_thread(id) {
let item = http.get(`{api}/?id={id}`).body
if item.childs {
item.fetched_children = item.childs.map((id) => {
return get_thread(id)
})
}
return item
}
(untested, but you don't really need more than that, besides checking if the item was deleted)
Yes, and i've done it. But you still wind up having to make a separate request for each item, which makes building threads incredibly slow. It's also a waste of time if you're filtering out anything, because you still have to make the request and download the item to filter it out.
Which is why it would be preferable for the API itself to support these features.
Eventually we will get AI based title analysers that simulate the conditions to predict and optimise where your title will land on the front-page listing order.
81 comments
[ 1.7 ms ] story [ 122 ms ] threadUnfortunately it doesn't work anymore, but the titles it generated for the papers were all plausible but also ridiculous CS paper titles, e.g. Rooter: A Methodology for the Typical Unification of Access Points and Redundancy
Hard to argue with that logic!
I’m glad to hear it!
What an invention!
Mark Zuckerberg: “Haters are supporters; hate is a solution”
{"prompt": "A plausible Hacker News title:", "completion": " The Feynman Lectures on Physics (1964) (280 points, 62 comments) END"}
The space at the beginning of the completion is for tokenizing, and the END token is for use as a stop token in the generations.
Also unrelated but good one "Tether issue: under $1B and all of the tether tokens suspended". This would definitely get my click :)
Show HN: Daily XLSX to-do list with attached spreadsheet as material*
Banning JavaScript from web pages is bad for the user
Has Google become too social?
SQLite development from scratch from scratch
Armor-piercing lasers are not shooting lasers but missiles
We made a public blockchain off-chain
Apple sued for pricing user data against provider who did not provide refunds
100% Embarrassing Haskell Builds
Heroku Compose is not fit for purpose
Pain Enhancer
The distance between reality and fantasy is grows ever smaller.
I wonder if some sort of caching might help lower costs.
It happened after I mentioned “just be kind, please! Theoretically this could cost a lot of money.”
So there’s at least one person who will do exactly this just for fun.
Firebase customer support was super cool about it, but it still knocked us off the paid tier.
think of all the money you have saved by not having to hire a hundred engineers to maintain your website's infrastructure!
There's intentionally no caching, every batch is warm from the AI oven.
HN usually drives around 10k visits, so organically it's going to be well within my Saturday night budget. If someone decides to hammer it, well, the OpenAI account has a hard limit of $20/month. It will live until it's killed I guess.
You have to account for this not being a regular visit, but rather (I guess) 5-50 "visits" per visit, as people mash F5 to get more titles.
> OpenAI account has a hard limit of $20/month
That makes me feel better about the couple refreshes I did myself :).
My favorite so far: "Why isn't Linux perfect?"
There's also a Google dataset but I don't know the URL for it or if it's up to date.
[0]https://github.com/HackerNews/API
What's missing precisely? Seems to be good enough for every use I could think of.
One time I even downloaded every single item from it, with a threaded fetcher of I think 16 threads, iterating from 1 up to latest ID and it was done in some like 2 hours I think.
Pseudo-code to show how simple it would be:
(untested, but you don't really need more than that, besides checking if the item was deleted)Which is why it would be preferable for the API itself to support these features.
I used the Algolia search API, it has extremely generous rate limits and page limits.
[0]: https://github.com/HackerNews/API [1]: https://hn.algolia.com/api [2]: https://console.cloud.google.com/bigquery?p=bigquery-public-...
Show HN: A tool to quickly improve your skills
Amazing stuff.
"James Webb Telescope Recall" was pretty amusing as well.
This one seems too good to be AI
> Tesla Co-CEO says the autonomous vehicle ‘is for chancers’
> Tesla Co-CEO is trying to raise $800M for T4, but details are sketchy