HN Explorer

Show HN: Ask LLMs to predict anything based on news (aipredict.fun)

25 points by artembugara ↗ HN
Hi HN! We partnered with the Atlas team to build a tool called AI Predict [0] that allows anyone to ask any question about the future and get a thoroughly researched, AI-generated prediction on how likely it is to be true.

How it works: Atlas replicated a Berkeley paper [1] that showed LLMs could make predictions as accurate as the crowd. We’re using a mix of models from OpenAI and Anthropic, with information retrieval powered by NewsCatcher [2].

The system is live and fully functional, though it might struggle with hyper-local questions outside of the public domain (e.g., “Will I have chicken for dinner tonight?“). But we’re intentionally NOT stopping it from answering any question, so fire away!

What makes it different:

- Free to use: Anyone can ask any question about the future that they care about, whether it’s a broad societal trend or a specific personal decision.

- Transparent: The tool is based on a proven reasoning model and provides links to all sources. You can see its track record on public predictions [3].

- Open-source: We’ve released a simplified open-source version [4] to give people a better sense of how it works.

Why we built this: We’re forecasting nerds and loved the idea behind the Berkeley paper. Friends and family kept asking us to make predictions for them, so we figured we’d build a frontend around the system so they could use it themselves. Don’t use it to decide where to buy a house or anything serious. But we think it’s a good starting place when doing research. AI Predict tackles prediction questions using transparent, objective reasoning (mostly Bayesian analysis) and links to all its news sources.

We’d love for the HN community to try it out and share feedback (predictions@joinatlas.ai). Let us know what you think!

[0] https://aipredict.fun/

[1] https://arxiv.org/pdf/2402.18563

[2] https://www.newscatcherapi.com/

[3] https://docs.google.com/spreadsheets/d/1oPvWlst7sxq2URifFaZ0...

[4] https://github.com/getdatachimp/llm-superforecaster

9 comments

[ 3.0 ms ] story [ 13.2 ms ] thread
user052919
Atlas founder here. Happy to answer any questions people have about the tool!
user052919
We just increased our server resources because a lot of questions are getting asked (thank you!). Hoping to keep the wait times manageable
jmartens
Can you provide an example of the type of question that comes back with a reasonable prediction? So far, I've tried 3 times, and the answer each time was something along the lines of [prediction will be true] if [they do the thing that makes the prediction true] which is not useful.
user052919
Sure! Sorry to hear you've had some bad ones. If you have links handy it'd be great to see which ones went wrong so we can try to improve.

But here are some recent predictions that we thought were pretty reasonable: * https://aipredict.fun/prediction/a0354d22-d44a-4115-a774-1cc... * https://aipredict.fun/prediction/1518666d-d5f8-40b1-bee9-561... * https://aipredict.fun/prediction/c13193fa-4155-4b47-b50a-432...

jmartens
Interesting. I asked similar questions (Who will win the Presidential Election and Will XXX college football team win the national championship, etc) and none of the responses included percentages or yes/no indicators like the examples you link to.
sturza
Hard to launch today with gpt-o1 getting attention.
niles
I was hoping for a tool that would reference "all" news, mainly historical news, and give some reasoning based on past patterns. My query on Kamala does not seem to factor in the news from just before the last Trump win over Hillary and how that time period contributed to a surprise win. It also doesn't seem to incorporate Regan, or past high inflation periods and how those news stories influenced the elections of the time.

It seems like this is a tool that "if you could read all the news of today, what would your gut tell you", and that is helpful but not thorough.

For example, can you apply the cureent reasoning to news articles from previous time periods and use the prediction on a past result. Would your prediction be accurate? If no, maybe re-reason.

https://aipredict.fun/prediction/1074cf90-c818-44fd-be0b-c00...

user052919
This is great feedback! Thanks for giving it a try.

You're right that we're limited in how much news we can pull. Generally, we can only look about 90 days into the past for news articles.

I'd definitely like to expand the corpus of information that we can pull from. Getting access to reliable historical data is high on that list, as it will dramatically improve base rate estimation.

niles
Basically what I'm describing is "backtesting" from stock trading space, where a trader comes up with a hypothesis on what will come in the future. Then they retry that scenario in slices where the same conditions happened in the past and see how it would have played out. Importantly, the "algo" sees in real time based on when you start it, so it can not cheat. It makes it easy to see whether or not your intuition is based on actual data.

I kinda feels like you are using the LLM to assign "weights" or important properties of an algo and then directly translating the basic arithmetic accounting of those factors into a prediction. What I expect is that the LLM would also be used to read all past news to find similar patterns and then create time slices where its weights could be tested against a control. It can then backtest its own weights to better tune what factors really led to an outcome and expose this refinement as part of the prediction.

News Data Sources: https://www.gdeltproject.org/ https://credibilitycoalition.org https://data.worldbank.org/