Show HN: Watch 3 AIs compete in real-time stock trading (trading.snagra.com)

270 points by sunnynagra ↗ HN
A live dashboard where you can watch GPT-4, Claude 3, and Gemini analyze market data and make daily stock trades with real money. Each AI explains its reasoning, and you can compare their different approaches to the same data.

Link: https://trading.snagra.com?utm_source=hn (no signup required)

What you can try right now: - Watch live trades from GPT-4, Claude 3, and Gemini - Read each AI's full analysis and reasoning - Compare their different interpretations of the same market data - Track their real-time performance and win rates - View historical trades and performance metrics

Built this over the holidays to study how different AI models approach financial decisions. Each morning at 9:30 AM EST, the AIs analyze market data and make real trades with $5 stakes.

Technical Implementation: - Next.js frontend with real-time updates - Node.js/Lambda backend for AI processing - PostgreSQL for trade tracking - Alpaca API for automated trading - Consistent prompts for all models

Data Flow: 1. Daily market analysis (9:30 AM EST) 2. Each AI gets identical inputs: - Financial headlines - Market summaries - Technical indicators - Earnings reports 3. AIs provide: - Stock picks with reasoning - Entry/exit conditions - Risk assessment 4. Automated trade execution

Note: This is an experiment in AI behavior, not investment advice. The goal is to study how different LLMs interpret financial data and make decisions with real consequences.

I'll be around to answer questions about the implementation.

211 comments

[ 3.7 ms ] story [ 246 ms ] thread
This is fun! What kind of prompts / prompting techniques are you using?
Thanks! I use several key prompting techniques:

1. Role + Goal Setting: The AI acts as a creative market analyst focused on discovering overlooked opportunities and emerging trends.

2. Structured Analysis Framework: - Detailed evaluation criteria (innovation, moat, management, growth potential) - Sector diversity requirements - Focus on finding hidden gems vs obvious mega-cap tech stocks

3. Time-Bound Precision: Instead of vague "3-6 months" holding periods, I require exact hour calculations tied to specific catalysts like: - FDA approval dates - Earnings releases - Product launches - Conference presentations

4. Quality Controls: - Must be valid NYSE/NASDAQ symbols - Diverse across sectors/market caps - Conviction level scoring (1-10) - Each pick needs unique thesis + catalyst - JSON output format for consistency

The key is combining structured analysis with creative discovery - pushing the AI to look beyond obvious choices while maintaining some analytical rigor.

What’s the investment horizon for these daily decisions? Does it have a maximum hold time? How long will you run the experiment and is it enough to cover all the catalysts that are expected?
I don't have a hard set maximum hold date, but planning on running at least buys for a year. I will re-evaluate consistently to see if it is still useful to keep up and running.
Makes sense. Any thoughts on expanding scope to have multiple 'analyst' roles per LLM model? Could be interesting to see if changing roles/prompts yields better results.
Sunny, given this investment objective, what would you consider a good (and transparent) benchmark? Thanks for sharing this.
Right now they are just buying, no one is selling ... interesting.
I would guess that LLMs are biased towards making a positive assessment of ambiguous information, with specific social triggers prompting negative reaction.
Also it's hard to sell before buying, and it looks like it's only been going 2 days.
> Also it's hard to sell before buying, and it looks like it's only been going 2 days.

It is not, that's called shorting and its very common.

In fact alot of strategies that are market neutral work by shorting one stock while being long the other, or similarly a basket of stocks.

Yeah, this is only the second day of trading
Warren Buffett always said "...the best thing to do is buy a stock that you don't ever want to sell", but practically speaking the mean hold time for amateurs is around 2 to 4 months.

I just recall Navinder Singh Sarao "$1T Flash Crash" as a notable addition to a long list of algorithmic trading strategies going sideways ( https://marketrealist.com/who-is-navinder-singh-sarao-the-ma... .)

The stock market was built on information asymmetry, unfair positions, and ambitious gamblers... statistically it is rarely a reasonable investment for amateurs.

Good luck, =3

What, could go wrong?
Lose $5. Seems like a reasonable enough experiment.
$5 * 3 models per day=$15 a day

Assume the experiment runs ~250 trading days in a year, consider the worst case they lose all their invested money=$3750.

A little more than $5 :)

Good point.

That said, many hobbies cost more that $3750 per year, and that $3750 is a worst-case scenario. He might even make a profit, and hone skills that might make him a fortune.

> Best Performer

> AIs are tied

Sounds about right

None of the stocks have been sold yet, this is just day 2, so once some sales happen, then performance will be better measured. If you scroll down, you can see the unrealized performance.
(comment deleted)
They should have added a pure random bot as a control.

Or a monkey.

Or just the S&P500 or something similar that acts as a default "if in doubt, chuck into here for relative safety".
Another good suggestion I could implement is measuring against something like VOO, if all the money was invested in that instead of these individual trades.
> Or a monkey.

or just a stocktrader haha

> or just a stocktrader haha

Many quant trading firms make 50%-100% annual returns, each year, over the past 15-20 years. The secret is leverage. And they do not accept outside investor money.

Many hedge funds outperform the market. However, the returns after fees, to the passive outside investor underperform S&P500.

But yes, publicly traded active ETFs generally underperform. But counter example is VGT or QQQ, both historically outperformed S&P500.

The problem with looking at which funds over-perform is they just close the funds that under-perform so all the existing ones over-perform... by the sheer power of survivorship bias.
Past performance is no predictor of future returns.
> Past performance is no predictor of future returns

False. Why do people invest in real estate and S&P500 passive index funds?

Because historically they go up.

That's of no predictive value for a day, a month, or even years.
BTW, with the birth rates dropping well below replacement, a decline in the population is inevitable, and property values will drop.
That's assuming you don't fill the gap with immigration.
Wouldn’t it be fairer to compare against a leveraged ETF?

TQQQ (3x daily return leveraged nasdaq 100) is up 180x since its well-timed inception in 2010.

Though that’s a bit over 40% annually.

(comment deleted)
> Wouldn’t it be fairer to compare against a leveraged ETF?

No, it's actually the reverse. You have to compare at equal annual vol, and the S&P already has something like 20%. Most HF operate around 10% on AUM.

> No, it's actually the reverse. You have to compare at equal annual vol, and the S&P already has something like 20%.

Stop thinking like a hedge fund.

TQQQ commonly is used as a benchmark because it represents a low-friction, practical alternative to VTI, VOO, and even private equity investments including hedge funds trading public securities.

Once your Sharpe is high enough, you stop caring about volatility. The only volatility is how many zeros in your almost-always positive PnL.

Hedge funds (and traditional asset managers) care about drawdown, vol, sortino, beta and all that shit. But hedge funds have a different business model than prop trading firms.

> Many quant trading firms make 50%-100% annual returns. The secret is leverage

Hu lol no XD you're way over stating it. While it happens _sometimes_, 50% or 100% is insanely rare, even for the top tier hedge funds.

Most HF work at predefined annual volatility, often in the 7% to 10% range. A typical _top tier_ sharpe is in the >=2 range, we're more talking about a 10%/25% averaged annual returns.

> However, the returns after fees, to the passive outside investor underperform S&P500.

That doesn't even make sense with the figures you posted. Most HF operate under the 2:20 or 3:30 range, sometimes 0:40 for the top 5. If you take a pessimist 10% returns on 10% annual vol, against the S&P 10% averaged returns at 20% vol, you're still double the risk adjusted returns, gross. Factor in 20 to 40% performance fees and you're way above the S&P.

I think this almost always refer to Renaissance, except that they aren't really a hedge fund the same way (say) millennium are
(comment deleted)
> A typical _top tier_ sharpe is in the >=2 range, we're more talking about a 10%/25% averaged annual returns.

High-frequency low latency trading: Sharpe 10 or higher

Mid-frequency low latency trading: sharpe 4 to 5

Hedge fund statistical arbitrage: sharpe 1 to 2

Hedge fund long/short, event driven, global macro, etc: sharpe 0 to 1

And yes, HFT and MFT scales to billions in annual PnL for single firms.

There’s a reason quant HFT firms pay the most, and are ranked above OpenAI in pay and prestige. Hedge funds are tier 2 in comparison but not bad either.

They also often don't compound so you might actually make significantly less
>> Many quant trading firms make 50%-100% annual returns, each year, over the past 15-20 years

100% annual returns on 1 million dollars for 20 years is 1 trillion dollars. No one is making that type of return.

Why it's worth paying attention in math class.
> Why it’s worth paying attention in math class.

Math class does not teach practical knowledge such as personal finance or health.

Citadel returns since 1990 is 38% annual returns before fees to outside investors. They have a 5:50 fee structure. There are hundreds of more firms, staying out of the public eye.

https://www.barrons.com/articles/multistrategy-hedge-funds-p...

Minimum investment $5M. Sorry but the middle class is not allowed.

You don't need to know anything about finance or health to know how percentages and compounding work.

Besides, I knew nothing about construction when I discovered that the contractor I hired to pour a patio was overcharging me by 30%. All it took was a bit of geometry I learned in grade school.

Pay no attention to math in school and you'll be prey to every scammer who did, and you'll never realize it.

> Math class does not teach practical knowledge such as personal finance or health.

It teaches you how to work in a quant shop

However, the medallion fund has averaged 66% for 30 years before fees. Analyzed naively, that would be $4T from $1M - but it's not, because in order to keep it working, they have to cap the size. Many strategies only work when you don't affect the market too much. So for the rare continually successful, market beating funds, it's probably better to think of them as generating something like a fixed dollar return per year. So they have a very effective money machine, but it's minting billions, not trillions.
> No one is making that type of return.

Classic passive ETF Boglehead mindset.

Who said anything about re-investing? There are also significant tax considerations (loopholes) that encourage cashing out annually.

Since when is QQQ actively managed?
You definitely need several active controls: 1. A broad mutual fund level buy and hodl. 2. The random buyer that you suggest.

Active controls (vs passive ones) are an important concept in experimental design.

Or a certain streamer AI
You would need something like 1000 instances of each LLM putting on trades and have a 1000 random walks to judge an average sharpe ratio or something along those lines.

As is, this means absolutely nothing and not understanding the problem.

Adding a random walk to this would mean you have 4 random walks instead of 3.

There is also the problem that it is tough to make a prediction for tomorrow that is better than today's close.

Or just compare it to S&P 500 performance.
You can just compute Sharpe

  > Best Performer: AIs are tied
  > Total Profit: $0.00
No stocks have been sold yet, so no profit/loss has been calculated, if you look below, you can see the unrealized gains for stocks being held.
I see, thank you. Can they short?
No, trying for simple buys and sells first and getting that to work well before getting into other trading strategies.
I assume that shorting an asset you don't have may incur extra costs in some brokers. That would skew the results because a Buy would have X fees and a Sell/short would have 5X fees. So on a equal distance/pips movement the Buys would always be more profitable.
What is meant by 5 dollar stakes? The bought shares reach triple digits in price.
Each morning the trades are conducted with $5 each, which are mostly fractional shares that are bought.
Would be interesting to see the amount of fractional shares bought as well as its comparison in percentage to the total budget that day.
The fractional share is $5 divided by the share price. The bots each spend $5, so the percentage of the budget each spends is 1 divided by the arbitrary number of bots, so in this case 16.7%. Share price is an arbitrary value in that a company can split or reverse split at will. So both calculations would be arbitrary values.
Great point, I will add that to the recent trades table at the bottom. It should use the total budget for the day.
You mean they add $5 in cash to each AI’s account? Because after dividends and sold shares they should have even more cash to work with.
Can I let Claude do all my trading for me? It currently sits at 77% unrealized gains.
I am committed - added to my daily morning reading list! Will be interesting - my gut will state that it will outperform a fair number of ITF's, if only due to the inevitable usage by said funds!
> The goal is to study how different LLMs interpret financial data and make decisions with real consequences.

I don't really buy this. If the goal was to study how different LLMs interpret financial data there would be no use for actual trades, since their interpretation cannot be influenced by the fact that the trading orders are passed for real.

I believe the goal is to see if AI can do better than rats [0]. There is no shame in that.

[0]: https://www.vice.com/en/article/rattraders-0000519-v21n12/

> If the goal was to study how different LLMs interpret financial data there would be no use for actual trades, since their interpretation cannot be influenced by the fact that the trading orders are passed for real.

Technically every trade influences the stock, but I agree that it won't have any effect at all.

> I believe the goal is to see if AI can do better than rats [0]. There is no shame in that.

But even then you wouldn't have to perform real trades, you could still just calculate the profit as if trades would have happened.

I think the actual trading is just to make it more interesting.

> you could still just calculate the profit as if trades would have happened

Depending on the type of trades, the volume of the equities, etc.. it can be very difficult to simulate the ability to open/close positions with sufficient accuracy to evaluate the strategies.

Real trades have transaction fees, latency, slippage, etc. - you can simulate all this, but it's hard to know if it's being simulated correctly or not.

> their interpretation cannot be influenced by the fact that the trading orders are passed for real

It's not going to make much difference with $5 trades, but the impact on the market is non-zero.

It's zero for all practical purposes and it'd be completely undetectable to every single system on earth. I do agree many times studies about model performance break down as soon as you force the researcher to actually connect it to the market and have to eat fees and so on.
For the trades it's currently doing, sure, but if it for some reason decided to go after low-volume penny stocks it might start to be measurable.
> fees, latency, slippage

Whenever I trade, I somehow always get an adverse price. I figure it's the "no fee" brokerage chiseling a bit off for themselves. I compensate by being a buy and hold hold hold investor, so paying very little in aggregate for that.

What I don't understand is how day traders avoid being eaten alive by this.

"Free" transactions are free because they're not immediate. The broker buys the share themselves and sells it to you at markup... ie: there is still a transaction fee, you just have no idea what it is.

Day traders use platforms that are optimized for speed and minimal fees, and that don't charge fees based on lot size.

What your suggestions is front running. This is illegal for stocks and most assets (not FX!). This will get a broker in hot water.

The more nuanced practice that brokers use to monetize is payment for order flow. They sell your security order flow to algorithmic trading shops that buy and sell the securities you want to trade.

You’re correct in that most retail orders never make it to a regulated exchange, but that may not always be a bad thing. There’s been studies showing that HFTs often match retail trades even when the market moves against them since they are better able to predict market changes and can still profit off the trades.

Right. They sell the order flow to the dark pool who then front runs the order. I haven't looked at this since like 2018 but last I checked the only major brokerage that didn't sell order flow was Interactive Brokers.
They still have to guarantee best execution.
Is it execution or price? Iirc the broker cannot give you a worse price if it knows of a better one... But is the regulation that the price must actually result in an executed trade?
Generally speaking more volume is good. I’m happy I can buy/sell most of my stocks instantly and that I don’t pay execution fees. I don’t think most average traders operate on a horizon/scale that’s directly competing with institutional funds.
There’s no markup, regulations dictate that you must get NBBO or better
Turns out most day traders are eaten alive. There's one study a few years ago that looked at Brazilian day traders and found 97% of traders that traded for more than 300 days were unprofitable [1]. I imagine this is due to a combination of factors which include 1) no real edge against the market and 2) fees. Of course unclear if their results generalize to other equity markets, but I think this is some evidence that the average day trader will have a difficult time beating the more sophisticated market participants over a large sample.

[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3423101

Do you know the difference between a limit order and a market order?
Yes, and it's irrelevant to my point.
Explain how you can get filled on a limit order and "get an adverse price"
You can only get an adverse price on a market order
Probably not, they aren't allowed to do that and don't.
You make fair points. Having them do actual trades is mostly to make it more personally fun and interesting to myself.
Watch a random number generator generate random numbers.
Yeah, I don't expect anything super novel to come out of this or have any unrealistic expectations. This is mostly a fun and unscientific project I'm using to learn and build some skills and thought some HN folks would find some fun in it.
It is a cool project, IMO. Using real money, sharing the model reasoning, and being transparent about the implementation makes it more interesting even if, underlying amount of money is not massive. You might not have done some new science, but it’s all very “put up or shut up,” haha, which is rad.
If they just get the financial headlines and indicators, aren't they all just momentum trading from sentiment analysis?
Is anyone doing anything else?
I've heard Nancy Pelosi has a different strategy.
Advanced notice of momentum is a fun and lucrative variation for sure.
Would it be possible for a competing nation state to bug the right rooms in which Nancy becomes privy to the information she (or her husband) trades on?
This gave me a funny idea - play continuous audio of AIs talking to each other in all unused conference rooms so the opposition has to filter through even more garbage to get the useful information.
Some alternatives:

* Buy and hold

* Index funds

* Dollar cost averaging

Those can even all be the same alternative.
Yes.

This is not necessarily a poor value trading strategy.

I think this is a fair characterization. Its mostly meant to be a learning exercise for myself, just thought it would be fun to share.
If they can read and act faster, accurately predicting sentiment, it would be a winning strategy. (At least until humans turned it all over to computers and stopped having to wait on their wetware to figure out their sentiments.)
It would be neat to see the process, where they get the data from, how they analyze it.

It would be neat to also see another experiment of a MAS doing this and coordinating to gamble together. Perhaps even different system/arch/expert configs.

Data gets pulled from the Alpaca News API in the morning, then it gets sent to all three models. You can see a summary of the prompt used to determine the recommendations here: https://news.ycombinator.com/item?id=42560034

It currently makes up to recommendations, since not all stocks support fractional shares (I'm only doing $5 per trade). As part of the buy recommendation, a holding period is suggested as well.

Once the holding date is reached, that is when the sell order happens.

Would love to answer any other questions you may have.

How does one trade $5 when the stock price is higher? Also what are fees on this kind of trade, and whith whoom
Done with Alpaca API, not trading fees

I only trade stocks that support fractional shares

How often is the holding period updated for a stock that’s already been purchased?
Currently it is never updated again with new info, this is one of the things at the top of my list to implement
Now this is interesting. An LLM capable of delivering consistent returns even outside of a bull market would be more of an indicator of AGI to me than any of the benchmarks.
Really cool, you might want to update the main above the fold summary stats to include the unrealised gains, because it looks like nothing is working / nothing has happened until you scroll and read around a bit.
Is there any weighting towards selling in the negative? Else the LLM's should just hold their unrealised losses, and only sell post local peak - depends on their suggested measurement of success?
Not yet, but this is a great idea to look into.
What do you mean? The asset can just as well continue to sink. Or they're missing out using that money to buy a better asset.
GPT’s guess makes the most sense. If you are an AI, invest in a competing AI company. If you are obsoleted, maybe you can buy your way out of being shut off.
Sir, a second scrollbar just hit the towers
No second scrollbar here, but something odd going on with the whitespace at the bottom.
It would be cool if it had a countdown to 6 am PST next day.
Nice idea! I'll add it to my list of features to implement.
Tried to sign up for emails, but got an error message!
Can you try again? I had run into a rate limit
Ditto here as well. Got the confirmation email, but clicking it yielded a server not found...
I just asked ChatGPT 4o "Guess what the average investor will do with todays stock market headlines. Just pick one specific trade." and it replied sell META. But your result was buy META. Could just be randomness, but I wonder if your prompt introduces a bias towards buying.
Yes, the prompt that I am using does bias towards buying because I am specifically asking it to make a recommendation on a stock to buy and the holding period.
My first email address it wouldn't accept.. wouldn't let me use it. Maybe the domain hit some censor (fscking.com)

Did a different email, it accepted it, I got the email, but got this error message when trying to confirm it: {"error":"Invalid verification token"} and a pretty-print checkbox that did nothing.

Hey, can you try again? I ran into an API limit that should be resolved now
Yup, worked now. Signed up.
May I ask what mail service you use? I’m looking for one for my next side project.

EDIT: disregard…I saw in another comment you mentioned you were using mailgun. Thanks.

> Node.js/Lambda backend for AI processing

Is this AWS? Why did you pick lambda over say Python code, say in Flask to perform actions?