I think multi-agent (eg _different_ underlying LLMs) everything is really the future. Code produced via multi-agent workflows and reviews seems noticeably better. I've been experimenting with a multi-agent message board recently: https://github.com/pjlsergeant/dogpark
Experimented with multi-llm analysis for problem solving over summer, combined with multi-agent approaches it can tease out interesting angles to problems that I never considered. Expensive but use only for my high value problems.
I've built agents that call different LLMs and keep separated memories. Remember, agents are just long-running workflows with some nodes calling LLMs and that sometimes can be started as tool from other "agent".
There are times when I wonder if couldn't just draw then in a BPMN designer that allowed me to write custom code for nodes. Is BPMN still a thing?
>Isn't a single agent with a good harness better than any multi agent env?
Multiple agents can each have their own isolated LLM context that is focused on their respective narrow scoped tasks. That way, the accumulated "memories", "instructions", etc do not stomp on each other. E.g. the agent loop that is gathering "signals" from the market has a context memory full of instructions related to scraping websites and using REST APIs and it doesn't interfere with the other agent loop that does sentiment analysis that has context memory dedicated to text of news transcripts correlated with historical price signals.
You can have a single agent acting as a top-level "orchestrator" to manage other agents. However, you can't have a single agent that does "everything" for long complicated workflows with high quality. The drawback is the more agents you spawn to take advantage of isolating contexts to improve task completion quality, the more you multiply the tokens consumption and costs.
Another other driver for multi-agents is scaling out cpu usage, especially across different machine boundaries.
103K stars, so clearly it's popular. Has anyone here used it, and what are the outcomes like, and importantly, who is the target audience for this? I can see the logic behind crawling social media and news feeds to determine some 'evidence', but am not sure if that's the best approach or even if an LLM is the best way to get an assessment.
In case there is interest, I've got a fork with some custom improvements like a Web app, batch mode, Portfolio allocation planning, Richer reports, Portfolio monitoring, timeout handling, date validation, stop-loss direction checks, and extra utility helpers.
Having worked in hedge funds for the last decade, this seems to miss the mark. Firstly we often reward skillstacking ie a technical person later becoming a trader. The more one person knows the better. These people are rare though hence the reason there is still many seperate job functions, so a person can specialize. But an AI agent? They all have the same brain, so why nerf them by specialising.
Secondly, browsing reddit for sentiment and doing technical analysis is not even a feature in the trading world. At the most basic level, these are lagging indicators. Something on options IV and premiums would have been closer to the mark.
Hedge funds are akin to the maintenance crew for markets, we keep them efficient and liquid. The process is quite scientific, you come up with a theory and validate with real data. Or you go from data to theory.
As local contexts/skills get better at depicting what's to be expected and what an agent can work with and work to get better at, having more and more little specialized agents working as a swarm get you, with a field-skilled human as a supervisor, really good results even in highly niche and technical fields
Given what a know about the 2008 financial crisis, wouldn't an AI analysis in the years before that crisis of the real state funds helped to understand the risk of them better and avoid the big exposure.
The problem wasn't the analysis, given it was found out before it happened. The problem was politics, and as usual pushing the system to its limits and beyond.
Actually there is a really good reason, it has to do generally with making sure that all the aspects that you want evaluated are actually evaluated.
Now it may be possible with models like Astra that you no longer need to do this, but in earlier models it was beneficial.
So I might want a macro economic read which leads to a market thesis. Then I would hunt for exposure, then evaluate the candidates across different aspects. Breaking the process up at least made sure no steps were missed and the different aspects considered.
The obvious and dumb theory for trading on LLM recommendations is that lots of people are doing it so the price of stocks LLM recommend should go up. (ie: following the herd).
This part is lost on many. The value of data is in the theories it confirms or more importantly disconfirms, and defining the trading edge is not easy when accounting for costs. I am wary of black-boxes that produce an edge - not only because it is a black-box, but also because regimes shift unpredictably, and what works today may stop working tomorrow. That being said, AI can be useful to help automate many routine processes just like any other software.
Anyway, I have been running my own trading experiment and so far it has lost a bit of money. That being said I have not tried to optimise anything - just let it do whatever it wants. The losses are small and it might be able to recover later this year. Who knows.
I am thinking to output all the chat logs to HF as well for research.
You can run your own trading agents that communicate over a message buss in your own terms by downloading the CBK platform and running it locally with your own models. I have also shared my trading blueprint if you want to give it a go. https://chatbotkit.com/hub/blueprints/trader
The "blog" says that it's just holding cash or am I misunderstanding something? Judging by the "buying power" number, you allow it to use 4x leverage against the cash it has? That seems dangerous unless you are fine with losing 380k+.
I spent about an hour looking at the code and found some glaring issues that should be fixed before trusting it with real money.
- Yahoo News is introduced twice (sentiment and news analysis) which double weights it
- Sentiment analysis prompt primes the model to be bullish on Nvidia.
- In the self learning loop there is a complex parsing bug that results in hallucinated memories when agents return truncated responses
- You can completely control sentiment analysis of a subreddit by simply maintaining a majority of the 5 most recently posted messages, regardless of any quality metric
- The reflection prompt states the agent must cite alpha, which in a market wide downturn causes it to think correctly placed calls were losses
There is also problem with underlying models. There was a study, where they always repest the same investing/management strategy: trust strangers, be open minded/adopt to new unproven ideas, prefer cooperation... Basically they were trained on disney-boomer bull(shit) market of last 15 years.
They have zero guards against market manioulations, and will get wiped without bull market!
What is the purpose of this repo? Is it to simulate the market so you can reliably backtest trading strategies?
Whatever the stated purpose is, where can I read the test results to show it accurately fulfills that purpose.
Anyone can make a markets simulation that models interactions between market participants. Making a simulation that is accurate enough to be useful for anything is hard.
Although I don't think it even matters. They could easily cherry pick a period that is favorable for them. Wasting time and money on short term trading, rather than long term investment, using LLM or not, is never a good strategy for most people.
They backtested with public models, wouldn't the model weights already have the data? I double checked with chatgpt and looks like agents also had web search tool available so they could just lookup the past.
I am sure it won't be long until some rando from /r/wallstreetbets/ turns something like this loose without considering the ramifications and loses $250k.
I'd be curious as to how correlated development on these frameworks (ai or otherwise) is correlated with the market cycle. It seems during bull runs would be traders think they have some edge - whereas they're probably all just buying the trend.
This would make sense only if the LLMs would be constantly updated with a new data set and training phase every day. In that case, I'd could see this approach as having some sense. But, otherwise, these are just stochastic machines trained on static (outdated) data and I don't see how their predictions should be better than any other method around or even better than a human guessing.
I vibe coded a little stock market sim game then i wrote an agent who’s job is to win the game. A couple friends and family members play the game too. If it works I’ll just follow along and hold the same portfolio the bot does. The bot, named stonker, makes its first trades in about an hour actually. Assuming it works I mean hah
72 comments
[ 0.22 ms ] story [ 18.3 ms ] threadhttps://github.com/monkeydust/rightmind
There are times when I wonder if couldn't just draw then in a BPMN designer that allowed me to write custom code for nodes. Is BPMN still a thing?
Multiple agents can each have their own isolated LLM context that is focused on their respective narrow scoped tasks. That way, the accumulated "memories", "instructions", etc do not stomp on each other. E.g. the agent loop that is gathering "signals" from the market has a context memory full of instructions related to scraping websites and using REST APIs and it doesn't interfere with the other agent loop that does sentiment analysis that has context memory dedicated to text of news transcripts correlated with historical price signals.
You can have a single agent acting as a top-level "orchestrator" to manage other agents. However, you can't have a single agent that does "everything" for long complicated workflows with high quality. The drawback is the more agents you spawn to take advantage of isolating contexts to improve task completion quality, the more you multiply the tokens consumption and costs.
Another other driver for multi-agents is scaling out cpu usage, especially across different machine boundaries.
https://github.com/skanga/TradingAgents
I'll also be merging the upstream commits ASAP
Secondly, browsing reddit for sentiment and doing technical analysis is not even a feature in the trading world. At the most basic level, these are lagging indicators. Something on options IV and premiums would have been closer to the mark.
Hedge funds are akin to the maintenance crew for markets, we keep them efficient and liquid. The process is quite scientific, you come up with a theory and validate with real data. Or you go from data to theory.
There's always risk somewhere in a financial system.
Now it may be possible with models like Astra that you no longer need to do this, but in earlier models it was beneficial.
So I might want a macro economic read which leads to a market thesis. Then I would hunt for exposure, then evaluate the candidates across different aspects. Breaking the process up at least made sure no steps were missed and the different aspects considered.
This part is lost on many. The value of data is in the theories it confirms or more importantly disconfirms, and defining the trading edge is not easy when accounting for costs. I am wary of black-boxes that produce an edge - not only because it is a black-box, but also because regimes shift unpredictably, and what works today may stop working tomorrow. That being said, AI can be useful to help automate many routine processes just like any other software.
but...
options premiums imply volatility.
Lagging indicator would still be useful if it was accurate before the event. After the event, its just lagging history.
technical analysis is using historical data to make future predictions. No professional trader would trade anything without looking at price history.
Anyway, I have been running my own trading experiment and so far it has lost a bit of money. That being said I have not tried to optimise anything - just let it do whatever it wants. The losses are small and it might be able to recover later this year. Who knows.
The agent writes a blog about its progress here https://trades.chatbotkit.space/
I am thinking to output all the chat logs to HF as well for research.
You can run your own trading agents that communicate over a message buss in your own terms by downloading the CBK platform and running it locally with your own models. I have also shared my trading blueprint if you want to give it a go. https://chatbotkit.com/hub/blueprints/trader
I had done a small experiment with 100$, but would be hesitant on more than that. I suppose if it's money you're willing to lose.
I actually have a profitable trading agent, and hell no I will not open source it. It also looks nothing like this.
- Yahoo News is introduced twice (sentiment and news analysis) which double weights it
- Sentiment analysis prompt primes the model to be bullish on Nvidia.
- In the self learning loop there is a complex parsing bug that results in hallucinated memories when agents return truncated responses
- You can completely control sentiment analysis of a subreddit by simply maintaining a majority of the 5 most recently posted messages, regardless of any quality metric
- The reflection prompt states the agent must cite alpha, which in a market wide downturn causes it to think correctly placed calls were losses
They have zero guards against market manioulations, and will get wiped without bull market!
Whatever the stated purpose is, where can I read the test results to show it accurately fulfills that purpose.
Anyone can make a markets simulation that models interactions between market participants. Making a simulation that is accurate enough to be useful for anything is hard.
Although I don't think it even matters. They could easily cherry pick a period that is favorable for them. Wasting time and money on short term trading, rather than long term investment, using LLM or not, is never a good strategy for most people.
https://chatgpt.com/share/6aa00d5e-6920-83ee-8e3e-9cbf23f7bd...
stonks.jettdigital.app