Ask HN: Has anyone used machine learning to trade?

Yes. The intraday movement of stocks is small enough that you'd need to be using a large fraction of your capital on every single trade in order to overcome trading costs; and based on the Kelly criterion you'd need to have unrealistically high odds of betting correctly in order to avoid going broke.

The institutions have to hold on for longer periods and can't really "scalp" trades per se given that the bid/ask volume would collapse if they didn't distribute and "average in" slowly overtime to accumulate a position vs. buying or dumping their entire position all at once.

if you really want intraday, think about news.google.com and http://rs2007.limsi.fr/index.php/TLP:Page_12 to automatically tweak the few k base parameters (SVM, associative memory etc).

http://www.bbc.co.uk/science/horizon/1999/midas.shtml

Options, Futures and Other Derivatives, J. C. Hull, Prentice Hall Int’l

Financial Calculus, M. Baxter and A. Rennie, Cambridge University Press

but the ML part is almost irrelevant: it is not the lever in the leverage (think CDO trade).

While they do make it up on the rest, you have to ask yourself if you can afford the costs (brokerage commissions and fees) of executing your strategy in that context (they all work for banks and hedge funds).

If you really want to get into this, you might be better off applying to a quant-based hedge fund.

A firm like that will have the capital and brokerage connections to make the game profitable.

Also, take a look at Ed Thorp's web site (http://edwardothorp.com/index.html).

He was one of the original quants and has many interesting articles on the industry.

I spent a bunch of time with Neural Nets and SVM and that data. Sadly I didn't find a profitable strategy. I would love to hear about anyone else who is looking in that area.

1) Limited market data: As an individual, you have limited access to market data. At best, you probably have real time top of book data but without paying for it, you almost certainly aren't getting a real time depth feed. Going off of only last trade data gives you even less to work with. In most cases free quotes are delayed making them worthless to day traders (although not longer term investors).

2) Limitations of market data: Even if you have a lot of historical data with a full order book, that is still of limited utility for simulations. The problem is that everything you do affects the market so any trade you simulate needs to be doing small enough quantity to not make your simulation unreasonable. How to deal with simulating market-affecting trades is a complex problem and the only way to truly know is to run your trades in the market.

3) For day trading, there is a lot of profit to be captured simply from the microstructure of the market (its moves back and forth, etc.). To capture much of this it helps to be fast but you won't be able to colo boxes at an exchange with only 10K.

Machine learning seems more suited to longer term investments and trading on a slower platform than making a quick profit arbing or scalping. You may want to consider this rather than day trading since not only does day trading require much larger amounts of capital, but its also extremely risky.

Also, if you plan to trade equities, enjoy all of the ridiculous SEC regulation (order marking, etc.).

If you want more information on this, there was an IAMA on Reddit about 2-3 months ago, on a guy who claimed to do exactly this. It was incredibly informative, and went into a lot of the details that you ask for here.

http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat...

The reason for this is that the large companies won't talk to you unless you're trading at those volumes since they charge pretty high transaction fees. The smaller companies that will talk to you all have accounts with the larger companies and make money by increasing the spread between the bid and ask price they receive from the large companies. The expected return on a trade is usually somewhere in the region of 1-10 pips (where a pip is one ten thousandth of a dollar), with 10 pips being a really good return and 3-4 being more realistic. The smaller companies usually increase the spread by up to 3 pips in each direction on the price they get from the larger company which swallows up your return.

While millions of dollars per trade might sound like a lot the leverage in this market is also very high at 50-100 times meaning that with your 10k you could trade 500k-1M though on a sum that low you might get less leverage. However if you lose money on a trade you could eat substantially into your capital. So if your $1M trade goes against you by 10 pips you'd lose 1k.

I'm not ruling out making money on that market, but I'd say it's very unlikely unless you have a lot more money to start with. I am not an expert in this area though. This is just what I learned from the small amount of time I spent working with the company.

And $10K will almost certainly not be enough. You'll spend all your money paying commissions or bid/ask.

And realize that this is a fanciful pursuit which is likely to lose you money, because the guys you're going up against have $ millions, ns-scale lag times, free or near-free transaction costs, and PhDs in theoretical physics.

Anyone know of any particular markets where a small timer might have a better go at it due to insufficient volume to attract the bigger players, or for any other factors?

The neural network was a fairly normal feed-forward network with GA-inspired, but heavily modified learning process. It combined layers of neurons with memory and memory-less neurons. It is essential to keep the network size small, because a big network will just memorize the data you use to train it.

Also, I used several other inputs, such as the frequency of trades per minute, overnight rates for each currency, and stock market indexes for the relevant countries.

Finding historic data was the biggest problem. I used Bloomberg data from my university, but that was a pain in so many ways. There is NO free data for most of the inputs I needed. Good data from Thomson Reuters or something like starts at $5k/mo.

Overall, the model worked fairly well, generating about 10% annual returns even with a few handicaps I put to be on the safe side. This was about an year ago and the only reason I stopped working on it was that my partner in the financial startup quit. I'm sorry if some of my terminology is wrong - I haven't been working in that field since then. If you have any questions, I'd be happy to talk about this. My email is in my profile.

D.E. Shaw, Renaissance Tech, Goldman Sachs, etc. have already hired hundreds (if not thousands) of people smarter than you to do exactly this. They have more Math, Physics, Econ, CS PhDs than you can count/

Does anyone have any experience in doing this?

So take everything with a grain of salt-- there seems to be a lot of advice going around by people who don't have skin in the game-- especially those who say it can't be done. Unless they've put money down to test this, then their opinion should be discounted.

For those saying commissions are to high for a shorter term timeframe: trading has been commoditized. Lightspeed offers .40 per lot, and IBKR offers around that given a certain amount of trading volume. Forex brokers make money off the spread, and so the commission that you pay is only from the bid ask. Since you are trading small amounts, you will get hit on the comission structure, but if you pay per share rather than per trade it won't hurt as bad. You also might want to look into equity futures as well-- very liquid, good tax advantages, and good commission structures.

Biggest thing about small sizes is that you're subject to the pattern daytrader rule-- you can't make >3 intraday trades if your portfolio is <25k. SEC rules.

I'm a discretionary trader, and not a system trader. I am currently developing some systems to go live but they are not ready as of yet.

I also know of many who are quants, and it's possible and profitable.