Ask HN: Is it feasible to do high-frequency trading as an individual?

The only person likely to get rich off that is the broker...

It's not a dumb idea, but you certainly should give up. The high frequency game is one you will lose, because many of the large investment banks have relationships with the stock exchanges allowing them to get information faster - and respond to that information faster - than other traders.

See this NYT article (including the graphic) for more: http://www.nytimes.com/2009/07/24/business/24trading.html

I would ignore the "HFT" moniker and focus on the algorithmic part. HFT is a big-money game for companies who can afford to build a data center next to the exchange. But a longer-term strategy based on technical indicators can still do very well.

My main recommendation would be to take things slowly, browse through the COTS options in the field(which are plentiful), and make some of your own trades in order to learn and develop your strategy.

Check out Interactive Brokers at http://www.interactivebrokers.com/. Their brokerage fees are substantially lower than that.

Cheaper? How. 10/order makes no sense at all if you are ordering 10-20 shares at a time. It makes a lot of sense if your orders are 100,000-200,000 (or more) shares at a time. Individual traders (most often) only do the 10-20 shares or so... Check to see if your broker has limits on the number of shares/order.

High frequency algo trading, as presently committed to by the big banks, relies (as others have noted) on advanced notification (if only, at times, milliseconds in advance...) which, as an individual, you probably won't get. But that just means you can't make use of the same algorithms as the banks. You'll want to derive your own algorithms to exploit some specific pattern(s) that are different from those that rely on advanced word. It's possible, but unless you're already in possession of some wealth, unlikely that you'll be able to design, implement and tweak your algorithm to profitability in anything like a year or so. I certainly wouldn't try it as anything less than a full time occupation... certainly not something that can be done as an 'experiment'.

What about low frequency algorithmic trading? As in you run a program that is looking to make money holding a position for hours to days so the exact buy price isn't as big a deal. Or does it become to much of a game of chance then?

Stay away from HFT, big players will eat your lunch. HFT isn't even a game of milliseconds anymore, it's a game of microseconds. You can still do well with algorithmic trading where speed of execution isn't an issue. Work on your strategy, test it on historical data, execute it manually, if it works then automate part or all of it.

I can't even get real time (meaning not delayed by 15 minutes) stock quotes for my startup here in Brazil.

You should give up ... reason is that HFT costs for the big houses are just a fraction of what you are paying, like less than a penny per trade. You could not hope to gain back in margin, what you lose in transaction costs.

HFT is a game of speed and low fees. Not a single individuals broker can offer substantial speed (microseconds count) and fees (cents count). It is a game for prime brokers and big institutionals. Moreover it is true that some exchanges (like NYSE) offer lower latencies for a serious monthly fee (not the one an individual can afford). Thus I don't think one person can solve all these complexities, however this topic is a good startup field with lots of ineffeciencies and unsolved problems.

Algorithmic trading is an option, though needs careful research on topics of broker setup (InteractiveBrokers should be a good starting point, ThinkOrSwim is another good one, both has rather tolerant programming APIs) and software for time series analysis and trading (among the most popular are Marketcetera and WealthLab + lots of libraries like Incanter and Weka). On some of these topics http://elitetrader.com forum should be helpful.

A little OT: People here are always talking about trading stocks, anyone dabble in forex?

Someone on reddit a few months back was succesful at this:

"I used to work as a software engineer and started developing and trading automated strategies in my spare time in 2006. I went full time in 2007 and have been profitable every quarter since. AMAA"

http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat...

Since everyone has said no so far, I'll go ahead and say it's possible, but very difficult.

HFT algo commission is much lower than $10/trade. IBKR offers 0.008/sh, and it can get much lower if you trade in volume.

Like others said, it is a game of milliseconds. So you need to own a server near the exchanges in new york (i.e. http://www.ubiquityservers.com/data-center/new-york.php).

The best way to start learning is develop algos not for HFT. Instead of milliseconds think about 15 min - daily holds. If you're trading under 25k, you're only allowed to make 3 intraday trades a week.

So find a broker that has a FIX api, buy some data for backtesting, code some stuff out using R/python, and forward test your strat using papertrading for about a month-- then put real money to work.

True high-frequency is very hard. Most of these guys got started in the 1999/2000 time frame and didn't have to deal with many of the start-up issues that new entrants face. For example, just consider the data you need to trade that quickly. Not only is the real-time feed expensive, but if you want to source the data from the execution venue then you have a lot of code to write. The amount of data is also quite large. For example, the US equities market executes approximately 45 million trades per day and there are about 700 million quotes per day, not including the non-top of book quote activity in the various ECNs. I collect between 2 and 4 GB of data per day, if you were to go direct to the execution venues you'd be looking at >50 GB per day, probably.

So, anyway, the "true" high-frequency game is very tough today because we're already 10+ years into it. The markets have changed and the edge has gotten smaller, but there is still plently for a lowly individual automated trader to scratch away at.

Instead of looking for millisecond opportunities, look for second or minute opportunities. Go where the big guys can't because there isn't enough capacity. Can you find an edge that, on average, keeps you in a trade 30 seconds, for example? There's still plenty of alpha left, just don't step onto their playground and expect to get onto the swing set.

A few practical notes:

* You need to look into unbundled or cost-plus commission structures. These fee structures charge a per share commission and pass through all fees and rebates from the executing venue. This is required to do any sort of size with reasonable cost.

* Most "retail" brokers are not sufficient for any sort of high-volume algorithmic trading. Interactive Brokers is barely ok if you are in any way interested in limit order trading because they have fairly large cancel fees for direct routed orders. If you're model doesn't rely heavily on strictly offering liquidity or you're ok with letting IB route your order then IB is ok and offers an unbundled commission structure. Lightspeed Trading and Lime Brokerage are two that cater to active individual and institutions.

* Data storage is a big deal. Effective storage of regular (evenly spaced) and irregular time series will require you to engineer something. There are commercial solutions, but you can't afford them. When you're dealing with high-volume intraday trading this is one of the first issues you'll face. How do you store, query, and manipulate data that includes 45 million new rows per day? The relational DBs fall apart pretty quickly and even if they didn't they won't give you the time series operations you need/want.

* Data feeds are expensive, but required. Look at DTN NxCore. It is a full market feed that will give you the best you can get w/o going direct to the exchange. Some brokers will give you access to raw exchange feeds, but you'll need to engineer feed handlers and a ticker plant for them. This is a non-trivial task, but not impossible. Once you've done that, you'll need to figure out how to get all that lovely data off your co-located server and back to your home base for analysis. Network engineering will be required because your broker doesn't want you pushing 10-20GB per day through their network connection, so you'll need a circuit from an on-premise carrier.

* The banks are players in HFT, but not the original or best. Most of the guys that started it are still independent. Look at GETCO, RGM Advisors, etc.

* Flash orders, what most folks in this thread are refering to when they say the exchange gives the HFT firm a first look, are no more. That edge existed, and I'm sure HFT took advantage, but no HFT firm was built on flash orders. When they started flash orders didn't exist.

It is possible to be a successful, independent, automated trader. It is even possible to do it on a purely intraday, high-volume basis. Don't get caught up in the hype of needing to be high-frequency or not.

---

EDIT: The other ...

If you're not connecting directly to the exchanges, no.

Successful HFT desks are either large broker-dealers, or sophisticated hedge funds, both of which are well-capitalized, have expensive infrastructure, great location (close to exchanges to minimize lag), ultra-low trading commissions etc. Furthermore, only the hedge funds are pure prop groups. Most the broker-dealer desks act as market makers and aggregators for internal flows (almost free money).

So it's going to be very hard for you to compete with them, even if you come up with better algorithms.

I recommend looking at mid-frequency or low-frequency algorithms, which will offset the impact of your higher trading commissions and don't get affected much by infrastructure and location issues.

And yes, $10 per order is too much unless you talking really big money. Check out IB, FXCM etc.

your internet-based API is nowhere near fast enough to compete with legit HFT firms. low to medium frequency with longer holding periods, sure, but real HFT via your broker's API? not gonna happen.

As others have said, I think focusing on HFT is a bad idea. You are going to be in competition with other players in the market. What are your weaknesses compared to them:

1) You are alone; you are competing with teams mixing 3-4 math/physics ph.d.'s and 3-4 programmers that might implement their algos on the latest GPUs (and this is very time consuming).

2) You have no experience. You are competing with very smart people that have been playing this game for years.

3) You (probably) have little money. So you want to minimize cost of historical data acquisition, data storage, data analysis, hardware, collocation …

IMH, these points pretty much rule out HF trading.

You also have some advantages compared to the competition. One of them is that you have little money, so you can invest in assets that are illiquid to someone that wants to move a lot of cash.

I think it is possible to trade as an individual; hard but possible. And if you are successful at some non-crazy frequency -- a few days, a few hours, a few minutes -- then, maybe, maybe have a look at HF.

And another thing, if you can, go shopping for a broker with lower fees. $10/trade? It looks as if you did not do basic homework -- but mabe you live in a country where that's the cheapest you can get.

I like lrm242's comments. I started my own high freq algo firm back in 2007 and crushed it for awhile. In the end though, the governing dynamics of the marketplace change and can change very rapidly, so any machine learning or neural networking you do perform can become obsolete fairly quickly. Unless you've developed HAL you're probably going to have a rough go of it alone...hope this helps!

High Frequency trading, is really meant for people or companies that have their servers in the same network as the exchange they are HF'ing on. In fact, true HF'ers buy and sell in fractions of a second, some lasting seconds, or minutes.

I've researched this, and tried doing the same, but, in the end, trade fee's, and commissions kill the idea dead. In fact, HF Trading falls under Day Trading, which required that you must have the legal minimum $25,000 in equity on hand to day trade... that is the main thing that killed it for me... read for yourself: http://en.wikipedia.org/wiki/Day_trading

If you'd like to try it out before you trade live (here comes the shameless plug), you can use the trade simulation I made (buy/sell/sell short) and set your trade fee's and commissions totally free at https://algxchange.com

As an individual it is unlikely. Your best bet would be to develop an algorithm with massive amounts of recent historical data that shows you could hypothetically be profitable. Shop that to trading firms or angel investors and use their capital to get a direct feed. You probably won't be able to do this to a major exchange, but there are lots of other options. If you find a smaller market you can also potentially get a market maker agreement. This will mean that you have to have certain positions open at all times, but you can place any orders you want in addition to those, and all of them will have $0 fees.

All that said, I would suggest you stick with vanilla algo trading to start, and leave HFT to the big boys (until you are one).

I'll share some experience a friend had with this stuff. I don't have the low-level details, but this should be enough to make an informed decision.

He was working on what he termed "high speed automated trading" -- basically all in this realm of picking pennies up in front of bulldozers.

He's a very smart and talented guy -- Harvard Physics/Comp. Sci double major, etc. So we're not talking about an amateur throwing around some code.

He spent about 6 months and over $50k setting up his system -- he had developed some algorithms to trade spreads between different securites (on the CBOE, the NYSE, and NASDAQ). He built a relationship with a clearing agent, a direct broker, etc. He got machines in colo facilities as close to the exchanges as possible (one data center in Chicago, one in Jersey City, etc.) SLAs on low-latency DS3 lines, etc. The whole nine yards.

I'd call it a "pro-sumer" level setup -- everything done by one guy, but done basically as well as a bigger firm would set it all up.

Here's what he discovered after about a week of trading: He didn't have even remotely a shot at competing. Like, not even close. If he had 10ms pings to the exchanges, someone else had 5ms. If he got down to 2ms, someone else who was physically at the exchange itself had 1ms. He got killed with commissions -- even if he did make a few $$ on some trades, it vanished with commissions.

Why? Because he was competing against guys who paid NO commissions. The broker-dealers and clearing companies themselves had internal automated trading setups. They had deeper relationships and deals that traded commissions for a cut of the profits, etc.

In other words, he figured out pretty quickly that this is not an algorithm game or speed game - it, like many things on Wall Street - is a 'who you know' game. No matter how smart, how fast, how sophisticated you are as an outsider, the likelihood is that someone on the inside has a similar trade idea and can do it faster and cheaper than you.

Now, the corollary to that: My friend is smart, but he also tends to give up too easily ;) Clearly if you've got an idea for a better/smarter/more innovative trade, then you'll make money on it. But that's a trade idea, not a speed advantage.

So just recognize that there's potential to make money, but "same idea, just faster" won't cut it.

No.

Your ping time will be > 20ms. factor in a bit of latency in the data feed and call it 30ms. Nyquist says you'll get aliasing unless you sample 2xfreq. So The highest frequency you can possibly trade is 60ms. Not at all high frequency.

And I really doubt you'll get a 20ms ping time. I get 100ms to my broker. Can you do mid-frequency trading? Absolutely. But you don't have the money or access to resources required for hifi trading.

Also, $10/trade is VERY high. I'm using interactive brokers, which is $2/side+some costs which are small enough I don't bother accounting for them.