Anyone who does algorithmic trading, what platform do you use? Do you roll your own? I’ve had a passing interest in this field for years but have not done anything myself.
That depends on what you mean by "platform." Professional quantitative traders do not use services like this one. Virtually all of them - profitable and otherwise - work within firms that have separate teams for strategy research, strategy development, infrastructure, IT and internal software development. Typically there are sub-teams specializing in research and implementation (each) for risk management, execution, data processing and portfolio optimization.
So the most honest answer to your question, insofar as the question makes sense, is that algorithmic traders roll their own under the auspices of the firm employing them. In fact the way that this service structures their pricing is strange and unintuitive to me, and I can't imagine professional traders would agree with it.
While I have known people to trade on their own in a way that can be legitimately called "quantitative" (and even do so profitably), they all worked within a firm before doing that, and all of them targeted strategies that allowed them to capture a greater amount of the profit for an otherwise small total revenue that would generally be considered too illiquid and capacity constrained for institutional trading.
Quantitative trading in a solo capacity is basically a myth. Most of those who actually have a non-trivial attempt at it think they're doing it because they're working with a large amount of data and doing "backtesting", but that's well below table stakes. To put it in perspective, that's like saying you wrote and maintain an internet-wide search engine half as good as DuckDuckGo. It's a meteoric amount of work for one person to keep all aspects of an algorithmic trading strategy running smoothly.
Most of those who actually have a non-trivial attempt at it think they're doing it because they're working with a large amount of data and doing "backtesting", but that's well below table stakes.
Can you define "table stakes" used here? The tone is a bit dismissive, but if there are individual investor doing algo trading with a few $100k and making some multiples of 10% gains annually, in bear and bull markets, seems like non a myth to me? Ie a couple of these people post here on occassion.
Before I answer your larger question, I'm going to outright contest this particular claim:
> a couple of these people post here on occassion.
I'd be frankly shocked if there are more than five people active on HN who are "doing algo trading with a few $100k and making some multiples of 10% gains annually, in bear and bull markets" on their own, with no outside capital. There might be folks automating some kind of trading that relies on fundamental financial analysis empowered by information asymmetry, but that's not really the same thing (though it's also hard). The implicit idea of "algorithmic trading" is that an automated trading strategy can capitalize on an (empirically true) hypothesis about the world. That means it should be consistent; automating certain deployment, execution and risk management processes isn't sufficient for algorithmic trading. That's just empowering the trading process with software. You can't really deduce a rigorously defensible and profitable trading algorithm just by filtering a giant amount of data through a backtesting engine.
As for the larger question, table stakes for a quantitative trading outfit is typically 1) a strategy research team that identifies and analyzes opportunities primarily via mathematical modeling, 2) a strategy implementation team that develops and deploys the algorithms which capitalize on those opportunities and 3) an infrastructure team that develops the internal research, data processing and trading platforms used by the former two teams. More abstractly, these are three separate functions (research, implementation and maintenance) spanning several core competencies (advanced math/theory, software engineering and reliability). Backtesting and whatever flavor of the week definition of "big data" are generally involved, but they're a relatively smaller part of a much larger engine in the context of quantitative trading. Backtesting in particular is extremely sophisticated and, in some ways, largely automated in the context of professional funds.
Theoretically one person (or a very small team) could be competent at each of those functions, especially if they're working with a relatively small amount of capital and targeting market inefficiencies that are too small, niche or illiquid to be productive and profitable for institutional trading. But that kind of individual almost certainly cut their teeth in an actual firm before going solo, and if they are working solo then they're doing way more than just finding useful data, using it to deduce a trading strategy and backtesting that trading strategy across historical data. It's comparatively easy to develop an algorithm that takes advantage of information asymmetry using a combination of domain knowledge, intuition and some untapped data source. But insofar as quantitative trading means something other than a marketing term, that's an insufficient level of modeling and an orthogonal use of data to satisfy the bar.
The other obstacle is that the stars kind of have to align in a very strange and particular way for someone to be not only competent enough, but incentivized enough to crank this entire Rube Goldberg machine on their own and with their own capital. Assuming their skills are generalizable, a hypothetical person who can be profitable while doing all research, implementation and infrastructure work on their own can likely pull down mid 7 - low 8 figures in annual compensation at a number of different hedge funds with much better career stability, financial security, personal risk and work life balance. If nothing else they could probably scale up and hire 10 or so others to found their own fund, and it would be strictly irrational not to (unless they're already capacity constrained, in which case they can't).
To clarify; if you read this persons history, they have automated a strategy that’s been very profitable, according to them. Is this algorithmic trading in the same sense you mean?
- Quantopian: Was good, has tons of data and makes things very easy. Unfortunately you can't easily live trade your algorithms anymore.
- QuantConnect: Open source, works okay, at the time I wanted it for options trading though and their support for options chains isn't so good.
- Zorro: Closed source, paid, uses a weird custom variant of C but works surprisingly well. Mainly focused on forex though.
- Others: There are others, like this one, that are mostly unusable because they miss something essential, like broker support or data.
QuantConnect is the only real packaged option these days I think but it's also not hard to roll your own. There are brokerages with _really_ easy to use APIs (Tradier is the first that comes to mind).
I do some algorithmic trading and automated investment in the crypto space. I trade intra-exchange and do arbitrage between pairs (tri-arb). I also occasionally rebalance into an index of coins I like. Note that I do this at a fairly small scale, so not sure if my answer really matters much.
Although there are a lot of platforms for trading crypto, when it comes to arb, I found that you need to have control over a lot of things, most importantly:
- which data center you host your code in
- the way you get data from the exchange - not just prices and order books, but also current balances and orders
- the way you push orders to the exchange
With "the way" I mean mostly which APIs you use and how, the last few milliseconds of optimization for me were usually gained by making multiple simultaneous connections to the exchange and trying to figure out the fastest one.
For my rebalanced index-like portfolio I made a little script initially and then turned it into a side project, plug: https://nazcabot.io
Don't the fees negate the profitability of this strategy? How often do you see the spread between the rates achieving profitability when factoring in taker fees?
Trades are only triggered when profit is > fees. Depending on the market there can be fairly large opportunities popping up (on the thousands) a few times an hour. In a triangle, you only need 1 of the 3 pairs to have an inefficiency in the book to trigger a trade.
During the december craze: very profitable. Now: better than not trading.
If I were american and had to pay higher taxes on short term gains, it would probably not be worth it.
Can you share compensation info and work life balance? Ive always wanted to break in on this industry? Also, what types of skill set they are looking for?
Work life balance depends on the firm. If you're talking say Virtu or Citadel (two of the best equities players), you have no life, but you're payed a king's ransom. I was single and hungry to be the best technologist possible and it was great before I got married. I worked for Madison Tyler (Virtu's original trading company which finally merged with it for almost 5 years). Others are more amenable to a sensible work/life balance, but it really just depends. DRW and Jump Trading are generally known to have pretty solid work/life balances, but again it just depends on the team and what you do.
The way to break into finance, besides some luck or a recruiter, is being exceptional at what you do. If you write C++, be the best C++ developer you can be. If you tune Linux servers, you better know every sysctl knob literally in and out AND where in the kernel those are actually applied. If the knob doesn't work, perhaps you'll need to write your own (kernel patch) to toggle a specific tcp/ip tunable. I breathe Linux in my sleep because I've always found it fascinating and got lucky when a random recruiter found me while I was working on the Core Systems backend Linux team at Ticketmaster. Then it is simply a matter of putting your resume out and/or finding a recruiter who works with those firms. Some of them are more niche. Jane Street Trading, as an example, requires every employee to write Erlang. Being good at erlang isn't super common so that might be an "in". Two Sigma does a lot of open source work (they have a github and have contributed a lot of stuff to both the Mesos and Kubernetes communities). Honestly man, just do some research, find something you want to learn, and strive to be the best.
Hi Jeff, thank you for sharing man! I'd like to be a quant researcher because I think I have an edge on math and statistics. Can you tell me a bit about the quant team at Jump? Do they have Phds, know machine learning?
flask + postgres + keras - it’s rather easy to train models based of arxiv research papers. eg i trained a model using this method (https://arxiv.org/abs/1601.01987) and trade fairly frequently on minuscule amounts using the kraken (cryptocurrency exchange) api.
Another question to who does algorithmic trading, is a software / engineering education with a strong math and data analysis useful to get into that?
Do you have some particular resource where someone can start to approach it?
You can get a degree in it. I've considered applying,(i live in Atlanta) but I have a family to feed and can't just quit my job for a new degree. Also most of the big money is in NYC so you'll likely move...but you'd retire in 10 years because you either made a lot of money, or have a lot of stress.
Algorithmic trading is the epitome of a meritocracy. I'm highly skeptical of anyone offering a degree in algorithmic trading other than as a historical perspectives. It's an evolving field and extremely cutthroat. I would liken it to going to film school and trying to become Steven Spielberg.
The industry has no customers to win over, no sales pitches, no marketing. You either make money or not. I'm skeptical of a degree for several reasons:
* Knowledge of what works this year may not work next year.
* Teaching people increases competition and dilutes any edge you might have.
* If they could earn a living doing this, why wouldn't they be doing that instead of teaching classes?
I'm unique hybrid as I have a MBA (finance) before the great depression and because of the great depression I couldn't find a job. So I taught myself how to code (~2010) and have been working in code ever since, while still doing some algorithmic trading on the side.
I build custom trading systems on the side. Interactive Brokers is the only firm that I have found that has a good API. TD ameritrade seems to be starting in that direction. As for Trader9, well best of luck. For a little self promotion...
I built my own platform too after discovering that most public platforms dont’t support Options, are slow on backtests or have bugs.
Thank you for the api list! In general the IB api is great to work with but it has some smaller annoying bugs. For example the market data api suddenly stops sending updates for the VIX and requires a re-subscribe.
Being that the account that submitted this is still green, I'm going to assume that the person behind it is either a user of Cloud9Trader or affiliated with it. If that is the case, I have a few questions:
1. Who is the intended demographic here? Based on pricing and features there's no way this is intended for institutional funds, so is this targeted at amateurs or small professionals? It states it's targeting "beginners to get grips with quant trading for free", but that strikes me as a poor market to try and sell to.
2. Can users' algorithms be seen by the platform? If so, that's a non-starter for many professionals.
3. Does Cloud9Trader provide historical data or just compute power? If historical data is provided, what resolutions are available?
4. Which brokers are supported? It seems like just FXCM and Oanda; are there plans for others?
5. How are backtest compute units measured? The pricing page states:
"A backtest compute unit is a measure of processing resource made available for backtesting. 10 units is equivalent to one month of backtesting every tick, or 10 months of backtesting minute data."
but it's unclear to me how an atomic unit of compute time can be generalizably the same for every user's backtesting workflow.
6. What is the non-pricing data you're providing? It looks like you're just reselling premium geographic data from Quadrant and free data from Quandl, but the naming scheme is not clear on what it's providing.
I guess while I'm here I might as well make this point: I'm deeply pessimistic that this tool would meaningfully improve the productivity or profitability of real-world quantitative trading. I also don't think it makes sense to target "beginners in quant trading" because quantitative trading is one of those things which mostly doesn't (and shouldn't) exist outside of teams and companies. I consider it somewhat predatory to offer a service that implicitly encourages inexperienced non-professionals to get into this - quantitative trading is very unlike programming and entrepreneurship in that there is no straightforward way to teach yourself or bootstrap.
That algorithm would have a poor Sharpe/Sortino/Calmar ratio. The risk-adjusted return is bad, though not as comically bad as some of the others on the homepage with 40%+ drawdowns.
I have a question for those in the field - is there a good tool to fingerprint the level of algorithmic trading going on in a particular market? I trade in a lot of very obscure and illiquid stocks and in some I can spot obvious bot trading, but in others I see none.
- Don't buy at the opening bell
- Volume = momentum (apply law of gravity here)
- Algo must reflect the Day Traders "spirit" (profit taking = specific % downward)
- Uniform circular motion is the key
Fortunately, the institutional traders who use that Cloud9 will never use this Cloud9 and the retail traders who use this Cloud9 will never use that Cloud9.
45 comments
[ 1.9 ms ] story [ 93.7 ms ] threadSo the most honest answer to your question, insofar as the question makes sense, is that algorithmic traders roll their own under the auspices of the firm employing them. In fact the way that this service structures their pricing is strange and unintuitive to me, and I can't imagine professional traders would agree with it.
While I have known people to trade on their own in a way that can be legitimately called "quantitative" (and even do so profitably), they all worked within a firm before doing that, and all of them targeted strategies that allowed them to capture a greater amount of the profit for an otherwise small total revenue that would generally be considered too illiquid and capacity constrained for institutional trading.
Quantitative trading in a solo capacity is basically a myth. Most of those who actually have a non-trivial attempt at it think they're doing it because they're working with a large amount of data and doing "backtesting", but that's well below table stakes. To put it in perspective, that's like saying you wrote and maintain an internet-wide search engine half as good as DuckDuckGo. It's a meteoric amount of work for one person to keep all aspects of an algorithmic trading strategy running smoothly.
Can you define "table stakes" used here? The tone is a bit dismissive, but if there are individual investor doing algo trading with a few $100k and making some multiples of 10% gains annually, in bear and bull markets, seems like non a myth to me? Ie a couple of these people post here on occassion.
> a couple of these people post here on occassion.
I'd be frankly shocked if there are more than five people active on HN who are "doing algo trading with a few $100k and making some multiples of 10% gains annually, in bear and bull markets" on their own, with no outside capital. There might be folks automating some kind of trading that relies on fundamental financial analysis empowered by information asymmetry, but that's not really the same thing (though it's also hard). The implicit idea of "algorithmic trading" is that an automated trading strategy can capitalize on an (empirically true) hypothesis about the world. That means it should be consistent; automating certain deployment, execution and risk management processes isn't sufficient for algorithmic trading. That's just empowering the trading process with software. You can't really deduce a rigorously defensible and profitable trading algorithm just by filtering a giant amount of data through a backtesting engine.
As for the larger question, table stakes for a quantitative trading outfit is typically 1) a strategy research team that identifies and analyzes opportunities primarily via mathematical modeling, 2) a strategy implementation team that develops and deploys the algorithms which capitalize on those opportunities and 3) an infrastructure team that develops the internal research, data processing and trading platforms used by the former two teams. More abstractly, these are three separate functions (research, implementation and maintenance) spanning several core competencies (advanced math/theory, software engineering and reliability). Backtesting and whatever flavor of the week definition of "big data" are generally involved, but they're a relatively smaller part of a much larger engine in the context of quantitative trading. Backtesting in particular is extremely sophisticated and, in some ways, largely automated in the context of professional funds.
Theoretically one person (or a very small team) could be competent at each of those functions, especially if they're working with a relatively small amount of capital and targeting market inefficiencies that are too small, niche or illiquid to be productive and profitable for institutional trading. But that kind of individual almost certainly cut their teeth in an actual firm before going solo, and if they are working solo then they're doing way more than just finding useful data, using it to deduce a trading strategy and backtesting that trading strategy across historical data. It's comparatively easy to develop an algorithm that takes advantage of information asymmetry using a combination of domain knowledge, intuition and some untapped data source. But insofar as quantitative trading means something other than a marketing term, that's an insufficient level of modeling and an orthogonal use of data to satisfy the bar.
The other obstacle is that the stars kind of have to align in a very strange and particular way for someone to be not only competent enough, but incentivized enough to crank this entire Rube Goldberg machine on their own and with their own capital. Assuming their skills are generalizable, a hypothetical person who can be profitable while doing all research, implementation and infrastructure work on their own can likely pull down mid 7 - low 8 figures in annual compensation at a number of different hedge funds with much better career stability, financial security, personal risk and work life balance. If nothing else they could probably scale up and hire 10 or so others to found their own fund, and it would be strictly irrational not to (unless they're already capacity constrained, in which case they can't).
To clarify; if you read this persons history, they have automated a strategy that’s been very profitable, according to them. Is this algorithmic trading in the same sense you mean?
Bingo !
- Quantopian: Was good, has tons of data and makes things very easy. Unfortunately you can't easily live trade your algorithms anymore.
- QuantConnect: Open source, works okay, at the time I wanted it for options trading though and their support for options chains isn't so good.
- Zorro: Closed source, paid, uses a weird custom variant of C but works surprisingly well. Mainly focused on forex though.
- Others: There are others, like this one, that are mostly unusable because they miss something essential, like broker support or data.
QuantConnect is the only real packaged option these days I think but it's also not hard to roll your own. There are brokerages with _really_ easy to use APIs (Tradier is the first that comes to mind).
Although there are a lot of platforms for trading crypto, when it comes to arb, I found that you need to have control over a lot of things, most importantly:
- which data center you host your code in - the way you get data from the exchange - not just prices and order books, but also current balances and orders - the way you push orders to the exchange
With "the way" I mean mostly which APIs you use and how, the last few milliseconds of optimization for me were usually gained by making multiple simultaneous connections to the exchange and trying to figure out the fastest one.
For my rebalanced index-like portfolio I made a little script initially and then turned it into a side project, plug: https://nazcabot.io
During the december craze: very profitable. Now: better than not trading.
If I were american and had to pay higher taxes on short term gains, it would probably not be worth it.
Disclaimer: I've worked for two of the largest HFT firms in the US over the course of the past 11 years.
The way to break into finance, besides some luck or a recruiter, is being exceptional at what you do. If you write C++, be the best C++ developer you can be. If you tune Linux servers, you better know every sysctl knob literally in and out AND where in the kernel those are actually applied. If the knob doesn't work, perhaps you'll need to write your own (kernel patch) to toggle a specific tcp/ip tunable. I breathe Linux in my sleep because I've always found it fascinating and got lucky when a random recruiter found me while I was working on the Core Systems backend Linux team at Ticketmaster. Then it is simply a matter of putting your resume out and/or finding a recruiter who works with those firms. Some of them are more niche. Jane Street Trading, as an example, requires every employee to write Erlang. Being good at erlang isn't super common so that might be an "in". Two Sigma does a lot of open source work (they have a github and have contributed a lot of stuff to both the Mesos and Kubernetes communities). Honestly man, just do some research, find something you want to learn, and strive to be the best.
https://www.scheller.gatech.edu/degree-programs/interdiscipl...
I'm unique hybrid as I have a MBA (finance) before the great depression and because of the great depression I couldn't find a job. So I taught myself how to code (~2010) and have been working in code ever since, while still doing some algorithmic trading on the side.
I build custom trading systems on the side. Interactive Brokers is the only firm that I have found that has a good API. TD ameritrade seems to be starting in that direction. As for Trader9, well best of luck. For a little self promotion...
http://www.strategic-options.com/insight/the-best-and-worst-...
Since you've been alive for over 100 years, what's the key to a happy life?
Thank you for the api list! In general the IB api is great to work with but it has some smaller annoying bugs. For example the market data api suddenly stops sending updates for the VIX and requires a re-subscribe.
1. Who is the intended demographic here? Based on pricing and features there's no way this is intended for institutional funds, so is this targeted at amateurs or small professionals? It states it's targeting "beginners to get grips with quant trading for free", but that strikes me as a poor market to try and sell to.
2. Can users' algorithms be seen by the platform? If so, that's a non-starter for many professionals.
3. Does Cloud9Trader provide historical data or just compute power? If historical data is provided, what resolutions are available?
4. Which brokers are supported? It seems like just FXCM and Oanda; are there plans for others?
5. How are backtest compute units measured? The pricing page states:
"A backtest compute unit is a measure of processing resource made available for backtesting. 10 units is equivalent to one month of backtesting every tick, or 10 months of backtesting minute data."
but it's unclear to me how an atomic unit of compute time can be generalizably the same for every user's backtesting workflow.
6. What is the non-pricing data you're providing? It looks like you're just reselling premium geographic data from Quadrant and free data from Quandl, but the naming scheme is not clear on what it's providing.
I guess while I'm here I might as well make this point: I'm deeply pessimistic that this tool would meaningfully improve the productivity or profitability of real-world quantitative trading. I also don't think it makes sense to target "beginners in quant trading" because quantitative trading is one of those things which mostly doesn't (and shouldn't) exist outside of teams and companies. I consider it somewhat predatory to offer a service that implicitly encourages inexperienced non-professionals to get into this - quantitative trading is very unlike programming and entrepreneurship in that there is no straightforward way to teach yourself or bootstrap.
So the following community algo (https://www.cloud9trader.com/algorithms/1085/code) claims to have averaged 0.6% monthly profit, which is a 7.2% annual return. Actually pretty awesome.
Why is it only trading between 5AM and 6PM?
5am -> 6pm is most likely targeting a specific trading "session". See: https://www.investopedia.com/articles/forex/08/3-market-syst...
- Apply algo between 9:30am and 1:00pm (greatest price movement)
- Keep in mind, greed is the destroyer...small gain but often and constant
Can you elaborate on this?
https://www.c9tec.com/trader-communications/
https://xkcd.com/1570/