131 comments

[ 3.0 ms ] story [ 179 ms ] thread
When HFT firm Virtu went public they announced they had one loosing day in 6 years.

A physics professor explained how this was possible by using the law of large numbers and some basic assumptions.

I was offered a job at virtu, and met people who discussed this in the interview. They had backtested the model but apparently it had a slight bug. After fixing it they claimed to have made much more than the loses.
I don't know why HFT exists at all. I would just pass a law that forces a bit of random latency/noice in the market data, in the order of seconds (in a similar way as GPS has artificial inaccuracy). Normal people are not gonna notice and all this HFT garbage is completely eliminated. Win win.
People who are pro-HFT see it as a way of increasing the 'liquidity' of the market. In their eyes it makes it easier to move volume of stock, and moves the bid price of the stock closer to the ask price of the stock.
That argument still makes no sense to me. How does skimming off of the top of trades make the market more liquid? It's not like they're offering any new product for sale. If a market is moving slowly all they do is artificially inflate the number of transactions without increasing liquidity.

HFT is just abusing an information advantage to skim off of the market. Because they have a slightly more accurate view of the market, they can front run trades and make money. That's all it is. There is no benefit to the seller, to the buyer, or to the market. It's pure parasitism.

before HFT these transactions were run by human market makers from giant banks who skimmed hundreds of times more profit off of them (this is the reason that you hear about it in the news all of the time)

Someone's job is going to be to sit between the people who just want to sell and those who want to buy- when we talk about liquidity it's just this. The fact that you're directing your anger towards some robots who are ruthlessly driving each other to the minimum possible gap between the buy price and the sell price on this is bizarre

I have a pet theory that anti-HFT sentiment is stoked by banks and other entities that aren't able to compete.
Unless you're a bank or have similar means you can't compete.
I'm not sure I understand the argument. The publicly traded top trading firms are much smaller than the banks. They're in the $1-2 billion range.

Of course you need money to compete. You need money to compete with Facebook too.

Heaven forfend that people should form their own opinions on first principles, even though those opinions might be mistaken.
Does HFT only hurt "other bots"? My understanding was that HFT could take advantage of faster feeds to front-run human traders.
It hurts anybody who sells a stock.

You sell a stock at a certain price, someone else looks up the price of the stock and sees that it is slightly higher than what you listed, because your information is still propagating through the network. That person offers to buy the stock at the higher price. The market matches your offers up. Then the HFT steps in and buys your stock and resells it to the buyer at the price he asked.

There is no value to the seller or buyer. Zero risk to the HFT firm. No improvement in market conditions--the HFT firm doesn't sit on positions, they can't increase liquidty. Just free money from abusing a latency advantage on their view of the market.

Something is providing the liquidity that has brought spreads down over the last couple of decades.
Computerized trading. Sellers basically publish their positions on a server and buyers query for the best price on a stock and the place the order. None of this requires or benefits from HFT.
Can you explain this in terms of actual orders placed? How do you "sell at a certain price"?
Respectfully, this is just plain incorrect. You're describing some situations that do happen and just choosing to ignore what happens the rest of the time. When demand precisely matches supply, you're correct--you need a matching engine to connect buyer and seller and no middle man/market maker need get between them. That also happens too.

Assuming that at some point bids and asks will line up and trades will happen doesn't a market make; that's the exchange's job. Making a market means being able to quote prices on both sides of the book and having inventory to trade at the current market price. Carrying that inventory has actual risk involved, and that's why it's not a free service that the exchange (or anyone else) provides. Market makers also get penalized by the exchanges if they're not making markets for some large percentage of the time that the products are trading.

Maybe a better way to look at it is that the exchange is there to match up buyers and sellers at the current price at a given moment in time. The market matches up buyers and sellers at the current price over a period of time. This market stabilizes the price over time. Demand and supply just don't line up perfectly like it seems like they should. That's the difference between the market and the exchange. If I'm buying a product now, I want an idea of the true value of it, and the less the price is whipping around waiting for demand to match the supply, the more I know what the current market rate is.

I'm not saying they're doing gods work, but to say that there's no risk involved or reason for them to exist is incorrect.

If you don't buy any of the above arguments, then I'm interested in your answer to the question of why the exchanges themselves pay market makers to make markets. If the current technology renders market makers obsolete, surely the exchanges would recognize that and keep the money for themselves, no?

No exchange operator is doing "gods work" :) Seriously though, you are correct. Market makers are an integral part of how the markets function. Do they skim money from trades because they are better equipped to read the market at a given point in time? Absolutely. Do they also provide liquidity for the retail trader in a given moment to buy/sell shares? They definitely do. Market makers are not a new facet of trading systems, they are just a natural fit for HFT firms. Would it be better for your order of X shares of N company to go unfilled, or to pay .01 cents more per share to have it executed when you want?
Market making and HFT are different services, they may be done by the same company, but you don't need sub-ms views of the market if you are actually holding positions.

This is why HFT firms end every day with empty books.

Literal front running is illegal.

The thing to accuse them of is book stuffing.

You are right, the Wall Strret banks are known for their rigid moral codes and absolute devotion to the law over the potential for free profits.
Thank you. I always have a hard time getting this message across.
HFT aren't the firms putting the markets on computers. HFTs sit between the computers and the retail traders and use that information advantage to front run.

There's no reason there needs to be a middleman here. Buyers can buy from sellers directly. In fact that's what they think they are doing, except that the HFT firms are basically adjusting the price on them underneath the sheets.

The real losers are the sellers. They put up some product at a price, someone else on the other side of town sees a slightly different price because the market only moves at the speed of light. Instead of the seller getting a little bonus, that bonus is hoovered up by the HFT guy in the middle who has a more accurate view of the market.

What service did the HFT provide? Where did this mysterious liquidity come from?

In the old days the middlemen were responsible for actually finding the guy making the sale and matching him up with the buyer. This was real work and it makes sense that people would be paid for it. Now the computer does it for you, there is no need for a middleman and no value he can offer. It's a parasitic relationship.

Traditionally market makers do more than match orders from other parties:

https://en.wikipedia.org/wiki/Market_maker

This is because sellers don't always want to wait around for buyers and vice versa.

Sellers that are worried they have poor market visibility can just put in a limit order that they believe represents a fair price.

The liquidity comes from the HFT market maker's own money. Market makers are required to post bid and offers of a reasonable level of liquidity in all symbols they are registered to "make markets" in. It may seem like a simple skim off the top operation, but it's not that simple. If they do not maintain their bids/offers, they are disqualified from market making.

If they do maintain them, they reap a number of benefits including discounted pricing from trading platforms and IIRC, the ability to do naked short selling (which IMO should be off-limits for all trading firms). I'm not defending HFT here, just stating a fact that market makers are a part of the trading ecosystem.

Keep in mind that "making" and "taking" liquidity is not the same thing as buying and selling. Market makers are required to post both bids and offers. The difference is that market makers put their orders (buy or sell) on the "book", which means they are offering liquidity in both directions. The order that comes in to match (think "market" order to buy or sell) is the "taker". Firms that supply liquidity are rewarded by trading platforms (unsurprisingly, since those firms "make" their market), and firms that match those orders (takers), are charged for the service.

Not all HFT outfits are "market makers", but many are. I can't say specifically if Virtu is a market maker, even if I remembered :) The point is that liquidity providers don't pay for their trades, they are paid for them, so it's a natural fit for a smart HFT operation.

> which IMO should be off-limits for all trading firms

Out of curiosity -- why?

aka pure capitalism, which is very darwinistic, by design.
I can't help but to think the problem is being intentionally framed in an unhelpful way.

The problem is their ability to front-run people. They shouldn't be able to get a more accurate view of the market than anybody else, and they certainly shouldn't have the chance to roll-back their actions after they get a glance of the results. Frequency of trading is irrelevant.

If an HFT firm was "front running" as you put it, and you know about it, why doesn't the SEC? Front running is not legal. Or are you using the Flash Boys definition of front run which is "anyone that trades better than me" or "anytime I try to buy 50,000 shares but move the price"?
The SEC doesn't have infinite resources, but has to enforce somewhat selectively. It's possible that they disapprove of some kinds of HFT activities but doubt their own ability to persuade a jury sufficiently well to justify the cost of prosecution.

I don't mean this as a comment on HFT or the law, I really don't know. I'm just pointing out that the SEC is bound by budgetary constraints and that litigation is very expensive, so they have to do cost/benefit analysis and prioritize the cases that are more likely to win.

And this is before political considerations come into the picture. News articles described the 'resident's SEC pick as being more interested in capital formation than enforcement; that seems a pretty reasonable assessment to me, given that S&C was representing Goldman Sachs during the epic CDO litigation, the cabinet is stuffed with GS alumni, and the administration's general attitude seems to be less regulation for more muscular and dynamic capitalism.

http://www.reuters.com/article/us-usa-trump-sec-idUSKBN14N1Y...

Firms with enough money to spend _can_ get a more accurate view of the market. The issue is proprietary data feeds. A trading platform is required to disseminate information to the consolidated tape at the same rate they do for a private feed, but that only holds for the "walls" of the trading platforms network. The consolidated tape is a LOT slower than a 100MB multicast feed being slurped up by a co-located server sitting 20 feet from the trading system.
There's just a huge number of securities you can trade now. Most equities have monthly options (so e.g. about ~50 calls & puts * 6 months) and the indices have weekly now. And those are just the basic derivatives, the OTC stuff gets even crazier.
Interesting that you "don't know" something and would immediately start passing laws about it...
No they would not. Even if you introduce a 10 sec delay, there will still be someone which will be first after the 10 sec pass, and you can still create models which will be tuned to predict what the price will do 10 sec into the future.

And what do you think about a law which forces all news agencies to wait 1 hour before reporting major news, so that everybody gets a chance to report it, and not only the huge agencies/papers like AFP of NYT.

GP specifically said "a bit of random latency". Don't change the terms of his argument to make it easier for you to refute, that's dishonest.
There's no reason HFT can't or wouldn't exist even if you add random latency.
I agree that it's likely not particularly useful for the market to care about things in millisecond resolution.
It is useful if you start thinking about products whose prices depend on the prices of other things.

Suppose that I'm a SPY trader. I need to know where a bunch of other things are trading in order to figure out where I should be quoting SPY.

If I only know those prices on a second stale basis then I can't get my hedge off and so I can't quote SPY as tightly. I lose, investors lose, and investment banks win because they're better adapted to trading where prices aren't clear.

That's a very clever comment, but I think you should include something indicating why for those who don't know how HFT works.
If nothing else, it means that firms that can afford to collocate their servers in the cage next to the exchange's servers have an advantage over those that can't.

It seems like something like the roundtrip time for a packet from NYC to Tokyo would be a fair "speed limit" for exchanges, and ensure that all firms, globally, are on an even footing.

I think you'll find, when you try to get down into it, there's no easy way to enforce a speed limit. It's a naturally arising behaviour. You could try making a market that crosses orders every X seconds, but then you move the speed to the edge of those periods.

If you wanna slow down HFT, then kill the stupid restriction that stocks be priced in pennies. With 8 digits of resolution, HFTs would then be forced to compete on price.

That's an interesting idea. Is that rule just a holdover from the days of manual accounting?
From my own conversations with exchanges -- I think this is just the exchange appealing to layman investors. The exchange wants to appear liquid, and they like to do this by having a lot of shares on the best bid or offer. If you reduce the tick sizes, you'll definitely see fewer shares on the inside bid or offer (even though you'll see more shares on the inside x% for any x, aka. you'll see more liquidity), but the layman investor (e.g. mom and pop, retail traders, people like brad katsuyama, etc.) will suddenly think you're less liquid because the best price now has fewer shares.

That's, at least, the strongest argument I've heard for why tick sizes aren't reduced more rapidly.

The round trip time between NYC to Tokyo is very short; Any computer system could beat any human, even at that scale. I'm not sure what you think you'd accomplish here; you'd just have HFT with 150ms response times instead of 1us. But you'd still have HFTs.
Yes, obviously.

The point is that there would be no incentive for HFT firms to invest endless money and effort trying to eek out a few more microseconds because that would no longer be a viable trading "strategy" (if you can even call it that).

The problem with HFT firms isn't that they're fast, it's that their only focus is being fast. There are all kinds of interesting algorithmic trading strategies that use data analysis or complex models to make rapid trades: those systems are adding information to the market and making it more efficient. In contrast, pure HFT only adds dubiously-necessary "liquidity" by being quicker on the draw than other firms.

I'm not sure where you get the impression HFT firms only focus on speed. They focus both on being smarter (having better signals) and being faster; it's only very niche HFT firms which rely entirely on speed as their edge.

Also, what's your goal here? It doesn't sound like it'll necessarily impact the profits of the HFT industry as a whole if you do this, merely lower the cost, so on net making the average HFT firm richer.

Lowering costs should allow more firms to enter the market, not just pad the profits of existing participants.

If a HFT firm is richer because they're smarter, I don't have any problem with that. That's how investing works. Developing smarter ways to analyze data, setting prices accurately, having a market that responds to new information, these are all social goods.

I am less convinced that everyone trying to be in the same datacenter as the NYSE (https://www.bloomberg.com/news/articles/2016-04-13/inside-eq...) adds any value for society.

I mean, I'm all for lowering costs in the HFT industry, it'll mean the company I work at makes more money -- I just didn't realize you were arguing for the industry, not against it. I don't think any HFT firm would disagree with lowering costs to themselves, the same way they'd prefer that CPUs were never upgraded (so they didn't have to waste money buying the latest ones).

Co-location is good because it commoditizes the closest location to the exchange. Speed always matters, even in randomized auctions, even if you have artificial latencies added to your order, even if... Without co-location done the way it is now, you get into situations where a single firm buys the actual nearest location, and has a monopoly over being the fastest competitor. Now, anyone can access it.

> Now, anyone can access it.

Where "anyone" = any investment firm with millions of dollars to spend on their IT. :)

Yes, things costs money. It costs on the order of several thousand per month (on the upper end), not millions of dollars -- moreover, this is commensurate with the cost of actually running such a data center (which the exchange sponsoring the colo typically does). I would also argue that, for anyone who actually receives any benefit from being colocated, several thousand dollars a month in operating costs is not prohibitively expensive.
(comment deleted)
Because the stock exchanges are private companies that profit from every trade. It's in their interest to keep making everyone trade as much as possible. I doubt anything is going to happen unless all exchanges agree to institute the same policy, or a law is passed to that effect.

I don't know why you're being downvoted. It's a legitimate concern.

While it's true that stock exchange operators are private companies, they are also SROs that undergo significant scrutiny from various government organizations (FINRA, SEC, etc).

Most (all?) exchanges use "maker/taker" pricing, so while it is possible the trading platform might not make money on _every_ trade depending on the exact pricing rules, they do make money in the aggregate, and in general more trades equal more profits. Many of the opponents to HFT believe that the maker/taker pricing model is a big problem in the markets, and doing away with it would significantly curb HFT profits. Personally, I'm ambivalent about HFT. All stock trading seems to me far removed from the actual value of the underlying companies anyway :).

Disclaimer: I was an engineer at BATS Global Markets for 5 years, but have been out of finance for the last 4, so my info may be out of date.

Idk why I'm being downvoted either. I bet in 2 years there will be a blog post which refines the idea I outlined above and it will be on top of HN and everyone will agree with it. That's how HN works.
Do yourself a favor and lookup the history of the IEX exchange, and read flash boys. They do just this, but HFT traders still trade there and make money, just probably less of it.
Any reading of Flash Boys needs to be accompanied by Flash Boys: Not So Fast. Or just a really critical eye. Flash Boys is beyond bizarre in the silly things it implies (like some major manager's trading PC is hacked or something, but somehow that's fixed with IEX). Or how they lament some guy can't buy huge amounts of stock without moving the price. It's a nonsensical book.
Also fun fact for you. Flash Boys: Not So Fast, was written by Peter Kovac. I used to work with Peter @ Madison Tyler (which merged with and became Virtu ~5-6 years ago). He's literally right on with his assessment.

I should have mentioned that, excellent point.

Electronic arbitrage. Interesting, scalable with compute, and highly risky, if you aren't careful or fast enough.
Note that the key assumption for why losses are unlikely is that they make many independent trades that are each likely to be profitable.

But in real markets, outside events can suddenly make many trades all fail at the same time. This is the same reason AAA tranches of CDOs got those high ratings -- you only lose money if many obligations fail at once, but that is extremely unlikely if you think they have low correlation with each another, which was historically the case.

But in a market crash, all correlations will tend towards 1.

Not exactly, a firm like Virtu makes money due to volume and volatility. If the market goes up, they make money, if it goes down, they make money. They don't make money when they break things, or when the volume (and volatility) is low.

Source: Worked at Madison Tyler / Virtu for over 4 years, but left before their IPO.

Not sure I understand why we can assume that its all trades are entirely "independent" here.

Agree with you that HFT performance is not correlated with market direction. But given the volume of trades it would imply that many of the trades are occurring on a smaller pool of equity instruments. I'd think that given liquidity constraints and competition that there's a sweet spot in terms of number of stocks that a given strategy is actually efficient on.

So not entirely sure why we consider every trade to be iid - rapid shocks (flash crash), equity specific news etc would affect a number of trades at once changing the 51% probability?

Also does anyone know if the profitability numbers include payments to be a market-maker?

Market makers typically have a requirement to be in the market a certain (high) percentage of the time. In return, they get rebates (which can be a significant source of revenue) and protections from the exchange against overfilling. This can help them for certain shock events.

Most likely, though, their profitability % is tuned by edge and risk parameters. They can adjust size, widen out, or tighten up the spread to tune trade frequency and profitability. They are finding a sweet spot between # of trades and profitability.

Those number absolutely include payments. It's rare (in my experience) for an equity market making strategy to be positive in gross terms (prior to the maker-taker rebate). These numbers you hear from Virtu/KCG should be net of exchange trading costs/rebates.
They make money NOW either direction; this doesn't mean they will always make money. If a competitor suddenly starts being faster, they are going to start losing money no matter which direction the market goes.
My friend at a different hit firm says they tend to make MORE profit in high volatility regimes
That's exactly what SEJeff is saying (profit is positively correlated with both trading volume and price volatility).
Correct sir! And I've spent the past 9 years of my career as a Linux monkey in electronic trading / HFT / whatever the hip term for it is today.
Note that I'm not saying that market maker profit is correlated to the performance of the wider stock market, just pointing out that market maker performance can and will be affected by market-related factors that can cause all their market bets to go wrong at the same time, so the actual risk of losing money will always be much greater than what just naively applying the LLN says.

Here's a couple of plausible factors:

1. A flash crash is just a huge coin flip -- even if your market making firm has circuit breakers that stop it from making more trades (and successfully kept its position small), if the exchange ends up ripping up trades en masse you may suddenly have a large position that loses you a lot of money

2. Someone screws up an algo [1] and you burn through more than your entire market cap in half an hour

[1] https://www.bloomberg.com/news/articles/2012-08-02/knight-sh...

Correlations will tend towards 1 when using a large-enough time frame. On a microstructure scale, correlations go haywire in liquidity distress.
Yeah, the analysis in this article is not really answering the question of WHY they are profitable every day; in reality, it is just explaining the law of large numbers.

The whole thing rests on the premise that the company makes a profit on 51% of the trades, a loss on 24%, and break even on 25%. It is easy to show from those assumptions that you will never have a losing day if you make enough trades.

The REAL interesting question is "How do they make money on 51% of the trades they make, and are those results sustainable?"

Is this question as interesting as it might seem to someone unfamiliar with trading? Electronic trading firms do more than make markets, but stipulate that market-making is the sort of baseline approach they take to making money. Are you really asking "how do market makers make money on 51% of their trades"? Because that question is easy to answer; it's the premise of a market maker, whether done by a computer or by a person in a pit.
I probably don't know enough about market making and trading to really say whether this question is interesting to people who are.

My thinking, and why I found it interesting, is that the connotation behind showing that the company 'never has a losing day' because of the law of large numbers is that the company is taking practically no risk. I was trying to ask a question about that point; do these companies really have no risk? Are they basically printing money?

I am sure the answer is no, because no company has zero risk. I want to know where the risk is, and I figured the answer to that question is answered by answering the 'how do they make money on 51% of their trades' question.

That might not be the right question, but I don't think my question was equivalent to asking 'how do MARKET MAKERS make money on every trade', because traditional market makers (i.e. the stock exchanges) are not the same as HFTs. A stock exchange has inertia on its side to protect their profits; a competitor has to fight against the network effect to take them on. An HFT has none of that protection.

HFT firms make money by having sufficiently more accurate predictions of the future price than those they trade against -- here, for prediction, you can take any reasonable metric, such as the change in price in the next 10 seconds. Note that this can be done by simply being faster, as having the same prediction in 400ns that took everyone else 2000ns to discover is still having better predictions.

HFT firms typically have anywhere from little to very, very little trading risk, so long as the above conditions hold. Their actual risks are operational (How Knight blew up) or the above situation no longer holding (How Getco, Teza, etc. died).

Also, one thing I have to mention in your post -- stock exchanges were not the market makers of the past. That role was played by someone called a specialist; you can think of them as doing exactly what an HFT market maker does these days, except far, far dumber and far, far slower. Those people were typically (always?) not employed by the exchange. An exchange is simply a centralized place to display orders, not a counterparty that you trade against.

They're minimizing their market risk, but the trick is doing that while making payroll. You can trade practically risklessly by quoting an unrealistically wide spread. Practically no one will trade with you, but none of your orders will sink the company.
(comment deleted)
to me this is a perfect example of how incredibly indifferent capitalism can be to creating value. no one is bad here, per se, but its useless as far as i can tell, and someone gets very very rich. nothing wrong with capitalism of course, but i feel the need to keep harping of the fact that it doesn't necessarily imply anything about value or desert.

edit: every time i make a comment along these lines, its interesting to see... it seems like HN is split about 50/50 on this. is it that this point is trite? or that you disagree?

i'd say it is a bug of capitalism that HFT is a profitable line of business. it's easily fixed by taxing, but maybe it's already taxed properly, i don't know.
HFT is not necessarily a feature (nor a bug) of capitalism, although something you might recognize as capitalism is a prerequisite for the phenomenon, but of the fact that information sources and financial exchanges form a distributed system, i.e. one that is spread out across space and time.

Of course, in the absence of markets altogether, we have no exchanges, no finance, and no HFT.

But... Why do you see this as a problem to be fixed? Who does HFT harm besides those whose bottom lines they cut into: the big, fat, and slow dealers who wish they could drop a block anywhere and have it fill?

Trading firms create value by providing liquidity.

If they can buy low and sell high, they will be buying when there are relatively fewer other buyers, and sell when there are relatively few other sellers.

This activity creates value for other people who want to trade in those circumstances.

If no value was created, there would be no value to capture.

And the prima facie evidence that they are creating value is that they are capturing value from willing market participants.

This is fundamental economics.

nah. its on such a small time scale, i really dont think it does.
If a company provided X amount of liquidity to both sides of the market, even if the company itself only entered a position for a millisecond, it still provided necessary liquidity. It did so in a very efficient manner and turned a profit on that.

Saying that you dont provide liquidity just because you enter both sides of the trade is like saying that a cab going into and a cab going out of manhattan cancel each other out and provide 0 net value to their passengers.

sorry, i guess there was an ambiguity. i didnt mean to say it didnt "provide liquidity". i meant that what it provided was pretty worthless, in the non-monetary sense.
if you look at the market as a whole, there is a thing called the "efficient market hypothesis". It assumes that markets are perfect and you can not make money trading securities, because all information is already priced in.

This is obviously not true, because HFT makes money, but you could say that HFT contributes to getting markets closer to being efficient.

Having efficient markets is worthwhile for society because an efficient, liquid market provides a more accurate picture of the "actual" value of things.

If you buy this argument, then HFT would be contributing to finding truth, which is inherently valuable in the eyes of many people.

At the end of the day, HFT is one of those things that people just get jelly about because someone makes money on something they dont understand and gets insanely rich "doing nothing".

What people get mad about is completely arbitrary. Everyone hates google because ads, but without google, their world would barely operate. Everyone hates facebook because fake news but nobody has the balls to actually not use facebook. A social network.

Everyone hates uber because "scandals" and poor drivers not being paid but still uses it because comfortable.

I'm not really in the stock market but when I want to sell stock, I'd rather there be some market maker who is obligated to buy it. Maybe he makes more money off of me than I ever will selling stock to him, but that one time when nobody else wants the falling stock and I potentially use a large chunk of my savings, I really need them.

Consumers sit on some amazingly high horses sometimes.

You are not in a position to talk about people on high horses after posting that. There is quite a lot of research from the behavioral economics field (tl;dr: most people are somewhat irrational) that undercuts the EMH.

Also, there's a lot of straw man argumentation in your post. Consumers can and do make sacrifices of utility in line with their values, even if such behavior is a standard deviation or two outside the norm. And while your argument for market making is sound, do you really need to make that trade within milliseconds, or would you be equally happy as long as it closed within a couple of minutes? You seem oblivious to the possibility that your stock might be falling due to a liquidity event caused by a race condition, for example, although in cases such as the 'flash crash' such trades are usually unwound afterwards, presumably at considerable expense.

I'm being called oblivious by people who confuse HFT and market making which each other. That's cute.

There are companies that provide liquidity. They do nothing else. They don't take a position in the market. Its their job to quote both a bid and an ask and to trade with anyone who wants to hit those quotes. They are not allowed to not quote a price, unless trading is suspended. Those are called market-makers. They need to be able to change their quotes quickly, because being taken out on one side of the market creates positional risk for them. The only time they make money is when they can trigger a buy and a sell at the same time. If they cant do that, they have to hedge their position. They have to compete on spread with other market makers, which is possible to do in 2 ways - being fastest to quote or having a narrower spread. Trades at the price are executed in the order of submission.

There are companies that engage in statistical arbitrage. They look for opportunities that are statistical in nature. Think of technical analysis but in a way that works. Think stock A and stock B being more correlated than price reflects. They want to execute fast because those opportunities do not exist for long. This kind of trading smoothens out the curve and smooth curves are nice to have, mathematically speaking.

There are HFT firms who scalp dodginess in price by renting colocated servers. Very tiny but almost guaranteed returns if they can get execution. Nothing wrong with that either. Low margin business model.

Then there are companies, or rather teams inside companies, that send buy and sell orders into the exchange that they never want to execute, on millisecond timescales, to disrupt the market. They are trying, in simplified terms, to create signals for other market participants that do not really exist, and then execute trades that they know will be profitable by exploiting other participants for whom they generated buy signals. This is your evil HFT. The T in HFT is a misnomer, because they dont really trade. They abuse stock exchange mechanics to fuck with other market participants and then try to rob them. This is mostly illegal and not allowed by the exchanges themselves. The problem here is to prove that someone didnt play by the rules. Google "optiver the hammer" for an example of a couple years ago. They have counter-intelligence teams whose entire job is to not trade the market for profit but to ruin other companies profits to drive them out of business. But most of this is illegal, its just difficult to prosecute.

The problem with banning folks is that most trading companies engage in all of these things at the same time. There are market making teams at optiver and there are evil HFT teams at optiver and if you ban optiver, you wipe out a major market maker which leaves the stock exchange exposed.

Another problem with "banning HFT" is that where do you draw the line? Market participants can suddenly not try to get their trades executed anymore? Maybe that's beneficial to the market as a whole, but how do you even execute such a rule?

You're projecting far more into my post than I said. I have not proposed banning HFT, for example.

Also, you really need to work on your manners, which is why I'm not going to bother addressing the rest of your remarks. Perhaps we can have a more constructive conversation some other time.

>> Then there are companies, or rather teams inside companies, that send buy and sell orders into the exchange that they never want to execute, on millisecond timescales, to disrupt the market. They are trying, in simplified terms, to create signals for other market participants that do not really exist, and then execute trades that they know will be profitable by exploiting other participants for whom they generated buy signals.

This is well-known. But for some reason, only kids trading in their parents basement, across the ocean, over a non-fibre connection to the internet, get charged and convicted.

Your points are valid, but not entirely correct IMO. What you are describing as abuse is "quote stuffing". Sending in buy or sell orders to move the market with no intention of fulfilling said orders. When I worked for an exchange operator we routinely reported this type of abuse to the SEC. It' difficult to discern the difference between a fast moving market maker (and many successful HFT firms are indeed market makers) and simple re-positioning of the spread because of shot term volatility. Your "evil" HFT firms may be abusing the system by cancelling an order placed a millisecond prior, but they don't make any money if a trade does not occur. Quote stuffing without a trade on the modified price does not produce a profit. Manipulating the market in this way is not a fool-proof way of making money - there still has to be a minimal position taken by the trading firm in order to realize a profit.
I really think EMH is a red herring here, for what it's worth. This is more a market microstructure issue. You can entirely disbelieve the EMH and still benefit from HFT.
But why is this liquidity necessary? Maybe if you can't find someone to take your trade that's an important price signal, which is being dampened by the liquidity. Forgive me for abusing the metaphor, but just as water is essential for life, consuming too much of it can kill you (by lowering the level of electrolytes and disrupting intracellular signaling). It's rare for people to die from drinking too much water at once, but that's not gonna be a lot of comfort if it happens to you, is it?
liquidity is necessaryy because people need to be able to buy things. not having liquidity is like having empty grocery stores. doesnt sound like a big deal but oh so painful if it ever happens. people in this thread lump market makers and high frequency traders together which is not really correct, but market makers are hired by exchanges to make sure markets stay liquid. market makers guarantee that they will always be in the market for at least "x amount of security", typically 100 lots. that means that at any given point in time, you can at least 100 lots, which covers any kind of smalltime investor who doesn't have access to professional execution traders. this is important because you need to be able to get out of potentially deadly trades.

this is not just for stocks. the potential loss of a stock is the price of the stock. that is finite. some options, even ordinary options, can generate potentially infinite losses and stock exchanges want for everyone to be able to move their stuff.

as far as you are concerned about "signals", there is something called the order book. as a normal trader you dont see this, but the stock exchange lists every position in the market and you can pay to see that. there is literally no difference between no market makers in the market and market makers having their spread quoted on top of the available market, in terms of signals. market makers just make sure that you get a good, if not fair, price for the stuff you want to move.

illiquid markets are extremely painful, especially for those companies that exist in them, because their value can be extremely misrepresented. to make an example, assume you are tesla, but you are traded on an exchange that only publishes spot once a year, in january. in august, you need to raise money at fair value and youve grown 3x since january, but nobody can give you fair price because you are quoted at januaries value.

liquidity is a good thing for every market participant.

you need to drink 5 liters of water over a period of 30 minutes to kill you potentially. thats not going to happen by accident and you will be in ridiculous amounts of pain long before you hit the lethal dose.

I studied economics. Also, I used to install trading systems for hedge fund managers. Your reply is not responsive to the question I posed, and tells me nothing I don't already know.

And yes, people do die from drinking too much water. https://en.wikipedia.org/wiki/Water_intoxication

> Trading firms create value by providing liquidity.

Obviously true when they're added to market with little to no liquidity.

But hard to see as true when you're adding them to market that is already extremely liquid. At least, it's a statement that needs some empirical justification in that case, to show that the value from marginal liquidity being added (which is tiny) offsets the waste of the incredible amount of human effort used to create it.

(comment deleted)
Just substitute "cost of trading" with "liquidity".

It's true that at this point, now that market making is pretty much all automated, there isn't much more cost to squeeze out of this component of the market. But be careful not to imply that liquidity is binary: the more liquidity you have, the less it costs to buy or sell something.

I'm not implying that liquidity is binary. I'm implying that it does not scale linearly, but rather exponentially towards an asymptote. That is, you can add as much as you want, the total amount approaches a limit.

Said a different way, with your "trading costs" substitution: If x amount of human effort is required to reduce trading costs from, say, 5% of a transaction to 0.001%, perhaps (let's assume) that's justified. But is it then wise for a society to encourage expenditure of 10x more effort just to reduce costs to 0.0001%? (This analogy doesn't really hold, because cost of trading could be 0 and there could still be 0 liquidity, just no other parties interested in buying or selling, but it still seems to point at a real issue when restricted to talking about liquidity.)

I think an interesting point you might consider -- let's suppose there is such a thing as "too much" liquidity, and as a corollary, there's a "perfect" amount of liquidity. What do you think happens when more liquidity than the perfect amount is introduced?

I think a lot of arguments which support the idea of HFT being too much liquidity seem to take, as a premise, that the extra liquidity costs consumers -- namely, non-HFT participants in the market trading against the HFTs, the "buyers" of liquidity. Does it? If you have too much liquidity, doesn't that just mean that the buyers don't buy any of it? They saw as much liquidity as they needed, completed their trades, and went home. It seems like the actual cost of too much liquidity is HFT firms which send orders that don't get traded against, so they have no benefit (to the HFT), but still have a cost (the firm has to run, after all). Yes, there's some cost to society here, in the same way that there's a cost any time someone takes on a speculatively profitable business that turns out not to be.

I'm sure there are some strong arguments you could make that HFTs are unnecessary, but the argument of "too much" liquidity always felt shaky to me (in part, because it's one of those arguments that relies on playing fast and loose with nebulous terms).

But he didn't argue we have "too much" liquidity. The argument is that we have "enough" liquidity and are now expending a huge volume of capital on a literally imperceptible increase in liquidity.
My point is that I don't think it's a huge volume of capital. The extra wasted capital here is the cost of running an HFT firm while you test your strategy (aka. see if your liquidity is valuable or not) -- if it's not profitable, it's not a huge loss (HFT firms, in the grand scheme of things, are relatively small) and the firm shuts down; if it's profitable, then clearly they're trading against non-HFTs that value their liquidity, so what's the issue here?
> waste of the incredible amount of human effort to create it

The size of the HFT space is almost universally overestimated. Virtu recently tried to purchase Knight for 1.3 billion, about as much as it would take to buy Sears. Those are 2 of the giants of the industry, combined they would dominate it.

The estimated revenue numbers for all US market makers was 1.1 billion or ~3x snapchats number.

For something that speeds up & cheapens virtually every trade on the planet that doesn't seem like an outside amount of effort. At least not in comparison to showing ads to teenagers.

I don't buy this argument, whose implicit corollary is that there can never be too much liquidity. I'm not an expert and don't have an opposing theory, nor do I want to do a bullshit refutation by just citing something I found in a few seconds on Google, but:

I feel there must be some ideal level of liquidity, which is after all subject to laws of supply and demand like everything else. Cannot an oversupply of liquidity result in fiscal inflation, as opposed to the monetarist kind? I'm reminded of the Asian financial crisis during the 1990s, when there was a ton of money flowinginto newly-developing markets int eh Asian region, but then a spasm of political uncertainty suddenly led to a catastrophic withdrawal.

Echoing one of the other posters here, I don't feel that anyone at this firm or the firm itself is acting in bad faith, but assuming this mathematical demonstration is correct and it's virtually impossible for Virtu to lose money other than by abandoning its winning strategy, does this not invite an arms race and a stampede of eager traders hoping to also get rich by picking up pennies from in front of steamrollers?

I don't want to say you're wrong, but to question the applicable scope of your premise. Virtu itself may not be big enough to cause market instability, but 100 firms doing the same thing might.

I apologize for repeating a point I just made, but I'll make it here too:

You can mentally substitute "cost of trading" for "liquidity".

It's true that you can get costs down to a point where they don't matter. But I think you'll have an easier and more intellectually honest time engaging with the issue if you avoid the jargon and focus on the impact of the jargon. Cost is why we care about liquidity. As markets get less liquid, spreads increase, and the market takes inflicts more and more of a penalty on us for trading at all.

I get this. I just have an intuition that some friction in a dynamic system is actually a good thing, and also wonder if the growth of HFT might not lead to an unsustainable deviation from an underlying power law distribution of trading activity.
I think are pretty large barriers to entry to hft. You need enough capital to develop a sophisticated, heavily error checked trading strategy, set up your own network to ensure low latency, the IT infrastructure/engineers to manage this,.. it's not like a few burnt out traders can open up hft firm in their garage so I think any stampede is unlikely
> prima facie evidence that they are creating value is that they are capturing value from willing market participants.

is this referring to market value or economic value?

Yes. People mistakenly think it's a meritocracy. It's not.

But it is good at making the world rich.

I think you're confused about how HFTs create value. They create value because they're the first ones to make a market in something.

They send out limit orders in every product. An investor then sends a marketable order to the exchange that fills one of the HFT's resting orders. If the HFT wasn't there then the investor's order would just fill a bank's order that would probably be wider and wouldn't respond to changing situations as quickly.

The innovation is that HFTs are able to make better markets with fewer people. This means that less money ends up in the pockets of intermediaries and more money stays in real investors' pockets.

Are HFTs always doing the right thing? No of course not, but HFTs are pretty clearly some of the most ethical firms out there.

I think you're confused about the meaning of "value".

HFTs don't provide any value, but they do extract a lot of money from what used to be called a "market".

> but its useless as far as i can tell,

That's because you, as well as 99% of people accusing financial firms of being useless, don't know what you're talking about.

One of the reasons planned economies usually fail quite spectacularly is that individuals are very bad at predicting global market behavior. It's too complicated. Local stochastic optimization (free market economies) works quite well, even if it's maybe not globally optimal; doing better than free markets via central planning requires an unfeasible level of omniscience about all market actors and interactions.

This refrain of "traders do nothing useful" is a great example of that; most people are too myopic and/or economically illiterate to see otherwise.

Here's what traders do; they communicate price information (extremely important) and charge a small commission for doing so. In the absence of regulation on "Wall Street" (or whatever your regional synechdoche), there would be more competition among people communicating price information and profit margins would probably be a lot smaller. If you can do a better job communicating price information (possibly by predicting future prices), you can make more money.

Another auxiliary job that traders do is provide liquidity, which is very important but (in my opinion) less important than the first.

My apologies for being a bit blunt, but I think people deserve a bit of admonition for attacking that which they clearly don't understand enough to even criticize coherently.

This refrain of "traders do nothing useful"

That was not how I read the comment at all. Maybe slow down and reconsider it in the context of a specific discussion about this specific approach to HFT, rather than assuming the GP is dismissing the whole underlying principle of market making.

wasnt attacking that. was arguing that the marginal/imperceptible improvement in liquidity isnt really doing anything for anyone except the people profiting off it.
The way I think about HFT is:

- one effect of an economy is to find an approximate solution to an important problem: optimising resource allocation with regards to growth

- within this paradigm, markets exist to process all publicly available information and turn it into real allocation of resources

- considering the difference between trades executed once a day vs once a week, the more frequent trades will provide information that is up to date, higher resolution, and will lead to a better solution to the problem

- HFT uses a huge volume of very frequent trades based on very weak correlations, meaning that the information it adds is newer than that of longer term trades, is very high resolution, and may take into account factors ignored by longer-term strategies

- therefore, HFT leads to better overall resource allocation by providing greater visibility into the value of the things being traded

This ties into the comments on the efficient market hypothesis. HFT brings us a little closer to the impossible ideal by making the market a little more efficient.

i dont disagree with the basic framework. but why is going from milliseconds to picoseconds important? im pretty sure its not. thats all
Beyond the "providing liquidity" argument, another side effect of HFT and arbitrage strategies that one might see as valuable is the promotion of global invariants across the financial network, which is rather complicated distributed system. Would it make sense if there were two very different prices for something at two nearly identical venues? If large block of a stock is dropped on one exchange, would it make sense if there were no effects on the price of the same name on adjacent exchanges?

All that aside, I think I have some larger questions about your worldview: What does it mean to "create value?" By what standards is value assessed?

In the absence of any direct harm to another, under what circumstances should someone be deterred from engaging in profitable activity?

Is the assumption that profits accrue to those who provide value to others utterly false, or just false in this (and possibly other, isolated) case? If merely the latter, why in this case?

>> Beyond the "providing liquidity" argument, another side effect of HFT and arbitrage strategies that one might see as valuable is the promotion of global invariants across the financial network, which is rather complicated distributed system. Would it make sense if there were two very different prices for something at two nearly identical venues? If large block of a stock is dropped on one exchange, would it make sense if there were no effects on the price of the same name on adjacent exchanges?

i dont disagree with arbitrage and liquidity, and a globally invariant price for goods. i just think HFT isnt really improving that in a meaningful way. i ask you- at what timescale does it stop mattering? thats really what im saying. not that liquidity and invariant prices are suspect, but that things are happening on such a minute time scale that it doesn't matter. hey, i could be wrong, but no one has argued why going from milliseconds to picoseconds is improving our lives. thats the thing i would like explained.

>> What does it mean to "create value?" By what standards is value assessed?

tough to pin down, but surely its more than how much money it makes you... right? of course it varies from person to person. but i think its very lazy to say that its "whatever someone will pay for it". then value changes when laws change. i think most people probably agree that some things that are able to be sold for alot are not a value to society. i think its fair to say that there are some basically invariant things that are valuable.

>> In the absence of any direct harm to another, under what circumstances should someone be deterred from engaging in profitable activity

probably shouldn't be deterred. my point was that it looks like this is more like a game people are playing, with no real positive externalities. didn't say it was bad prima facie. i think its weird that people are very strongly clinging to the idea that it has some wonderful benefit to society. seems like a pretty weak argument, and kind of pathetic. its ok to to just admit its a game, and its for the players. poker players dont go around claiming they are providing everyone a service. i just get the sense there is an ingrained idea in our culture that there is some nobility in "finance" that i dont think belongs there in some cases.

>> Is the assumption that profits accrue to those who provide value to others utterly false, or just false in this (and possibly other, isolated) case? If merely the latter, why in this case?

not utterly false, but far from universally true. very difficult, broad topic that i am, or course, not in a position to say definitively. we can all point to cases where almost everyone agrees that someone gets overpaid, or underpaid. seems like a lot of people readily accept the dual premises that success == making money, and that money is a proxy for value. i take issue with both- i think its important to think about success and value more fundamentally. it seems like a lot of people have a tautological idea that you are paid what you deserve because its what you got paid...

>> i dont disagree with arbitrage and liquidity, and a globally invariant price for goods. i just think HFT isnt really improving that in a meaningful way. i ask you- at what timescale does it stop mattering? thats really what im saying. not that liquidity and invariant prices are suspect, but that things are happening on such a minute time scale that it doesn't matter. hey, i could be wrong, but no one has argued why going from milliseconds to picoseconds is improving our lives. thats the thing i would like explained.

This is a little more complicated than it would initially appear, I'll try to explain from a market makers perspective why the speed race exists and why being faster (as a liquidity provider) is better for the market, at least the way it's currently structured.

Market making 101 is basically that you want to come up with a fair value for the product you're trading, and then put out orders to buy for a little less than FV, and sell for a little more than FV. If you buy and sell at those prices you're providing liquidity to the market and capturing a small spread for your effort, great. Do that repeatedly and you have a business. But how much should your "a little less" and "a little more" than FV actually be? The smaller the better for the market, and this ideally should be the primary vector on which market makers compete with one another.

Okay, so in our optimal scenario you'd always quote as tight as possible (limited by the granularity of pricing on the exchange) around the fair value. (I'm glossing over a lot here, calculating the fair value is non-trivial and your level of uncertainty about it will also determine the spread you can quote, but ignoring that for the moment). The problem with this is that if the fair value moves then some of your orders become mispriced, as they represent an opportunity to buy below or sell above fair value and if they execute will be a loser. Smart participants will recognize this and race to pick those off before you can reprice them. If this happens too often, you're not making money anymore, crap. Really only two options here, #1 is to widen your quote so that you are less sensitive to such movements and you are capturing a fatter average spread which compensates you for the losing trades. #2 is to get faster than those other guys.

#2 yields a better outcome for the market, but necessitates a speed race as an additional vector of competition. Exchanges recognize this as well and have long played around with various schemes to give liquidity providers a systematic advantage, e.g., via rebates, or otherwise. It's a tough problem.

I generally like HFT. I do not like front running or information advantages that have happened in the past.

Spreads these days are the lowest in history. What many people fail to realize is that before electronic market makers (I almost want to eliminate the name HFT), people sat in between these trades. It was slow, inefficient and they took a larger spread on the trade.

this seems contradictory to me. HFT is basically predicated on front-running information advantages. It wouldn't be profitable otherwise.

maybe you can explain in more detail?

Apologies. What I am referring is some of the issues of the past. It has been a number of years but IIRC it was a NASDAQ data feed that cost a good chunk extra but gave you a time advantage. Thats what I am referring to, or the fact that at that same time certain exchanges were catering order types for their big clients and leaving those unpublished (also illegal).
HFT is in basically no sense predicated on front-running. Front-running is an agency problem: it occurs, for instance, when you're trading on behalf of someone else, and before you execute their orders, you submit your own orders that benefit you at the expense of your client.

The whole premise of the market is that people have informational advantages. They don't work without it. The point is that they aggregate the information of all the participants. You can call that, or anything else, "front-running", but that's a meaningless definition.

let me be more specific since I think you're misinterpreting me because I wasn't specific enough.

hedge-funds that execute a strategy based entirely on leveraging HFT as a means to take advantage of information arbitrage (such as they kind that a fund taking trade orders from its clients would have) are predicated on having that information advantage "front-running".

HFT in different contexts is just machine execution of trade orders and isn't what I was asking about.

That's not how a hedge fund works. Hedge funds trade on behalf of other people, but their clients have equity stakes in the fund itself; they're not asking the hedge fund to trade their own positions in things. Hedge fund customers don't inject tradable information.

The canonical example of a front-runner is a broker trading on behalf of (say) a pension fund. The pension fund wants to offload (say) all its shares in CSCO, and pays a broker to do that. That trade is complicated and will move the market. The broker front-runs by first trading CSCO for its own accounts, at the expense of its clients.

An "HFT-enabled" broker could front-run its own clients, but (a) they don't need HFT to do that; they have (relatively speaking) all the time in the world to act on the confidential information that their client is offloading a large block of CSCO, and (b) they're the ones complaining about HFT and setting up new exchanges to combat it.

> It wouldn't be profitable otherwise.

This is absolutely not the case. The advantage it is predicated on is getting there first.

Summary:

-- Their trades are profitable f = 51% of the time, and they do N = 3 million trades per day.

-- Their net profitability per day is thus (well approximated by) a normal random variable with a mean of f and a standard deviation of sqrt(f(1-f)/N), or 3e-4

-- The probability of this value being less than 50% is well approximated by norm.cdf(0.5, 0.51, sqrt(f(1-f)/3e6)) which gives 2.4e-263

In other words they only expect one loss per 10^263 days, which is much larger than the age of the universe. They are actually doing much worse than expected because they lost on one day.

So something is obviously wrong with their model.
Most likely fat tails. A normal distribution is probably not a valid assumption