Because investment bankers are just salespeople? Nobody is suggesting AI will replace salespeople in the near future. I could see there being fewer grunt analysts in the future, though.
> Nobody is suggesting AI will replace salespeople in the near future.
They aren't? Because I see this happening all the time. Automated checkout lines, E-Trade and online brokerages, etc. Low-value transactions are handled more and more by electronic "salespeople". All it takes to reach high-value transactions as well is for the clientele to decide they trust machines more than humans, which is not all that unlikely given the many sources of human error involved.
It's possible that humans will be involved in high-value transactions like investment for a long time, since the marginal cost of human labor is low as the numbers get bigger, but it's not impossible to replace; we already have the technology.
I think that the term 'salespeople' here refer to those who proactively push their products to the market and by doing so widen the market for the company they represent.
The examples you give are indeed places where buying or selling happens, but where does the incentive to do these transactions come from? It's still from humans. Sure, we are bombarded left and right with ads that were carefully targeted for us by complex algorithms, but I don't think I've ever bought anything from Amazon without reading a review about the product from another user, or bought stock without going through several opinions written by more-or-less respected analysts.
You'll note that for a lot of online financial services, there wasn't anyone at all to serve low profile customers before robot-advisor/self-serving services came in.
There is a line (few hundreds k) where it's enough to play but not enough to pay for the costs of a human to help you (sales people/financial advisor/trader type).
>Nobody is suggesting AI will replace salespeople in the near future.
Huh? For one, investment bankers are "just salespeople" in the same way doctors are just nurses.
Second, everybody and their mother is suggesting AI will replace retail level salespeople in the future. In fact lots of places, from British supermarkerts (where you can bill and pay for your purchases yourself), to Amazon's Go experiment, and numerous other uses of automation in retail.
I think grandparent is referring to salespeople as in "human specialized in marketing and selling special/expensive/complicated products one-on-one to entities", not wholesale retailers.
> Salespeople are people who convince you to buy something. Higher-end stores have them
Many more lower-end stores did too, pre- or earlier in Amazon's and online shopping's proliferation. It's already happened to a not insignificant extent.
Well, in that case, travel agents have been replaced by "buy your own tickets online", real estate and rental agents have seen similar automation through web apps (+ reviews, photos, walkthroughs, etc for the "convincing part"), and lots of other salespeople jobs.
Bookstores is probably a famous example, where the people working there were far more than cashiers, but they still got decimated by Amazon and co.
Look up the comp ratios of some of the major investment banks. It can easily be 50%. As with all professional services firms, the biggest costs are people and real estate.
But it is large compared to what the shareholders earn, and they are the ones ultimately pulling the strings. The days of the white-shoe partnership are long gone...
The complete lack of any sign of remorse on their faces about having burned a trillion dollars made the hair on the back of my neck stand up. I've never seen such a blatant display of psychopathy.
Regarding your airplane example, it's actually the opposite: plane crashes are so infrequent that you have to believe "I am special" (with a negative twist) if you think you are likely to be in a crash.
Scheduled flights inside or between advanced countries are incredibly safe. Unscheduled flights and any flights in non advanced¹ countries aren't that safe.
The overall statistics are not useful for evaluating your next trip. You must use the ones from your cohort.
1 - Mostly, the countries with broken governments. If human rights aren't respected, that's a great predictor that flights aren't safe.
If some belief applies to everybody, but on a different degree (per individual, or per job sector, etc), then the mere fact that it applies on everybody doesn't say anything to explain this variability.
And if what you say is that it applies to everybody with no variability, then that is obviously wrong. People in specific job sectors are way more worried about automation than people in other sectors, even if their jobs are not yet starting to get automated. E.g. office workers vs surgeons...
I think the difference here is "task" !== "job". You're automating tasks. Your job is to keep the system running as effectively as possible (I don't know what you actually do). Meanwhile, I do get that you're not actively trying to lose the responsibility of these tasks, i.e. "I wouldn't be giving the scripts to my employer", but that is because you're trying to hold onto the relaxed transition phase between tasks X, Y, Z to tasks 1, 2, 3. If you really had job security issues, you would not have written those scripts.
Are you sure you haven't perhaps fallen prey to too much job security? Such a large sense of job security that you don't care about automating X part of it away. You, like me, like other "rockstar" software developers, (perhaps[0]) believe it doesn't matter if you automate X, Y, Z, because you're so good, that when you're out of responsibilities you will be offered the followup/orthogonal tasks 1, 2, 3.
Anyways, I'm just trying to highlight that you too, perhaps subconsciously, think "I am special, it wont happen to me", because at least I who also don't care about automation feel this way.
[0] I know for me, this is the truth. This overconfidence is why I don't care about automating my tasks away. Honestly, even giving the scripts to my employer makes little difference to me.
To some extent what you say is true, but only partly.
I'm not part of the cult of the "rockstar programmer". I do write a lot of good code, and I tend to do it much faster than other programmers that I work with. But programming is not actually my job description.
Without going into details - I do something else for living, that uses computers quite heavily, and automating my job is more of a hobby. I have a huge amount of flexibility in what I do, so there is an element of substituting 1,2,3 for X,Y,Z because I find them more interesting. My boss wouldn't care one way or another - he is more interested in whether or not the tasks get finished, and maintaining a high quality level in the final product.
Not at all. I do keep part of the code that I write secret. This is not a form of sabotage. My employer likes the fact that the things that I am responsible for run smoothly.
Due diligence on the financials of a company (what investment bankers are supposed to do) is actually really hard to get right with the algorithms we have today. Much of the data and insight compiled by an I-banker today does not exist in an easily parse-able form for automated algorithms, and a substantial amount of the computation relies on common sense knowledge.
> ...actually really hard to get right with the algorithms we have today
assuming, very few companies which would have publically available published financial going over a large (> 50 years) timespan, would it be safer to conjecture that it is actually the _paucity_ of data which impoverishes the algorithms ?
> Due diligence on the financials of a company (what investment bankers are supposed to do) is actually really hard to get right
Diligence is automatable, in the long run. There is a human element to it, but it is small in most contexts.
Bankers solve trust problems. An AI would need to be trustworthy in a personal way to replace bankers--this only happens with AGI. For a long time, as long as humans control capital, there will be other humans interfacing those humans with each other.
> Diligence is automatable, in the long run. There is a human element to it, but it is small in most contexts.
I'm not sure I agree with the first sentence, but I agree that there is a human element. If true, then at least some investment banking jobs must remain immune.
More to the point, most investment banking valuations are just guesses and understood to be as such. Estimating discount rates and growth rates in particular are very much gut-driven and aren't expected to be precise, no matter how complex a model the analyst comes up with.
There's only so much you can ask a kid 10 months out of Harvard econ to do, no matter how many pounds of cocaine and borrowed Excel sheets they have.
At the more senior levels, banking is relationship based, and making partner is a function of how much business you can bring in. This means maintaining good relationships with the clients you work with and reaching out to new ones. Show me an algorithm that can send cards on kids' birthdays, drink champagne, and play golf.
Not to mention the difficulty of an algorithm that can come up with a convincing explanation for why last quarter's predictions were way off but these ones should be fine.
That explains the current state, but has little prediction value. The availability of parsable data can change at any moment either because of regulation or through market forces (either external companies providing parsable data as a service, or enough companies realising that providing plentiful parsable data about themselves attracts capital).
Once enough data is available, common sense and gut feeling can be programmed as a combination of fixed rules and machine learning.
HN title is significantly different than the question the submission actually asks which is "why do TRADERS in investment banks . . . " and should be updated
It looks like the question title was changed, in order to aggregate similar questions/answers. Most of the answers answer the question in the current title. I agree that trading and IB is a big distinction, and traders are 1.5 feet out the door already.
It does make a difference. Investment banks employ traders sure, but "investment banking" as most people understand it is, M&A advisory, IPO underwriting, syndication and so on. These are relationship businesses first and foremost.
The job is actually extremely complicated. I wear many hats daily including working as a software developer, IT project manager/architect, salesman to clients, point of contact to other institutions trading desks, legal drafter, tax/accounting policy design, deal negotiator, risk taker/manager, systematic/fundamental strategy research etc. My job title is "Trader". There aren't any dumb big swinging dick 80/90's style traders left shouting buy and sell (if they are it's purely relationships and at small shops). Nearly everyone on my desk has a CS degree or similar quant background.
AI/deep learning is currently nowhere near being able to master one of these fields let alone cohesively manage all together in a direction to actually make money (just pay a "trader" a 200k-1.5mn USD a year its much easier). Most grunt work is automated and most outfits do in fact make use of statistical methods/classifiers/predictors and differentiable networks where appropriate. At the crux of it someone needs to manage, design the systems and fill in for them with a bit of intuition and common sense where there are gaps. The world is a chaotic place and its not getting simpler. Machines are not robust against chaos.
And this is why traders won't be replaced by AI - Because most are smart individuals who are continuously increasing their worth by learning new skills and ensuring they work WITH new technology rather than opposing it. They regularly elevate their skillset e.g. from manual trading to monitoring, performing trend analysis on a more automated trading system.
I worked in the finance industry as business facing software dev for 10 years and now work in a different but similar industry - my experiences can't be more different. My current industry's business is packed full of "traders" who don't trust the systems, won't work with them and outright refuse to admit the systems can outperform human's PnL numbers (especially not their own) despite being faced with raw numbers saying just that. It's a continuous battle between the technology teams being told to deliver automated systems to reduce costs and increase profitability vs plenty of "traders" who seem to see the systems as some sort of threat. As you say, most of the 'dump big swinging dick' style traders are long gone in finance - they're definitely still around in mine. My guess is that because our industry is significantly smaller meaning less employee opportunities, and the fact that almost all of these "traders" are relatively poorly educated (compared to those in the finance sector) they find it tough to elevate their own skillset to do things such as statistical quantitative analysis etc.
If it wasn't for the very high difficulty of entry, my industry is ripe for statistically minded, methodical, quant people to come in and make a killing.
> "Would you trust purchasing a house from a seller, without meeting/talking to them, or a single person before and throughout the purchase?"
You mean I can get an unbiased look at a house in peace, compare the numbers, look at the plans, measure the humidity and do my due diligence without a sales person breathing down my neck?
I found my apartment online. Set up the meeting. Negotiated the documents and corrected them where necessary. The broker I paid 10% of my first year's rent? Apparently he bought a ticket to Paris the morning of our closing.
For my first appartment I used a broker as well. At the end of the process I discovered their added value was less than 0. Disregarding their fee, I paid too much for the appartement and they encouraged the high price, because they just wanted to close and receive their fee. When I bought my next house I did not use a broker. That's just the way things go, you live and learn.
People are more and more selling without a broker as well, or at least they start to use cheaper brokers where you only buy the service you need.
Most apartments in New York City have a broker involved somewhere. In this case, the building owner had a sell-side broker. A condition for renting the building was paying the building owner's fee, directly to the broker, and while simultaneously absolving the broker of any responsibility to me, the paying party, over the building owner, the technical client.
> If you negotiated the documents yourselves, why were any broker involved ?
It's a scamola with a similar racket being Ticketmaster's "convenience fees". Most apartments in NYC will require going through their broker who collects a fee (usually a percentage of the first years rent). A chunk of that fee ends up being a kickback to the owner of the apartment. Ticketmaster does the same thing with venues (tack on $15 fee, give the venue $7.50).
since a company is made up of people, and the performance of a company is closely tied to the ability of the management, and when you buy over the company you might be concerned about your affinity as an owner to the group of people that makes up management, then I think an investment banker fulfils the same role as an interviewer or a recruiter for a job candidate.
would you hire a person for your team just by looking at degree transcripts, LOC written, and the resume? maybe you would, but I'm not too inclined to do that.
I did exactly this when buying my home through Redfin. I only met the selling agent in person because they unlocked the door so I could see inside. The rest was though email mostly.
This is the classical hn bias. Normal people would like to talk to humans. Hn users would rather get info, plans, etc from houses and eventually buy them and sign the contract using a REST API with a node.js client..
Exactly lots of people want the experience of the salesman in picking a house, information on the neighbours and what its like to leave there & why the previous person is getting rid of it.
The HN bias is a bit like the salesman is the inefficient part disincentivized to help you and trying to rip you off.
The salesman is incentivized to screw you. They work on commission so the faster they can move houses, the better. Real estate agents are screwing both the buyer (feed you bullshit to get you to buy) and the seller (convince to back off of higher prices because the 0-15% difference in price isn't worth the weeks more effort on their commission).
Like a restaurant though, if they were bad at their job they wouldn't attract new clients. Under a HN analogy if they screwed clients they'd have a 0 rating and lose future clients.
The assumption you have is they are simply a market matcher but they're offering more than that.
Their incentive is to get the highest possible sale price in the shortest amount of time and a good reputation. That high price has to also be clearable meaning it cannot be just up in the air and seemingly arbitrary (it can cost either time or reputation).
Without them there is much more information asymmetry. Using an online version you basically have access to the same information you would as with them but without the human element to judge through nor their experience on offer.
This is not true. Some professions are based on screwing the current client and moving on to the next one. How many times do you get to buy a house? This is why most people use an agent to sell their house. They can lie without seeming they're lying (sorry was unaware of xyz...that's what the owner told me)
The incentive for the agent is actually highest cash in down payment because they don't want to deal with the bank to get their money plus they get to keep the down payment if the buyer pulls out.
Perhaps things are different internationally, but if word got out that agents from the estate agencies I work with were regularly screwing buyers, they'd sink quickly.
> plus they get to keep the down payment if the buyer pulls out.
Judging by the way estate agents (realtors in the US) are viewed in the UK I don't think anybody likes dealing with them. I suspect most people would rather learn JavaScript, Node and a REST API rather than have to deal with them.
That's a rational buyer...what do you think the bank does? They run the numbers to see if the person qualifies, has great credit, etc...do you think they care how your day is going? Lol...
> > > "Would you trust purchasing a house from a seller, without meeting/talking to them, or a single person before and throughout the purchase?"
> You mean I can get an unbiased look at a house in peace, compare the numbers, look at the plans, measure the humidity and do my due diligence without a sales person breathing down my neck?
No, I don't think anyone means that. They mean purchase the house.
You can't go into a company and get them to show you their sales pipeline and their staff's performance evaluations before you (potentially) invest. If you want to do these things, you need to make a series of smaller deals first.
Human beings can be fluid, can enter into non-disclosure agreements and make these kinds of bespoke deals, while a web-based transaction form with a buy-it-now button cannot.
For a decade now, I can't figure out why we still bother with Realtors. It's basically eight courses, and an easy test. Oh, and the experience. Don't get me started.
It's the biggest purchase most of us will make, so I understand the need for hand holding, but times have changed. The typical buyer is educated, and in all likelyhood knows about as much as the typical ex-cheerleader Realtor.
I thought the profession would be gone by now. It is ripe for elimination.
The only reason I would use a realtor is out of convention. And believe me, I would want to see a fresh copy of their omissions & errors insurance premium. (Need someone to sue, but that could easily be the Seller if they lied.)
Sorry if I sound bitter. I just don't get giving away 3% on huge amounts of money for essentially hand holding.
(In CA, the Realtors Lobbyists tried to cut down on new Brokers by making the Brokers licence harder to obtain. Gov. Schwartzeneger saw right through the ploy, and vetoed the bill. He knew they just wanted to decrease the amount of Brokers, so the Cheerleaders would have less competition. Gov. Brown let the bill ride right through. All of you should remember that when paying thousands of dollars (full commission) for a few hours of a Realtors time. When that bill passed, I became an Independant.)
This was my first thought. Of course their jobs can be automated. Automation increases the productivity of the average employee to the point where fewer employees are needed. Investment bankers are extremely vulnerable to this given how much fat there tends to be in their organizations.
Trading illiquid products is all about knowing your market, and having a good idea of who you will be able to sell something when you accept to buy it (you are not in the business of taking a position you cannot get out of). Having a good understanding of the flows, who is buying, who is selling, who still has room for this exposure, etc.
This is achieved by discussing with sales people, who themselves discuss with investors, as well as traders at other banks (through brokers).
Now the definition of illiquid varies. Many products that used to be illiquid are now pretty liquid (interest rate derivatives) while many other are liquid only for small trade size (certain bonds). FX is an interesting example. It is very liquid but it is also a market where many clients need to make jumbo transactions on the spot, and these would be market moving if not carefully managed. That's something that could be probably automated.
I find the article very poorly written. "Investment bankers" is extremely vague and the breadth of very different product in different markets traded by investment banks makes this sort of generalization a bit absurd.
The other reason why I think the article doesn't make sense is that investment banks cater for certain giant markets (equity, FX, treasuries, etc) but also for a huge variety of niche markets, in which only a handful of banks and traders are active. If you add the salaries of the couple of traders in each of the active banks on one particular market, the savings you would do automating the market wouldn't justify the years of IT development to train and fine tune an algo, which would still need to be maintained as the market evolves (and then you end up overpaying an AI expert instead of overpaying a couple of traders...).
I think AI will shine at solving wide problems that affect a large number of people. Self driving cars. Butler robots. Building a house. Manufacturing something common. These markets have the scale to justify large investments.
I very much doubt that AI will replace every single complex task done by a man today, for the very same reason that software hasn't replaced every single manual task done by a man today: the cost of developing and maintaining software can easily exceed the salary of the few guys you are trying to replace. To overcome that with software, we need to dramatically reduce the cost of developing software, enabling ordinary employees to develop their own software. But even today we are very far from that. What's the % of a generation who can actually code out of college today? How easy and useful are the main programming languages? I'd argue we are now going into the opposite direction. Microsoft is poised to take VBA out of office. All major OS are evolving toward the iOS-style locked down platform where you can only run Apple/Microsoft approved software. Corporate IT is ever more locking down platforms with software whitelisting, etc. I wonder if the golden age of productivity improvement through software has not peaked.
>To overcome that with software, we need to dramatically reduce the cost of developing software, enabling ordinary employees to develop their own software.
I think this is the key to running an effective organisation.
It requires that most people know how to program, or at least know enough to understand what is possible to program.
I also don't see this coming, partly because the education systems are not really up to par, but also because it's really hard to develop complex systems.
I'd argue that the languages are not really the issue here even if the trends seems to go towards even crappier languages (like js, php) to really mess up the heads of beginners.
But even with a hypothetical "perfect language" the main problems is the huge amount of time and effort it takes to learn any programming language at all, plus the even more enormous amount of time it takes to write something useful even for a really experienced developer.
But sure. It's perhaps time for a new cycle of tearing down the "mainframes" (in someone else's basement this time) and reinvent personal computing for the 2nd or 3rd time...
I think there is a technical and experiential aspect to these 3 elements. I think the technical aspect can be met, but perhaps it's equally true that the experience of trust etc with another human can not be replaced (in a literal sense, given that it specifically requires a human).
Me neither. How much of the genuine human nuance is present in today's financial services or any commission driven industry - next to nothing. Every agent/broker is motivated by the highest commission he/she can make, nothing more.
Even if this was true (which it isn't: in most areas of finance blindly following this strategy is a rapid route to getting sued for breach of fiduciary duty) there's a lot of human nuance involved in actually selling the service to a client who won't take "well the model isn't tractable but here's last year's results and an article on ML" as an answer
So what is a more reasonable answer that the client will accept ? Ultimately, it's a pattern of words and data. Just playing the Devil's advocate here. Any ML system that can be trained on such patterns should be able to learn the nuance, per client. No ?
I'm certainly not convinced any ML system can pass a high-stakes, niche-interests version of the Turing test (and since winning people's trust back after losing their money is an emotionally charged thing, probably something akin to Philip K Dick's Voigt Kampf empathy test too). Even if they could, you haven't got a training set because (i) client conversations aren't usually on the record (ii) none of the previous human conversations that could be used if they had been recorded were remotely related to the whys and wherefores of the investment decisions the algorithm actually took (iii) the investment model generated by the ML process probably isn't tractable enough for even its own human designer to convert into words what it's actual strategy was and will be in the next period.
You don't need "any ML system" to be able to not only process the data, but also explain the strategy behind its data processing at various levels of granularity, answer abstract questions and do a convincing impression of listening and responding to client feedback, you need advanced general intelligence.
As for what the more reasonable answer a client would accept is, I don't think I really have the requisite years of investment banking experience to know that...
I can reasonably say that you make this comment because you know nothing about the business logic of trading desks.
A trader is not necessarily a sales or a broker, though. Nor is he a quant, or a dev. People rarely imagine how many different jobs are involved in the trading job, and how rich the business logic is. At my shop, traders are the piece that connects all of the jobs in the value chain.
I believe, currently, the business logic can be improved locally by learning systems (and it is), but there is no public example of an industrial learning application encompassing a scope comparable to what the usual trading desk handles. Sure, there are many inefficiencies ; traders work on heuristics, afterall. But I don't believe we have the necessary horizon to aptly predict the end of traders, because I don't see how we could make AIs with a better efficiency.
I'd say I know something about the business logic of trading desks. Some of my friends worked at GS. I'm not implying traders are brokers. I do agree that the trading business logic is very rich and that AI will be nowhere close to replacing a trader in the next 15-20 years at least.
I do foresee a future in which a trader can accomplish a lot more than he/she can today using AI. So in the future as the per trader efficiency increases, the number of traders required will most probably decline, unless there is a dramatic increase in trading volume that cannot be matched by the then state of the art AI.
This is essentially what's happening today with X.ai, Facebook messenger and the like. Sure the logic involved in booking air tickets for a group of 5 over 10 conversations isn't as complicated as the rich business logic of a trade, but 5 years back Facebook messenger would've seemed almost impossible, just like the trading business logic seems impossible to do with AI today.
I have no doubt new technology will be leveraged to improve productivity, but that's been quite common in the last decades.
On the other hand, the technology advances needed to transform the tools into standalone actors are not merely a matter of scaling current technology. Especially, the creation of training datasets is a problem for which we currently have no solution, that's why we fall back on human trainers (mturk, etc). That's why a solution to a problem with no clear, bounded model and no easy dataset seems out of reach.
Honestly, maybe they just don't care? Rather, what should they do about it, what would we expect people with such huge earning potential right now to do other than push forward with the plan that works under the status quo.
Answering my own question: I'd expect bankers to save more (of their own money) in anticipation of the good times not lasting as long. More conservative types will weather the storm and spendthrifts will get wiped out.
In other words: business as usual, up until the very moment it isn't.
Is there a fallacy name for this? i.e., asking a question that suggests something ("investment bankers have this feeling") as a premise, that may be not true.
because traders are used to seeing such predictions fail.
Reuters was trading FX electronically since the early 1990s. At the tier one IB I worked for the IT budget was 500m USD a year (across products), and that was in 1997! Huge resources were thrown at automation. However, to this day, large trades in FX (> 10m USD notional) are still almost exclusively performed by humans over a telephone or over the bloomberg messaging system.
That's because, no matter how much you automate stuff, there is still the 1% "edge case" scenario where something goes wrong, and when that happens, you most definitely want a human that you can "look in the eye", when you have that sort of execution risk. Remember that markets move really fast and there is a lot of risk in big trades that "go wrong" because unwinding said trade will almost certainly cost one of the sides a fortune.
Also, high finance is not just about what you know. It's inevitably about who you know, about "illogical" factors such as salesperson charisma, entertainment, and most importantly, a credible personality type that understands the edge case risks. These things are very hard to replicate with a machine. You'll say they should be, that these things are unfair, but they remain a fact after many attempts at removing them have failed.
As for AI, let's for now call it what it is: machine learning. Learning from the past. That's fine for recognising stop signs at different distances, angles and degrees of noise. But in finance, the past is often misleading. Sure there's trend, but there are also very big instabilities in the historical correlation matrix. Paradigms shift without you even realising it. The constant is change. AI is not good enough at that, yet.
BTW, that's not to say machines are not making inroads. It's becoming almost impossible to get a decent trading job now with knowing at least R and Python to a comfortable degree, and good quant programmers cost a fortune. There's massive demand.
> Also, high finance is not just about what you know. It's inevitably about who you know, about "illogical" factors such as salesperson charisma, entertainment, and most importantly, a credible personality type that understands the edge case risks
We were talking about the finance industry, which some say that it should be this cold, rational thing where no "personal", charisma-based decisions are ever made, it's all based on offer and demand. Otherwise I do agree with you, our particularities as a species have caused lots of "scandals" in the past (like world-conquerors getting drunk and killing their best friends because of it), but that's to be expected, we're not robots.
Machine learning is "learning from data." It is not the assumption that there are no dynamics, and that the future will simply be a repetition of the past. To the extent that the future is predictable, learning from data is the best that can be done.
The reality is that speech recognition, language translation, face recognition, object classification and detection, semantic segmentation, speech and image synthesis have improved by extraordinary leaps and bounds in recent years. If we used the logic that past failures inform a confident belief that future success in a challenge will inevitably fail, then we should've bet heavily against Alpha Go defeating one of the most accomplished human Go champions in the world. Self driving cars seem like a sci-fi fantasy until they become a mundane reality.
There's an irrational arrogance to human beings in general, and Wall Street types in particular, regarding the specialness / non-reproducibility of their intelligence. It's not unlike the belief that people had that organic molecules were somehow special, "vital," and not synthesizable from base elements.
Certainly, there's a long way to go to replicate the capabilities of a human brain, but I don't think we should exaggerate or fetishisize the human power to estimate and mitigate risk. We've seen many spectacular failures of that in recent years.
Maybe not, because it has not been trained for those dimensions. Then again neither have humans (except maybe on 13x13). That could be interesting, but I'm not sure what we could learn from this.
The difference is that chess, go, etc are all essentially rules based. Finance has very few rules that do not break over time. Just look at QE. Arguably the financial market represents the collective intelligence of a huge amount of very clever people. Machines are only just starting to challenge a single human at a rules-based activity. We're very far from beating a brutally darwinian, impressively adaptive, human hive-mind whose main skill is figuring out when rules are about to get broken.
Interest rates cannot go negative was a commonly accepted rule at the beginning of my career (and interest rate option traders where using models that did not allow for negative interest rates).
What are the rules for image / video captioning? For natural sounding speech synthesis? For realistic image generation? For semantic segmentation? For determining perceptual visual similarity between images?
Frankly, rules based AI is basically a bust compared to learning from data approaches.
The hive was pretty amazing at destabilizing the entire global economy within a brief span following fundamental financial de-regulation.
People flatter themselves in ways which satisfies their egos and self-interest. Of course they want to believe in, and have everyone else credit, their magical superpowers. No government (backstop) wires necessary.
Automatic image and video captioning is in a very sorry state. Machines can't do it.
Machine learning is rule based. The difference is just that it creates many more rules for more and more granular cases. But it will never be intelligent, it will always be a dumb digital bureaucrat.
Given results from ex. neural networks, ML "rules" appear to be granular and flexible enough that I'm not sure they usefully count as being rules anymore.
There are rules in that you're able to create an exhaustive labeled training dataset. The rules may not be generic, but they exist through a specification by example.
Beating a human when there is no discoverability to the problem, and when building an exhaustive dataset is a problem in itself is not on the horizon yet.
Hmm. I suggest that recent SOTA results contradict your contention that the field is in "a very sorry state" and you up the standard to "human like accuracy". Feels like the goalposts are being moved.
Anyway, this paper demonstrates some impressive results:
But the traders are trying to predict the future already! If it's impossible, then the machines are a shoe in, and if it is possible I'd still give the machines great odds.
The nuances of the game are different from strictly trying to predict the future.
Casinos don't try to predict the future of every spin of the wheel or roll of the dice. Every individual game is random. But over a large enough sample size (number of games played), the odds are not random.
The same principle is at play in trading the markets, except the "odds" are determined by the traders' skill - their "edge." Which is roughly some combination of access to the right people, access to better information (where better might but not necessarily mean quicker), and experience (gut feelings, hunches, what-have-you).
So the real question is, can machine learning algorithms develop an "edge", and if so, can they stay solvent long enough to do so.
> In casinos the games are designed so that in the long run the house always wins.
That's exactly what I meant by odds; they always win because the odds of the games are in their favor.
> The stock market isn't a game
It's a game in the same sense that life itself is a game.
> it isn't designed
The mechanisms that fit all the pieces together are very much designed by humans. True the behavior of the market isn't designed; it's an emergent property of the complex system we call "the markets."
> it continuously evolves
Yes, but change is also the constant. What underlies that change is supply and demand.
There are varying levels of sophistication from which data can be learned from. Pigeons can learn non-trivial word concepts and statistics but they do so at a rate that is much slower than a human infant.
Thus far, machines have not done well in scenarios of low stationarity. Those are scenarios where the past is not so good a predictor of the future or where the data manifold is rapidly changing. This occurs for example, when the act of predicting changes what is being predicted. The related notion of Antipredictable Sequences is the most interesting consequence of the oft misconstrued notion of the No free lunch theorem.
Humans are far from the ideal machine for this scenario but they vastly outmatch current algorithms. This is because humans are much better in the kind of generalization that is based off theory building. While not perfect, it does well enough at generalizing from observations (think Newton deriving the laws of gravitation from observations and Brahe's data) to scenarios that are not so close to the data.
This means that for now and in complex environments, simpler models that are adaptable to change online are better. Additionally, their inspectability means they are more easily modified to better match any very changed dynamics.
So your examples of: `speech recognition, face recognition, object classification and detection, speech`
Are of the type where a learner, such as a deep network, whose generalization ability mostly hails from interpolating to fill in missing structure based on data (implying poor out of sample performance) does well when you can give lots of examples which provide good coverage.
Language translation is sort of inbetween (the space is not as smooth but is still fairly stationary at the scale of years) and so we also find that the advantages of deep learning have not been as pronounced and dramatic as has been the case for image and speech.
The digital machine advantage is two. Their memory capacity is outmatched. Second, and related to the first, energy needs are not so pressing a concern.
No one is suggesting that AI is a solved problem. I simply cited a handful of areas such as computer vision where progress in ML has been remarkably rapid after many years where the problems seemed daunting and skeptics had written off the prospects of the field.
Citing Newton as a typical example of the superiority of human inferential abilities perhaps represents a case of cherry picking. How often do most human beings come to exhibiting the level of insight and reasoning power required to construct the calculus and discover the laws of classical mechanics? Inventing supernatural explanations for natural phenomena, burning sacrifices / witches / books (on Evolution or other heresies) seem more par for the course than Newtonian level revelations.
Having said that, learning the laws of classical physics from observational data strikes me as a pretty natural task for the right kinds of machine architectures, since they represent essentially geometric symmetries which hold over a vast range of scales, space and time.
My point was that you've underestimated the problem difficulty, when the signal is not of a smooth manifold and where the dynamics are far from stationary, it is incorrect to infer that techniques which have worked well in stable and smooth scenarios will continue to work well. In fact they have not and it looks, not for a while yet either.
> Citing Newton as a typical example of the superiority of human inferential abilities perhaps represents a case of cherry picking
Are hyperparameter searches across a sea of AWS instances cherry picking?
Anyways, I don't think I cherry picked. There is not a single machine that can do the same yet. And more importantly, a learning algorithm that notices model failure and works out the surgery required to correct or even completely replace it.
Newton is an example of the human brain at its best. But he was far from the only: Maxwell, Green, Einstein, Noether, Archimedes and so on the list goes. But we don't need to go that far since Crows are already capable of generalization that out matches what machines can do for the near future.
I would add a third big advantage to the digital machine: the ability to seamlessly connect to "classical" algorithms. For example, AlphaGo also used a classic MonteCarlo algorithm.
A hypothetical "robot-cat-AI" could for example have access to a really precise physics engine, and also could run at a much higher "frame-rate", so it could see the world in slow-motion and execute really exquisite moves as a result.
I bet Google "could" write some algorithms that can predict if a stock will go up or down seconds before change based on live search data. All it takes is for an article or something to come out, then watch people search for "Will IBM stock drop" - and perform live sentiment analysis across all such live queries involving the stock name.
In a sense, it's all about what information you have access to.
So the rumor is Goldman Sachs got out of the financial crash first because their systems saw the market move before anyone else. I don't know if that is apocrypha, but it is more to the point Google would not be predicting the market, but instead see market movements before anyone else.
I highly doubt that it is as technical as you seem to make it.
GS is very well connected, and they are constantly out there asking people for their opinions. Ten years ago they probably had some sort of database where people could share experiences, and that is most likely what led to them guessing the financial crash.
But it's not like they had some machine that one day told them to short mortgages.
I checked you bio are developing your own HFT strats? I ask because I am connected with a couple startup HFs. Ping me on Symphony {Brian Hewes} if your up for chatting about it.
The trouble with stock market systems is that any successful one is defeated by its own success - as its logic gets factored into everyone else's strategy.
This self-defeating problem is not present in the other AI applications you mentioned.
Again, I'm not trying to provide an exhaustive survey of AI or ML applications. Merely point out that the fallacy of using past failures of AI and ML to conclude that they will continue to fail at tasks against which they have thus far made limited headway.
Bear in mind that in the 90's only a handful of supercomputers had teraflops of computing power, where now you can get an 11 TFLOPS Titan X Ultimate for $1200. Compute power continues to grow exponentially, yet it has only recently reached a level where certain kinds of approaches are truly practical. As Heinlein said, "When it's time to go railroading, people go railroading."
It's interesting that you should talk about antagonistic systems, since Actor-Critic Models, dueling architectures, Generative Adversarial Networks (GANs) are an extremely hot area of AI/ML research at the moment.
A question - with the answer left as an exercise for the reader:
What do speech recognition, language translation, face recognition, object classification/detection have in common that is not true about predicting the future price of a security?
> It's inevitably about who you know, about "illogical" factors such as salesperson charisma, entertainment, and most importantly, a credible personality type that understands the edge case risks.
And this is why AI/ML will never be very disruptive in law.
The low-dow is: It's not about the facts, it's a social status game.
> because traders are used to seeing such predictions fail.
There was a joke from the 80s: soon the whole trading floor will be replaced by a computer, a man and a dog. The man presses the button to turn on the computer every morning. The computer operates all of the transactions and settlements automatically. And the dog is there to bite the man if he touches any other button.
yeah, i really don't know what these guys are talking about. the business day to day has been massively revamped by automation. there are hft ops that are blasting millions and millions of orders a second and probably account for 70%+ of all trading volume.
i trade financial futures and cash equity. i realize ibanks offer leveraged bets on all kinds of nonsense to provide a "service", but pretty much any strategy known to humanity can be achieved on an exchange with negligible counterparty risk
Being centrally cleared to reduce counterparty risk and being traded electronically on an exchange are two distinct things.
Electronic trading does change dramatically the shape and profile of a trading floor. Introducing central clearing with margining not so much.
You won't achieve electronic trading anytime soon on products where there are perhaps at most 200-300 buyers in the world, must of who are only interested in buying big chunks.
There are several floors that have "real" action. A couple of the options pits that are truly specialist and the CBOE VIX pits for instance come to mind.
But I don't think anyone believes those have more than a couple of years left in them.
This is a paraphrase of a semi-well-known quote from a respected business professor named Warren Bennis, about the "factory of the future" - who knows whether he coined it or adapted it from a common joke though. Here's his version of the quote, which I like a bit more:
The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.
The NYSE floor traders are only there for window dressing to be seen by the TV cameras. They are entirely redundant and watch movies for much of the day when there is no need to show a flurry of activity after opening or before closing. The entire building is no longer of any real importance.
Most of those floor traders is running a small business (2/3 guys/girls) based on past contacts and trader know-how (such as it is - some better, some worse.) It's really more of a 'bazaar' for small, likely well connected, trading firms rather than anything else. It has been 'hard' for them in recent years as order flow has moved to the largest, too-big-to-fail brokers (GS, JMP etc). But certainly the NYSE gets a lot out of the media coverage, but the brokers themselves are not owned by the NYSE.
There's a duality of thought going on here (with apologies to the false dichotomy). One says that all intelligence is ultimately materialistic/mechanical and that all we must do is retrace these steps with electronics and we can achieve whatever we want including high-value traders. The other seems to leave room for... let's call it the "irrational" side of things.
My only point here is that just about all values themselves are irrational. The same problem in trying to get a computer to recognize that the Mona Lisa (or poched eggs or Mozart or whatever) have value, is the same one that makes it hard to teach them to figure out if Tesla is still a good long-term investment etc.
One thing we fail to consider when talking about automation is that, at the higher end, tasks are complex and (man+machine) is much much better than machine alone. Further, the time-growth of (man+machine) will asymptotically dominate (machine). This will mean people with newer skills to master (man+machine) will be hired rather than jobs just going away. So predicting a particular job going away is a futile exercise at the higher end. The lower end is different, jobs there are simple enough to fully be automated away.
The trouble with stock market systems is they are great at predicting the past and pretty much useless for predicting the future.
A fellow student at Caltech was developing a stock market AI in the 1970s. He was very secretive about it, and was sure it was going to make him rich. I sometimes wonder whatever happened to him.
I just think that if AI could beat them, they were already replaced. Any innovation in trading is automatically implemented. May be this will be possible in the future but it doesn't depend only on deep learning techniques and having huge samples for learning because they have both.
The bulk of investment bankers are not picking stocks for funds. In the field of fund management, there are some people doing this, and it is a very common trend that passive indexes do better than the active funds - it is true.
In the field of investment banking trading (which is mostly market making) the amount of automation varies by asset class: very automated for some asset classes like fx, equities, much less for more illiquid asset classes like credit, commodities, bespoke products.
As well, senior traders operate as the 'business' making more decisions than pricing of products. They make the business decisions often judging legal, compliance, accounting risks. (and not always correctly.).
e.g.
do you accept to trade with a Dutch counterparty who wants to trade against your German legal entity knowing that you can only hedge the position in London? What is the risk between the two legal setups? What premium should you charge for those risks?
Do you trade the very large size that the counterparty wants, knowing it takes you over your balance sheet position limit - can you get approval for this from your senior management? Can you offset the position in the market without it moving against you? What premium do you charge for this?
Organisational inertia is huge at a big investment bank. It might take a decade to change their behaviour significantly, even when they're being out-competed by smaller firms. The big places simply do not need to compete so much due to the pseudo-monopolistic nature of their business.
If the role of investment banking is to optimally allocate capital, then part of that job is research. Think Andrew Left's exposing fraudulent Chinese tech stocks, or the Lumber Liquidators controversy. Algorithms can augment this work, but cannot replace it.
There seems to be some confusion going on about investment bankers and traders in the discussion.
Trading has been changing significantly since the 'big bang' when trading went from pits to electronic. From there on in you see the evolution of algorithm / program trading. This area has been using quants for decades at this point. There are a good few big names brands out there that are known for being 'algorithmic heavy', Man, Citadel, DE Shaw come to mind (I"m a few years out of date). That whole field has been open to introducing automation / algorithms to create a business edge and will probably continue to advance because its good for business. The profile of traders has also changed (Barrow boys versus PhDs)
Then I guess on the other side is investment banking such as m&a, equity and debt capital markets. Generally there its relationship based , juniors work on pitch books which from what I saw / heard were generally overlooked. This is potentially a lot harder to automate away. Then the bank would try to pull in some rain makers or grow them internally to land big deals. Usually these opportunities open up because their clients (Other companies) have learnt to trust the organization or at the least learn to expect a certain behavriour when enlisting their services.
Agree. But even the trading you are referring to is the trading of liquid products (essentially equity). A lot of OTC trading is still very illiquid and will likely not move to electronic platforms for the foreseeable future.
Interesting, I never thought about that. It seems like something very far off, though. Does anyone know about any research currently being done in this field?
Heh, it's the oldest thing there is in the field, almost. FORTRAN and COBOL were invented in the 1950s so that scientists and business managers could write their own programs without having to employ those expensive "programmers" who could write machine code.
Now it's 60+ years later, and there are still human programmers.
That's what (supervised) machine learning and neural networks do, isn't it. You specify the "what" and not the "how", they figure out the "how". Basically techniques for automating the task of computer programming given requirements.
So any advance in AI/ML/NNs can be seen as advances in the field of automating computer programmers. We don't tend to think of it like that because these techniques have to be applied, currently, by computer programmers, and the software the techniques "write" was generally infeasible to write by hand anyway: it's been so far purely additive and solving problems that hand-written software either couldn't do at all or where it sucked and progress was very slow.
But watch out. The tech is getting better scarily fast. Neural networks have been able to learn how to do basic algorithms like sorting given only examples. The primary limit is still the amount of data they need to work with, but R&D is driving that down too.
I can see a time when many tasks that today automatically require skilled programmers are instead done by domain experts who simply shovel data into an advanced neural network and discover it gets results that are good enough. Perhaps hand-written software made by a skilled programmer would be better, but the machine is a lot cheaper ...
People are working on it. Search for: artificial general intelligence. There was a book out last year called: "the master algorithm" that talked about it (the book is not that great). We all know about how well certain AI are doing now (neural nets etc), and historically there were high hopes for logic based systems (prolog) that are not as popular at the moment. Anyway, the author speaks of 4-6 different AI camps that are all separate (depending on definitions) and he believes that combining them together could result in gerenral AI. The book is a half decent review of some the less well known AI (that might be worth exploring) but it has the problem that it's too much info for the novice and not enough for a non AI CS person. He actually has code etc, can't remember the name of the system.
In an investment bank a trader has a high ratio of support staff around them: legal, compliance, IT, operations, quants, finance/tax, risk. These 'support' jobs are a significant fraction of the real cost of a trading seat - not just the trader's salary.
Automation has happened incrementally in the industry for years, like many others - starting with the easiest stuff (low hanging fruit) like some operations tasks and mechanical trading tasks, and leaving the more complex tasks for humans - or letting a human scale to do more.
The more complex tasks that are left typically require non-trivial intelligence, e.g. understanding why the new product brought to market by your competitor or counterparty is slightly different to what you are trading today, and deciding if you can/should transact in it. Understanding what impact the upcoming compliance rule changes have on your market and activities (there are always regulatory rule changes). Understanding what the limits of your trading are, to avoid concentrating too much exposure in one area. Understanding why your counterparty is upset about some aspect of the transaction. etc.
They are exactly what they claim to be: specialists. You need to become one in some area that banks deem valuable. Current white-hot areas would be FPGA, machine learning (but you'll also want a PhD in statistics, or at the very least a good degree in it from a good uni) or be a badass systems developer who can write absurdly low latency code in terms of allocation, cache coherency, network sympathy and so on. To get this good, you need to have been doing it for a decade or more, so start now. At the bottom. No one leaves university and gets one of these jobs. They leave university, maybe get a PhD or work for a decade and work up to them. You are jealous of people who have put in thousands of hours of very hard work into their careers, but will you do the same?
Really, you just have to start trying to do it. Find an old, good hashmap library in C somewhere, benchmark it and then try to reach feature and performance parity yourself.
Plenty of downsides to balance out the high pay and the excitement of working with big transactions. The hours and level of stress take a heavy, heavy toll on many people.
They've had to work insanely hard to get into and finish Harvard, only to be granted hundreds k of debt. Then they had to work their ass harder to get into hard-to-get into-companies and acquire the experience required to get into even harder-to-get-into companies (i.e. investment banking). After a decade, they're finally there and all they got is long hours, too much responsibilities and no free food.
Interesting challenges, brilliant people, nice co-workers, instant feedback loop, lots of resources at your disposal, large company benefits (i.e. pension/healthcare), quality matters, work with grown up.
A lot of which can't be found in the usual web startup.
231 comments
[ 2.6 ms ] story [ 247 ms ] threadThey aren't? Because I see this happening all the time. Automated checkout lines, E-Trade and online brokerages, etc. Low-value transactions are handled more and more by electronic "salespeople". All it takes to reach high-value transactions as well is for the clientele to decide they trust machines more than humans, which is not all that unlikely given the many sources of human error involved.
It's possible that humans will be involved in high-value transactions like investment for a long time, since the marginal cost of human labor is low as the numbers get bigger, but it's not impossible to replace; we already have the technology.
The examples you give are indeed places where buying or selling happens, but where does the incentive to do these transactions come from? It's still from humans. Sure, we are bombarded left and right with ads that were carefully targeted for us by complex algorithms, but I don't think I've ever bought anything from Amazon without reading a review about the product from another user, or bought stock without going through several opinions written by more-or-less respected analysts.
There is a line (few hundreds k) where it's enough to play but not enough to pay for the costs of a human to help you (sales people/financial advisor/trader type).
Huh? For one, investment bankers are "just salespeople" in the same way doctors are just nurses.
Second, everybody and their mother is suggesting AI will replace retail level salespeople in the future. In fact lots of places, from British supermarkerts (where you can bill and pay for your purchases yourself), to Amazon's Go experiment, and numerous other uses of automation in retail.
Many more lower-end stores did too, pre- or earlier in Amazon's and online shopping's proliferation. It's already happened to a not insignificant extent.
Bookstores is probably a famous example, where the people working there were far more than cashiers, but they still got decimated by Amazon and co.
Believing "I am special", is just built into us.
Although per passenger mile, it's true that air is very safe overall.
The overall statistics are not useful for evaluating your next trip. You must use the ones from your cohort.
1 - Mostly, the countries with broken governments. If human rights aren't respected, that's a great predictor that flights aren't safe.
If some belief applies to everybody, but on a different degree (per individual, or per job sector, etc), then the mere fact that it applies on everybody doesn't say anything to explain this variability.
And if what you say is that it applies to everybody with no variability, then that is obviously wrong. People in specific job sectors are way more worried about automation than people in other sectors, even if their jobs are not yet starting to get automated. E.g. office workers vs surgeons...
Of course, I wouldn't be giving the scripts to my employer, just spending more time doing other things...
Are you sure you haven't perhaps fallen prey to too much job security? Such a large sense of job security that you don't care about automating X part of it away. You, like me, like other "rockstar" software developers, (perhaps[0]) believe it doesn't matter if you automate X, Y, Z, because you're so good, that when you're out of responsibilities you will be offered the followup/orthogonal tasks 1, 2, 3.
Anyways, I'm just trying to highlight that you too, perhaps subconsciously, think "I am special, it wont happen to me", because at least I who also don't care about automation feel this way.
[0] I know for me, this is the truth. This overconfidence is why I don't care about automating my tasks away. Honestly, even giving the scripts to my employer makes little difference to me.
I'm not part of the cult of the "rockstar programmer". I do write a lot of good code, and I tend to do it much faster than other programmers that I work with. But programming is not actually my job description.
Without going into details - I do something else for living, that uses computers quite heavily, and automating my job is more of a hobby. I have a huge amount of flexibility in what I do, so there is an element of substituting 1,2,3 for X,Y,Z because I find them more interesting. My boss wouldn't care one way or another - he is more interested in whether or not the tasks get finished, and maintaining a high quality level in the final product.
assuming, very few companies which would have publically available published financial going over a large (> 50 years) timespan, would it be safer to conjecture that it is actually the _paucity_ of data which impoverishes the algorithms ?
Diligence is automatable, in the long run. There is a human element to it, but it is small in most contexts.
Bankers solve trust problems. An AI would need to be trustworthy in a personal way to replace bankers--this only happens with AGI. For a long time, as long as humans control capital, there will be other humans interfacing those humans with each other.
I'm not sure I agree with the first sentence, but I agree that there is a human element. If true, then at least some investment banking jobs must remain immune.
This happens with an unbeatable public track record. We trust AIs in lots of tasks already, and that is not because they smile and speech nicely.
There's only so much you can ask a kid 10 months out of Harvard econ to do, no matter how many pounds of cocaine and borrowed Excel sheets they have.
At the more senior levels, banking is relationship based, and making partner is a function of how much business you can bring in. This means maintaining good relationships with the clients you work with and reaching out to new ones. Show me an algorithm that can send cards on kids' birthdays, drink champagne, and play golf.
Once enough data is available, common sense and gut feeling can be programmed as a combination of fixed rules and machine learning.
cc @dang
[0]: https://news.ycombinator.com/submit
The job is actually extremely complicated. I wear many hats daily including working as a software developer, IT project manager/architect, salesman to clients, point of contact to other institutions trading desks, legal drafter, tax/accounting policy design, deal negotiator, risk taker/manager, systematic/fundamental strategy research etc. My job title is "Trader". There aren't any dumb big swinging dick 80/90's style traders left shouting buy and sell (if they are it's purely relationships and at small shops). Nearly everyone on my desk has a CS degree or similar quant background.
AI/deep learning is currently nowhere near being able to master one of these fields let alone cohesively manage all together in a direction to actually make money (just pay a "trader" a 200k-1.5mn USD a year its much easier). Most grunt work is automated and most outfits do in fact make use of statistical methods/classifiers/predictors and differentiable networks where appropriate. At the crux of it someone needs to manage, design the systems and fill in for them with a bit of intuition and common sense where there are gaps. The world is a chaotic place and its not getting simpler. Machines are not robust against chaos.
I worked in the finance industry as business facing software dev for 10 years and now work in a different but similar industry - my experiences can't be more different. My current industry's business is packed full of "traders" who don't trust the systems, won't work with them and outright refuse to admit the systems can outperform human's PnL numbers (especially not their own) despite being faced with raw numbers saying just that. It's a continuous battle between the technology teams being told to deliver automated systems to reduce costs and increase profitability vs plenty of "traders" who seem to see the systems as some sort of threat. As you say, most of the 'dump big swinging dick' style traders are long gone in finance - they're definitely still around in mine. My guess is that because our industry is significantly smaller meaning less employee opportunities, and the fact that almost all of these "traders" are relatively poorly educated (compared to those in the finance sector) they find it tough to elevate their own skillset to do things such as statistical quantitative analysis etc.
If it wasn't for the very high difficulty of entry, my industry is ripe for statistically minded, methodical, quant people to come in and make a killing.
If your job can be done overseas, it probably will be outsourced.
If your job is algorithmic, you'll be replaced by a computer.
A lot of M&A activity though doesn't fit that category.
You mean I can get an unbiased look at a house in peace, compare the numbers, look at the plans, measure the humidity and do my due diligence without a sales person breathing down my neck?
Hell yea. I'd pay premium for that.
People are more and more selling without a broker as well, or at least they start to use cheaper brokers where you only buy the service you need.
It's a scamola with a similar racket being Ticketmaster's "convenience fees". Most apartments in NYC will require going through their broker who collects a fee (usually a percentage of the first years rent). A chunk of that fee ends up being a kickback to the owner of the apartment. Ticketmaster does the same thing with venues (tack on $15 fee, give the venue $7.50).
would you hire a person for your team just by looking at degree transcripts, LOC written, and the resume? maybe you would, but I'm not too inclined to do that.
The HN bias is a bit like the salesman is the inefficient part disincentivized to help you and trying to rip you off.
The assumption you have is they are simply a market matcher but they're offering more than that.
Their incentive is to get the highest possible sale price in the shortest amount of time and a good reputation. That high price has to also be clearable meaning it cannot be just up in the air and seemingly arbitrary (it can cost either time or reputation).
Without them there is much more information asymmetry. Using an online version you basically have access to the same information you would as with them but without the human element to judge through nor their experience on offer.
The incentive for the agent is actually highest cash in down payment because they don't want to deal with the bank to get their money plus they get to keep the down payment if the buyer pulls out.
> plus they get to keep the down payment if the buyer pulls out.
What? Where does this happen?
In many places, the law is forcing you to go through an accredited agent to make a deal.
I don't know anyone who wants to pay that fee.
> You mean I can get an unbiased look at a house in peace, compare the numbers, look at the plans, measure the humidity and do my due diligence without a sales person breathing down my neck?
No, I don't think anyone means that. They mean purchase the house.
You can't go into a company and get them to show you their sales pipeline and their staff's performance evaluations before you (potentially) invest. If you want to do these things, you need to make a series of smaller deals first.
Human beings can be fluid, can enter into non-disclosure agreements and make these kinds of bespoke deals, while a web-based transaction form with a buy-it-now button cannot.
It's the biggest purchase most of us will make, so I understand the need for hand holding, but times have changed. The typical buyer is educated, and in all likelyhood knows about as much as the typical ex-cheerleader Realtor.
I thought the profession would be gone by now. It is ripe for elimination.
The only reason I would use a realtor is out of convention. And believe me, I would want to see a fresh copy of their omissions & errors insurance premium. (Need someone to sue, but that could easily be the Seller if they lied.)
Sorry if I sound bitter. I just don't get giving away 3% on huge amounts of money for essentially hand holding.
(In CA, the Realtors Lobbyists tried to cut down on new Brokers by making the Brokers licence harder to obtain. Gov. Schwartzeneger saw right through the ploy, and vetoed the bill. He knew they just wanted to decrease the amount of Brokers, so the Cheerleaders would have less competition. Gov. Brown let the bill ride right through. All of you should remember that when paying thousands of dollars (full commission) for a few hours of a Realtors time. When that bill passed, I became an Independant.)
There were still humans in charge of the algorithms, but they moved more towards Python programmers than market traders.
Many of the "old-style" traders bitched about what we did, and most moved jobs to banks that were less advanced.
(I was in the interest rates line; typical trade size is $10M)
This is achieved by discussing with sales people, who themselves discuss with investors, as well as traders at other banks (through brokers).
Now the definition of illiquid varies. Many products that used to be illiquid are now pretty liquid (interest rate derivatives) while many other are liquid only for small trade size (certain bonds). FX is an interesting example. It is very liquid but it is also a market where many clients need to make jumbo transactions on the spot, and these would be market moving if not carefully managed. That's something that could be probably automated.
I find the article very poorly written. "Investment bankers" is extremely vague and the breadth of very different product in different markets traded by investment banks makes this sort of generalization a bit absurd.
The other reason why I think the article doesn't make sense is that investment banks cater for certain giant markets (equity, FX, treasuries, etc) but also for a huge variety of niche markets, in which only a handful of banks and traders are active. If you add the salaries of the couple of traders in each of the active banks on one particular market, the savings you would do automating the market wouldn't justify the years of IT development to train and fine tune an algo, which would still need to be maintained as the market evolves (and then you end up overpaying an AI expert instead of overpaying a couple of traders...).
I think AI will shine at solving wide problems that affect a large number of people. Self driving cars. Butler robots. Building a house. Manufacturing something common. These markets have the scale to justify large investments.
I very much doubt that AI will replace every single complex task done by a man today, for the very same reason that software hasn't replaced every single manual task done by a man today: the cost of developing and maintaining software can easily exceed the salary of the few guys you are trying to replace. To overcome that with software, we need to dramatically reduce the cost of developing software, enabling ordinary employees to develop their own software. But even today we are very far from that. What's the % of a generation who can actually code out of college today? How easy and useful are the main programming languages? I'd argue we are now going into the opposite direction. Microsoft is poised to take VBA out of office. All major OS are evolving toward the iOS-style locked down platform where you can only run Apple/Microsoft approved software. Corporate IT is ever more locking down platforms with software whitelisting, etc. I wonder if the golden age of productivity improvement through software has not peaked.
I think this is the key to running an effective organisation.
It requires that most people know how to program, or at least know enough to understand what is possible to program.
I also don't see this coming, partly because the education systems are not really up to par, but also because it's really hard to develop complex systems.
I'd argue that the languages are not really the issue here even if the trends seems to go towards even crappier languages (like js, php) to really mess up the heads of beginners.
But even with a hypothetical "perfect language" the main problems is the huge amount of time and effort it takes to learn any programming language at all, plus the even more enormous amount of time it takes to write something useful even for a really experienced developer.
But sure. It's perhaps time for a new cycle of tearing down the "mainframes" (in someone else's basement this time) and reinvent personal computing for the 2nd or 3rd time...
Of course, they did know more than the machine did in certain situations, but not enough to make up the cost.
I don't think this is true. Not at all.
You don't need "any ML system" to be able to not only process the data, but also explain the strategy behind its data processing at various levels of granularity, answer abstract questions and do a convincing impression of listening and responding to client feedback, you need advanced general intelligence.
As for what the more reasonable answer a client would accept is, I don't think I really have the requisite years of investment banking experience to know that...
A trader is not necessarily a sales or a broker, though. Nor is he a quant, or a dev. People rarely imagine how many different jobs are involved in the trading job, and how rich the business logic is. At my shop, traders are the piece that connects all of the jobs in the value chain.
I believe, currently, the business logic can be improved locally by learning systems (and it is), but there is no public example of an industrial learning application encompassing a scope comparable to what the usual trading desk handles. Sure, there are many inefficiencies ; traders work on heuristics, afterall. But I don't believe we have the necessary horizon to aptly predict the end of traders, because I don't see how we could make AIs with a better efficiency.
I do foresee a future in which a trader can accomplish a lot more than he/she can today using AI. So in the future as the per trader efficiency increases, the number of traders required will most probably decline, unless there is a dramatic increase in trading volume that cannot be matched by the then state of the art AI.
This is essentially what's happening today with X.ai, Facebook messenger and the like. Sure the logic involved in booking air tickets for a group of 5 over 10 conversations isn't as complicated as the rich business logic of a trade, but 5 years back Facebook messenger would've seemed almost impossible, just like the trading business logic seems impossible to do with AI today.
On the other hand, the technology advances needed to transform the tools into standalone actors are not merely a matter of scaling current technology. Especially, the creation of training datasets is a problem for which we currently have no solution, that's why we fall back on human trainers (mturk, etc). That's why a solution to a problem with no clear, bounded model and no easy dataset seems out of reach.
Answering my own question: I'd expect bankers to save more (of their own money) in anticipation of the good times not lasting as long. More conservative types will weather the storm and spendthrifts will get wiped out.
In other words: business as usual, up until the very moment it isn't.
Reuters was trading FX electronically since the early 1990s. At the tier one IB I worked for the IT budget was 500m USD a year (across products), and that was in 1997! Huge resources were thrown at automation. However, to this day, large trades in FX (> 10m USD notional) are still almost exclusively performed by humans over a telephone or over the bloomberg messaging system.
That's because, no matter how much you automate stuff, there is still the 1% "edge case" scenario where something goes wrong, and when that happens, you most definitely want a human that you can "look in the eye", when you have that sort of execution risk. Remember that markets move really fast and there is a lot of risk in big trades that "go wrong" because unwinding said trade will almost certainly cost one of the sides a fortune.
Also, high finance is not just about what you know. It's inevitably about who you know, about "illogical" factors such as salesperson charisma, entertainment, and most importantly, a credible personality type that understands the edge case risks. These things are very hard to replicate with a machine. You'll say they should be, that these things are unfair, but they remain a fact after many attempts at removing them have failed.
As for AI, let's for now call it what it is: machine learning. Learning from the past. That's fine for recognising stop signs at different distances, angles and degrees of noise. But in finance, the past is often misleading. Sure there's trend, but there are also very big instabilities in the historical correlation matrix. Paradigms shift without you even realising it. The constant is change. AI is not good enough at that, yet.
BTW, that's not to say machines are not making inroads. It's becoming almost impossible to get a decent trading job now with knowing at least R and Python to a comfortable degree, and good quant programmers cost a fortune. There's massive demand.
That explains the LIBOR scandal.
The reality is that speech recognition, language translation, face recognition, object classification and detection, semantic segmentation, speech and image synthesis have improved by extraordinary leaps and bounds in recent years. If we used the logic that past failures inform a confident belief that future success in a challenge will inevitably fail, then we should've bet heavily against Alpha Go defeating one of the most accomplished human Go champions in the world. Self driving cars seem like a sci-fi fantasy until they become a mundane reality.
There's an irrational arrogance to human beings in general, and Wall Street types in particular, regarding the specialness / non-reproducibility of their intelligence. It's not unlike the belief that people had that organic molecules were somehow special, "vital," and not synthesizable from base elements.
Certainly, there's a long way to go to replicate the capabilities of a human brain, but I don't think we should exaggerate or fetishisize the human power to estimate and mitigate risk. We've seen many spectacular failures of that in recent years.
https://deepmind.com/research/alphago/
https://en.m.wikipedia.org/wiki/Quantitative_easing
Google is getting really good.
Frankly, rules based AI is basically a bust compared to learning from data approaches.
The hive was pretty amazing at destabilizing the entire global economy within a brief span following fundamental financial de-regulation.
People flatter themselves in ways which satisfies their egos and self-interest. Of course they want to believe in, and have everyone else credit, their magical superpowers. No government (backstop) wires necessary.
Machine learning is rule based. The difference is just that it creates many more rules for more and more granular cases. But it will never be intelligent, it will always be a dumb digital bureaucrat.
Beating a human when there is no discoverability to the problem, and when building an exhaustive dataset is a problem in itself is not on the horizon yet.
You may have missed various dramatic improvements over the past year:
https://scholar.google.com/scholar?as_ylo=2016&q=%22image+ca...
Anyway, this paper demonstrates some impressive results:
https://arxiv.org/pdf/1611.06607v1.pdf
And even if you could, acting on that knowledge would change the future.
Casinos don't try to predict the future of every spin of the wheel or roll of the dice. Every individual game is random. But over a large enough sample size (number of games played), the odds are not random.
The same principle is at play in trading the markets, except the "odds" are determined by the traders' skill - their "edge." Which is roughly some combination of access to the right people, access to better information (where better might but not necessarily mean quicker), and experience (gut feelings, hunches, what-have-you).
So the real question is, can machine learning algorithms develop an "edge", and if so, can they stay solvent long enough to do so.
The stock market isn't a game; it isn't designed and it continuously evolves.
That's exactly what I meant by odds; they always win because the odds of the games are in their favor.
> The stock market isn't a game
It's a game in the same sense that life itself is a game.
> it isn't designed
The mechanisms that fit all the pieces together are very much designed by humans. True the behavior of the market isn't designed; it's an emergent property of the complex system we call "the markets."
> it continuously evolves
Yes, but change is also the constant. What underlies that change is supply and demand.
Of course you're free to disagree. But if you'd like to learn more about the not-100%-accurate analogy I described in abbreviated form, the source is here: http://www.goodreads.com/book/show/253516.Trading_in_the_Zon...
Edit: Good reading on "life as a game": http://www.goodreads.com/book/show/189989.Finite_and_Infinit...
Life isn't a game.
Thus far, machines have not done well in scenarios of low stationarity. Those are scenarios where the past is not so good a predictor of the future or where the data manifold is rapidly changing. This occurs for example, when the act of predicting changes what is being predicted. The related notion of Antipredictable Sequences is the most interesting consequence of the oft misconstrued notion of the No free lunch theorem.
Humans are far from the ideal machine for this scenario but they vastly outmatch current algorithms. This is because humans are much better in the kind of generalization that is based off theory building. While not perfect, it does well enough at generalizing from observations (think Newton deriving the laws of gravitation from observations and Brahe's data) to scenarios that are not so close to the data.
This means that for now and in complex environments, simpler models that are adaptable to change online are better. Additionally, their inspectability means they are more easily modified to better match any very changed dynamics.
So your examples of: `speech recognition, face recognition, object classification and detection, speech`
Are of the type where a learner, such as a deep network, whose generalization ability mostly hails from interpolating to fill in missing structure based on data (implying poor out of sample performance) does well when you can give lots of examples which provide good coverage.
Language translation is sort of inbetween (the space is not as smooth but is still fairly stationary at the scale of years) and so we also find that the advantages of deep learning have not been as pronounced and dramatic as has been the case for image and speech.
The digital machine advantage is two. Their memory capacity is outmatched. Second, and related to the first, energy needs are not so pressing a concern.
Citing Newton as a typical example of the superiority of human inferential abilities perhaps represents a case of cherry picking. How often do most human beings come to exhibiting the level of insight and reasoning power required to construct the calculus and discover the laws of classical mechanics? Inventing supernatural explanations for natural phenomena, burning sacrifices / witches / books (on Evolution or other heresies) seem more par for the course than Newtonian level revelations.
Having said that, learning the laws of classical physics from observational data strikes me as a pretty natural task for the right kinds of machine architectures, since they represent essentially geometric symmetries which hold over a vast range of scales, space and time.
If our mathamechanical minions ever start musing about 'The Supreme Eigenvector' that would be the time to start worrying about machines taking over.
> Citing Newton as a typical example of the superiority of human inferential abilities perhaps represents a case of cherry picking
Are hyperparameter searches across a sea of AWS instances cherry picking?
Anyways, I don't think I cherry picked. There is not a single machine that can do the same yet. And more importantly, a learning algorithm that notices model failure and works out the surgery required to correct or even completely replace it.
Newton is an example of the human brain at its best. But he was far from the only: Maxwell, Green, Einstein, Noether, Archimedes and so on the list goes. But we don't need to go that far since Crows are already capable of generalization that out matches what machines can do for the near future.
A hypothetical "robot-cat-AI" could for example have access to a really precise physics engine, and also could run at a much higher "frame-rate", so it could see the world in slow-motion and execute really exquisite moves as a result.
When it comes to comparing the sophistication, adaptivity and functions of life to the in-silico mashines we produce and use very, very primitive.
In a sense, it's all about what information you have access to.
GS is very well connected, and they are constantly out there asking people for their opinions. Ten years ago they probably had some sort of database where people could share experiences, and that is most likely what led to them guessing the financial crash.
But it's not like they had some machine that one day told them to short mortgages.
This self-defeating problem is not present in the other AI applications you mentioned.
Bear in mind that in the 90's only a handful of supercomputers had teraflops of computing power, where now you can get an 11 TFLOPS Titan X Ultimate for $1200. Compute power continues to grow exponentially, yet it has only recently reached a level where certain kinds of approaches are truly practical. As Heinlein said, "When it's time to go railroading, people go railroading."
It's interesting that you should talk about antagonistic systems, since Actor-Critic Models, dueling architectures, Generative Adversarial Networks (GANs) are an extremely hot area of AI/ML research at the moment.
What do speech recognition, language translation, face recognition, object classification/detection have in common that is not true about predicting the future price of a security?
[1] https://www.bloomberg.com/news/articles/2016-10-13/this-bank...
And this is why AI/ML will never be very disruptive in law.
The low-dow is: It's not about the facts, it's a social status game.
There was a joke from the 80s: soon the whole trading floor will be replaced by a computer, a man and a dog. The man presses the button to turn on the computer every morning. The computer operates all of the transactions and settlements automatically. And the dog is there to bite the man if he touches any other button.
30 years later, still no dog on the floor!
Electronic trading does change dramatically the shape and profile of a trading floor. Introducing central clearing with margining not so much.
You won't achieve electronic trading anytime soon on products where there are perhaps at most 200-300 buyers in the world, must of who are only interested in buying big chunks.
I believe the remaining ones are effectively TV studios for business channels, so they have a completely different purpose now.
But I don't think anyone believes those have more than a couple of years left in them.
The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.
My only point here is that just about all values themselves are irrational. The same problem in trying to get a computer to recognize that the Mona Lisa (or poched eggs or Mozart or whatever) have value, is the same one that makes it hard to teach them to figure out if Tesla is still a good long-term investment etc.
A fellow student at Caltech was developing a stock market AI in the 1970s. He was very secretive about it, and was sure it was going to make him rich. I sometimes wonder whatever happened to him.
In the field of investment banking trading (which is mostly market making) the amount of automation varies by asset class: very automated for some asset classes like fx, equities, much less for more illiquid asset classes like credit, commodities, bespoke products.
As well, senior traders operate as the 'business' making more decisions than pricing of products. They make the business decisions often judging legal, compliance, accounting risks. (and not always correctly.).
e.g. do you accept to trade with a Dutch counterparty who wants to trade against your German legal entity knowing that you can only hedge the position in London? What is the risk between the two legal setups? What premium should you charge for those risks?
Do you trade the very large size that the counterparty wants, knowing it takes you over your balance sheet position limit - can you get approval for this from your senior management? Can you offset the position in the market without it moving against you? What premium do you charge for this?
Trading has been changing significantly since the 'big bang' when trading went from pits to electronic. From there on in you see the evolution of algorithm / program trading. This area has been using quants for decades at this point. There are a good few big names brands out there that are known for being 'algorithmic heavy', Man, Citadel, DE Shaw come to mind (I"m a few years out of date). That whole field has been open to introducing automation / algorithms to create a business edge and will probably continue to advance because its good for business. The profile of traders has also changed (Barrow boys versus PhDs)
Then I guess on the other side is investment banking such as m&a, equity and debt capital markets. Generally there its relationship based , juniors work on pitch books which from what I saw / heard were generally overlooked. This is potentially a lot harder to automate away. Then the bank would try to pull in some rain makers or grow them internally to land big deals. Usually these opportunities open up because their clients (Other companies) have learnt to trust the organization or at the least learn to expect a certain behavriour when enlisting their services.
Now it's 60+ years later, and there are still human programmers.
So any advance in AI/ML/NNs can be seen as advances in the field of automating computer programmers. We don't tend to think of it like that because these techniques have to be applied, currently, by computer programmers, and the software the techniques "write" was generally infeasible to write by hand anyway: it's been so far purely additive and solving problems that hand-written software either couldn't do at all or where it sucked and progress was very slow.
But watch out. The tech is getting better scarily fast. Neural networks have been able to learn how to do basic algorithms like sorting given only examples. The primary limit is still the amount of data they need to work with, but R&D is driving that down too.
I can see a time when many tasks that today automatically require skilled programmers are instead done by domain experts who simply shovel data into an advanced neural network and discover it gets results that are good enough. Perhaps hand-written software made by a skilled programmer would be better, but the machine is a lot cheaper ...
http://www.i-programmer.info/news/105-artificial-intelligenc...
well, until the day an AI will be good enough to write and improve itself, then we'll all be doomed equally.
Automation has happened incrementally in the industry for years, like many others - starting with the easiest stuff (low hanging fruit) like some operations tasks and mechanical trading tasks, and leaving the more complex tasks for humans - or letting a human scale to do more.
The more complex tasks that are left typically require non-trivial intelligence, e.g. understanding why the new product brought to market by your competitor or counterparty is slightly different to what you are trading today, and deciding if you can/should transact in it. Understanding what impact the upcoming compliance rule changes have on your market and activities (there are always regulatory rule changes). Understanding what the limits of your trading are, to avoid concentrating too much exposure in one area. Understanding why your counterparty is upset about some aspect of the transaction. etc.
You have to be bright, articulate, interested.
Don't be.
They've had to work insanely hard to get into and finish Harvard, only to be granted hundreds k of debt. Then they had to work their ass harder to get into hard-to-get into-companies and acquire the experience required to get into even harder-to-get-into companies (i.e. investment banking). After a decade, they're finally there and all they got is long hours, too much responsibilities and no free food.
A lot of which can't be found in the usual web startup.