44 comments

[ 2.6 ms ] story [ 91.9 ms ] thread
I asked him how the bonds worked.

“You put chicken into the grinder”—he laughed with that infectious Wall Street black humor—“and out comes sirloin.”

I'm speechless.

> Now that I was spending more time on the floor, I wondered why the men’s room always stank. Then one afternoon at three, when I was in there taking a leak, I discovered the hideous truth. Traders had a contest. Coming in at eight, they never left their desks all day, eating and drinking while working. Then, at three o’clock, they marched into the men’s room and stood at the wall opposite the urinals. Dropping their pants, they bet $100 on who could train his stream the longest on the urinals across the lavatory. As their hydraulic pressure waned, the three traders waddled, pants at their ankles, across the floor, desperately trying to keep their pee on target. This is what $2 million of bonus can do to grown men.
I'm sorry to have to point this out, but this comment doesn't deserve 30 points. It's just a quote from the story, it's not insightful and doesn't add anything of value to the comment thread.
"Once upon a time, this [securitization] seemed like a very good idea, and it might well again, provided banks don’t resume writing mortgages to people who can’t afford them."

While the banks shouldn't escape blame, didn't this securitization facilitate shoveling the risk onto the buyers of these 'bonds', encouraging sloppy lending practices?

I think the problem was the loss of semantics from one side to the other. As long as the buyers are informed, all else should be fine.
Is that even possible when mortgages are leveraged at a rate of thirty to one and sliced and diced and bundled in every conceivable way?
They are sliced, diced, and leveraged when they are resold to investors as MBS or CDOs, not when presented to the loan recipient
when they are sliced for specific dimensions, then, yes, the semantics should be a default property of the result. When sliced and diced for obfuscation, not so much. If they were indeed obfuscating purpusefully, then there is a fraud case in here. This however does not make the technique guilty by association.
"If they were indeed obfuscating purpusefully, then there is a fraud case in here."

Unfortunately, not quite. Our laws regarding these things are very primitive, which often means that you can be in perfect legal compliance, yet still commit some very severe fraud against the people you're making offerings to.

I find it rather unlikely that our regulatory system will ever catch up with the state of the art in finance. There's too much money to be made in the short term by shoveling risk off into the cracks and pretending on paper that it doesn't exist, and it's too easy to do.

How about regulation that requires revealing everything, but that does not require any minimum standards?
One of the flaws of these articles is that they are vastly simplifying the instruments built. From reading, you might think that you understand how to build a CMO or CDO, or to analyze one. I don't, and you don't. I am not actually sure that, after the author of this article's software went into place anyone could. The government has had months to examine the private books of some of our more nationalized-like asset managers, and is still throwing darts. Regulation that requires revealing everything, but does not require minimum standards, is like being able to bypass QA and security reviews by providing access to the source code... to multi-million line programs. Yes, eventually you can work it all out, but not in time to be helpful.
You have to wonder what the buyers where thinking. One problem was that the rating agencies (paid by the companies creating the products, BTW), rated the stuff AAA.
> You have to wonder what the buyers where thinking. One problem was that the rating agencies (paid by the companies creating the products, BTW), rated the stuff AAA.

The biggest reason why this "stuff" was rated AAA was that banks backed them fully, ie if they defaulted then the bank would step in and make the investors whole again. Hence the CMO's got the rating of the bank that provided them.

The thinking was that some mortgages would go bad but certainly not a large percentage of them.

...confessed to him that I had never studied finance, and I had only taken one course in computers

This lack of formal qualifications seems common on wall street, perhaps this isn't surprising.

I know I didn't want to work as a programmer for a finance company because I'd prefer to be a first class citizen in a software company even for much less money.

Perhaps wall street only gets second tier engineers, scientists and mathematicians?

I know of one specific case, where yet another person without formal training made it high up as a programmer at a well known large institution who's name I'll withhold.

This person was very well compensated for doing the kind of horrific wheel reinventing that only smart people without formal training can do.

Needless to say all kinds of wacky hi jinks ensued over that person's years at wall street.

In one instance he like all other programmers was forbidden to use malloc. And everything was on the stack and of size 512 - EVERYTHING. Who made that decision? Why the financial managers obviously!

And I'm thinking that a big part of wall street's downfall is all that under qualified personnel, there primarily for the money.

I think very few quants and people in their positions ever had the confidence and power to stand up to the finance guys.

I think if finance firms went to the trouble of hiring people who have better options they might not be in so much trouble.

Then again if you're gambling with other people's money and your compensation only has upside, no downside with risk - nothing would have prevented the crash.

This is not a story about 2nd rate thinking or a mediocre or bad software product. When a freshmen flunks a semester because he got a new video game, it is not because the video game's designers were 2nd rate.

And Wall Street attracts people seemingly as smart as you would ever want to be. Read Roger Lowenstein's "When Genius Failed" about how a coupla Nobel Prize winners in econ started a hedge fund (Long Term Capital Management) which blew up spectacularly in 1997. Other smart people knew what they were doing wrong, but that didn't matter to the people who saw how much money they were making over the few years they survived.

Actually, they knew what they were doing wrong. Merton and Scholes pulled much of their money out a few months before it crashed, walking away with a decent profit (they, of course, lost nearly everything on the money they still had in the fund when it was recapitalized, eating into their returns). That's all in "When Genius Failed" too.

Technically they didn't start it either, they were just on the board of directors.

Not even close! I don't think you'd want to hear it, but whereever a huge amount of money is, you will find some of the world's smartest people.

People and businesses don't make such astronomical amounts of money just on sheer dumb luck. They do it by being very smart, knowing the system and analyzing it for maximum profit. Of course, human nature being what it is, wealth can cloud the vision of even the smartest people, leading to the situation we have now.

But don't think for a second that Wall St. is populated only by "lower tier" thinking.

In Savings and Loan crises of 80s, wall streets banks lost all the money they made during the previous 25 years. In the current crises, I dont know the exact number but it is worse than that.

So if you believe that wall street banks actually make any money, you might want to reconsider your view points.

[shrug] When I die, I'll lose all the money I made over my lifetime, anyway. What's your point?
It's not necessarily the banks. Wall Street bankers made a lot of money.
Point taken, I've always suspected my personal experiences are not an accurate representation of the general case.

I'm just wondering if there is a difference between the quants at partnerships where the traders had skin in the game, and the quants where the people who made tons of money had no downside risk?

Perhaps wall street only gets second tier engineers, scientists and mathematicians?...I think if finance firms went to the trouble of hiring people who have better options they might not be in so much trouble.

There are quite a few fantastically smart people on wall street. It's quite easy not to care about being an N-th class citizen (for N >= 2)) when you make $500,000/year before you are thirty.

The trick is to be both a first tier programmer/scientist and to be able to put yourself into a situation where those skills matter.

You may _think_ that no "real" programmer would ever want to work for a bank when they could instead work for a software company, but that does not make the people working at banks any less talented or competent.

It just means they have different priorities than you do.

> In one instance he like all other programmers was forbidden to use malloc. And everything was on the stack and of size 512 - EVERYTHING. Who made that decision? Why the financial managers obviously!

Many financial software houses operate under very strict latency constraints, where a microsecond or two more means that somebody else got the trade you were going for. I've even heard of hedge funds colocating boxes on the exchange floor so that they can avoid the network delay of sending messages a few blocks to headquarters. In this environment, the cost of dynamic memory allocation can be significant, and garbage collection is most certainly out.

Same thing in the avionics industry and other hard-realtime situations: malloc is forbidden because its time complexity cannot be predicted by the programmer.

I have no idea if this is why he couldn't use malloc or whether it was plain management stupidity. But the idea is not as absurd as it seems at first.

(comment deleted)
In this specific case time constraints had nothing to do with that.

On other occasions time constrains were a very big deal as were atomic transactions, except that NO one around had heard of the terms real time and atomic transactions. So they did a horrible job of reinventing something like a good DB.

You can't blame yourself for what inevitably happened. It's the people who put only best case scenario numbers into every variable in the risk modeling. And even if you removed that from the equation, you were just providing a product banks wanted and if you hadn't someone else would. The disaster would still have occurred.
Well, a bank could also say "if not us, someone else would".

"the disaster would still have occurred" logic is flawed.

Not that I want to blame the guy, he wrote an impressive article after all :)

Great read.

I really appreciate it when posters take the trouble to link to the printout version. So much more readable, this is bliss. Thanks!

Unfortunately, the full-width text harms readability far more than removing the advertising and navigational clutter.

My browser window is typically sized to a width of around 1200 pixels -- pretty typical I would think -- so the printer-formatted version results in lines of around 30 words. That's too long to be able to read comfortably, and I think it's unreasonable to expect me to resize my browser depending on the type of website I'm reading (I have it as wide as 1200 pixels for a reason).

The standard, screen-formatted version (http://nymag.com/news/business/55687/) is specifically designed at a fixed width for comfortable reading.

I agree in principle that there are a lot of distractions on the screen-formatted article, but linking to the printer-formatted version does not seem like the right approach. Perhaps the best current solution is that readability Greasemonkey script that appeared a few weeks ago.

I solved that problem by putting a JavaScript bookmarklet in my browser that narrows page text. I found this online, but I forgot where; hopefully HN doesn't mangle the following text... (you'll need to remove linebreaks to get it to work)

    javascript:(function(){var%20newSS,%20styles='body%20{%20margin-left:
    auto;%20margin-right:auto;%20max-width:45em;}';if(document.createStyleSheet
    )%20{document.createStyleSheet(%22javascript:'%22+styles+%22'%22);%20}%20
    else%20{%20newSS=document.createElement('link');%20newSS.rel='stylesheet';%20
    newSS.href='data:text/css,'+escape(styles);%20document.getElementsByTagName
    (%22head%22)[0].appendChild(newSS);%20}%20})();
I've started to use the bookmarklet at http://lab.arc90.com/experiments/readability/ for most long articles -- it boosts the font size, increases the margins, and cuts out the background and sidebars. Unfortunately, it didn't seem to work for this particular article.
Great find! Much easier to work with. (But it did work on my system on that article; maybe it was just your specific browser config...)
Unfortunately for me, that bookmark and its code requires me telling NoScript that I trust each website - Not something I care to do, at this point. But I agree that it works quite nicely on sites I "trust."
This is probably the first substantial article I've seen on the lead-up to the recession written by someone who's "one of us." It seems like much of his career activity took place a decade ago, but much of what he relates still rings true for computer scientists and software engineers on Wall Street today (excluding those that go straight into trading, banking, etc., of which there are a lot).

Until the recession hit, the odds of you making well over six figures in three or four years without even being that extraordinary were pretty high if you worked for one of the big five investment banks, and if you went to work for a hedge fund, you could be making much, much more than that without having to become a "business person."

It's more demanding psychologically (ironically, sometimes what I think is most poisonous to technologists' morale on Wall Street is the occasionally subservient and submissive attitude of some of their peers towards the traders and bankers) than working for a normal software company is, and your absolute maximum monetary return is still the highest with a startup, but in a way, it's an interesting middle ground between the stability of a regular software job and the potential payoff of a startup (just looking at the money here, personal fulfillment is a completely different discussion).

And although the whole "second-class citizen" thing does sting, it is interesting to see how much softer the criticism of the article's subject is on the New Yorker site compared to the vitriol usually hurled at an actual banker or trader; in fact, a few months ago, in the midst of the worst part of market volatility, one of the technology directors where I work was stopped for talking on a cell while driving; the cop asked him where he worked, and after telling him, our director implied that he was only let off without a ticket because he was in tech and not a banker.

" I used to go to the trading floor and watch my software in use amid the sea of screens. A programmer doesn’t admire his creation so much for what it does but for how it does it. This stuff was beautiful and elegant."
A great read. It makes me think about the software I write on a daily basis and how it's effects may be felt much further than the end user I am providing it directly to. Interestingly, I currently write software (admittedly much less complex) for a mortgage company, so the article hit home.

Regarding the author - should he be "blamed" or feel ashamed of his work somehow? I don't think so, and I'm glad to see that he doesn't think so, either. The software he was commissioned to create worked and performed well for it's intended purpose.

What if a software engineer lands a job working for a defense contractor to create programs for missiles, bombs, or other weapons? Are they to blame for the world war? Of course not, but that seems to be what some people are thinking. [I'm not sure if that comparison is fair, but it makes sense to me.] Could they have turned down the job based on principal? Sure, but sometimes a job can be very tough to come by. Besides, who would think a simple programming job could possibly lead to such catastrophe, especially in the business world? Maybe that's a take away, or at least something to think about when looking for your next gig.

The crisis in a nutshell...

    "Oh, people who have defaulted on loans in the past. That’s why
    they’re called subprime," he informed me.

    "But if they defaulted on loans at 8, how can they ever pay back a
    loan at 16 percent?" I asked.

    "It doesn’t matter," he confided. "As long as they pay for a while.
    With all that excess spread, we can make a ton. If they pay for three
    years, they will cure their credit and re-fi at a lower rate."

    That never happened.
While there are those out there who would look at the writer like a gun manufacturer whose product ends up in the hands of criminals, it's really all the fault of the people who made this greedy risk, practically knowing that defaults were going to happen anyways, that really deserve the blame in this mess...
Good grief... another plum passage...

    I asked my colleagues, what was a CDO exactly? Like CMOs, they were
    structured products, but the underlying collateral was not limited to
    home mortgages. They could be anything...  "Diversity of collateral"
    was the pitch. Some things could go bad, but not everything at
    once. It never has, except during the Depression, and we’re so much
    smarter now. That could never happen again.
Sorry for the self-reply, but this article has taught me more about this fiasco than all the press and commentary over the last year. An impressive piece of writing...
As a redemption he should write that again and publish as an Open Source library - so that everyone could analyze what is going on.