Listening to Jordan ramble at the bottom was really fun, seeing his personality come through. He taught me my discrete class just a year ago, I got so lucky.
> According to one online source, Jeremy was a “Harvard-educated maths genius whose computer models alerted the bank to how small levels of defaults would quickly turn apparently sound assets into junk,” leading Goldman to start selling off at the end of 2006. OK, whatever.
Are you kidding me? This is huge; this is what prevented the financial crisis, as bad as it was, from turning into something much worse (it's "bad" for a bubble to pop, but it's worse for it to keep growing and pop later). This guy made the difference between billions of dollars going into houses that people couldn't afford and weren't worth the cost, and that same money going into productive investments. He probably contributed more to humanity than the rest of the list put together, with the possible exception of the other Goldman guy.
I know everyone thinks their own field is the most important, and I love academic maths, but goddam the snobbery towards the people who did something more directly useful with their talents is irritating here.
You ask why I said “That’s too bad” after learning that Gregg is a vice president at Goldman Sachs.
Perhaps the best analogy is, what if you knew someone who was one of the top high school basketball prospects in the country, but instead of ending up in the NBA he made a career as a sports hustler, using trick bets to make money off suckers. This is not a perfect analogy, as this sort of sports betting is illegal, I guess, but the point is that it’s sad to see a great talent that is being used neither in its pure form or for scientific progress or the public good.
If you're a talented mathematician who thinks that the financial industry is a net positive for society, you're most probably already working in it. It undoubtedly pays the best and generally recruits talent.
So, if you're a talented mathematician not working in that industry... it's reasonably likely you've made a conscious decision not to work there - perhaps because you don't agree that it's directly useful to society.
There's a reason why the financial industry has to pay the highest salaries. It's surely not out of generosity.
Its not that finance is bad, people who love to work in some field which doesn't pay well always tend to think lowly of people who work in other fields and get paid well.
Lots of people just don't think of financial modeling as a particularly interesting or beautiful area of math (this is of course a matter of taste). You could believe that finance is socially useful, and yet prefer for selfish reasons to work in algebraic geometry instead.
>Lots of people just don't think of financial modeling as a particularly interesting or beautiful area of math
I kind of surprised how those things are frequently mixed together. The math is a science while various industry modelling is an application of the known/established tools/skills. It is like designing a drill vs. actually drilling a hole. (note: i have an MS in Math (3.8 GPA) from one of the top Russian schools)
"There's a reason why the financial industry has to pay the highest salaries. It's surely not out of generosity."
They are one of few who can afford to pay higher, and give all kinds of perks. Everybody wants the best of breed to work for them, don't we? On the other side, while searching for a job, the lure of neat cashflow is a strong persuator for many.
Plus there is plenty of pretty clever people in banking crowd (maybe some of them don't use their talent to max or in a way they would find fulfilling, but that's another topic).
There were plenty more people who came to that conclusion many number of times. There were companies who changed their whole models just so they could sell the software because the result said "you will lose everything" to half the AAA securities out there.
That said, I too dislike his tone about finance. Why? Why is it bad to work in one industry over another?
Because the money industry doesn't actually create anything of value, it just moves money around and takes its cut.
Baker? Sure, making cookies adds value to the world. Software? Yeah, writing programs adds something to the world. Holding onto money, lending some of it out, and taking a cut? Not so much.
Yep, and all of those middle-man business that add almost no value should be squeezed down to razor-thin margins and turned into commodities by competition.
That's slowly starting to happen in the financial industry, with robo-advisors and cheap index funds taking trillions of dollars out of the hands of overpaid Wall Street firms, but that process takes time. Hopefully we'll start to see the same thing with Lyft and Sidecar eating into Uber's nonsensical valuations.
And yet you are both happy to do it; if you couldn't, you would have a worse living situation and he would be poorer. That looks to me like a paradigmatic case of value being added to the world.
It's counterintuitive, maybe, but any sort of trade adds value to the world even though all that's happening is that goods and/or money are being moved from one place to another. What enables this is that different people have different preferences; in the case of your mortgage, it's differences in how you value future money relative to present money -- you want money now and having less money later is a cost you're willing to pay, whereas your mortgagee prefers to have more money later and doesn't mind having less now.
And so you make the trade -- in this case, you exchange a pile of money for some obligations to make future payments adding up to somewhat more money -- and both parties consider themselves better off than before. If that isn't adding value to the world, I don't know what is.
It's not a very glamorous kind of added value: it's less exciting than inventing a new kind of machine, or shaping raw materials into beautiful sculptures. But added value it is.
(Assuming, of course, that you and the mortgagee aren't badly wrong about your preferences. If it turns out that you're going to be spectacularly unable to repay the mortgage, so that you lose your home and the mortgagee loses the money, then maybe everyone loses. See, e.g., 2008. But on the whole it seems like the existence of mortgages is a considerable net benefit.)
You're mistaking something that's sometimes necessary, or something that makes people happy, for something that creates value.
Nothing new is created when ownership changes hands, as in the case of me possessing the money to buy a house, where before I didn't.
A change of ownership, or a new contract that two people have signed, or a promise to do something, none of that adds value to the world. The value is in the doing.
Don't forget about the feedback loop... Do you think home prices would be as high as they are without the existence and wide availability of mortgages?
>This is huge; this is what prevented the financial crisis
Hey, I hate the anti-bank sentiment as much as anybody, but you do realize that every asset they sold by Goldman required a buyer, right? So, it prevented nothing, just moved the risk onto someone other institution's balance sheet.
Good friend of mine was finalist around the early '90s. He know has a totally simple 'senior dev' job at one of the big SV company's service centre. One of the things that keep reminding me that success depends on so many things, hard math skills are one of them.
Unfortunately, I'm a guy with a very bad impostor sindrome, and a) whenever I see guys like him in worse/intellectually less challeging jobs than me I think I must be just lucky b) whenever I see guys like him I think I must be useless. :)
It could be that you're lucky or maybe you're just suffering from impostor syndrome.
Have you considered that he might be perfectly happy where he is as he can basically relax at his job as most of the tasks that come up , even the difficult ones must be a cakewalk for him.
Sometimes these people tire of being constantly intellectually stimulated, or they take a different view on life than the purist view of the pursuit of long term development of humanity through research.
Myself, while I was not an Olympian in Math or Physics (the US fields some very competitive teams full of people who had better resources than me), I consider myself accomplished for what I had & the environment I grew in. I ultimately abandoned academia, in part because of the uncertainty, difficult work, and because my caring for friends won out - attempting to become wealthy enough to support friends in their endeavors, and giving them the resources I never had mattered more to me. I am perfectly happy being a lead developer, mentoring people to success & building products that help people's lives. I have no regrets.
It should also be mentioned that being someone who is supremely accomplished in competitions does not mean that the person is the smartest - there may be others smarter than him/her, and just did not have the fortune of growing up in an environment that could best foster that intelligence. Many of the smartest realize this, and don't hang their hats on any particular competition achievement.
Does he consider himself a failure? Some percentage of very intelligent people just don't care about "success". They have rich inner lives and hobbies and interesting friends, if they can find some. It could be that he's found a job that doesn’t cause him much stress, that he excels at compared to his colleagues, and allows him to pay for the pleasures of being alive. Much of pure math is the pursuit of difficult problems that may or may not be of any use to anyone. You ever hear this line: "Winning the Fields medal tells you two things about a person, that they were capable of achieving great things and that they didn't."
It's not clear to me that he would have contributed more to the world had he become some high-status academic.
Different goals lead to different outcomes, regardless of innate ability.
One always has to look at the whole person to understand whether he performs relatively well or not. The guy I was talking about is extremely risk averse. That makes steep careers quite difficult. But all in all, he's probably a happy person. Another example: MENSA says I'm way inside the top 1% when it comes to IQ tests (I didn't want to say 'intelligence'). I don't think I'm in the top 1% most successful people in my current environment, but that's all right as I'm coming from a poor village from a postcommunist country, and everything that comes with it (lack of good education, etc.). So compared to that, I'm doing all right, but naturally it's very difficult to catch up with people who could go to Stanford or Oxford. Not because it's difficult to learn the 'hard science stuff' -- I interviewed many Oxbridge guys and they're not that better than the others when it comes to hard tech/science knowledge. However, they're much better in social skills, self-confidence, usually have better network, family connections, etc. And those things matter a lot.
And I'm a big believer of luck and randomness. Since I've got a kid I wonder how do we even survive, life is so risky, starting from the birth itself.
Anyway, I think perspective gives you more happiness. At least it did for me.
>You ever hear this line: "Winning the Fields medal tells you two things about a person, that they were capable of achieving great things and that they didn't."
Would you mind explaining this a little bit more, please? I would imagine that significant or groundbreaking mathematical discoveries qualify as "great things" by someone's reckoning.
But not by the reckoning of whoever coined that phrase.
Most work in pure mathematics -- even outstandingly brilliant work done by outstandingly brilliant people -- is never useful outside pure mathematics. If you regard pure mathematics as very valuable in its own right, that's no problem. But if you care mostly about practical benefits, you might think that someone with the mental characteristics needed to win a Fields Medal would have done better to use that wonderful brain for something that benefits the world more.
(For my part, I'm glad that some people with wonderful brains use them to enrich pure mathematics, because I happen to love pure mathematics. But I think those more practically minded people have a point.)
>Most work in pure mathematics -- even outstandingly brilliant work done by outstandingly brilliant people -- is never useful outside pure mathematics.
Perhaps this is untrue on a timescale of decades or centuries.
Applications mine from pure math all the time. See this SO answer for a wide range of other fields' problems that were solved with pure math. I see things from cryptography, graphics, distributed computing, chemistry, ...
http://math.stackexchange.com/questions/280530/can-you-provi...
It's hard for me to understand the ratio of the relative body of work that turns out to be useful versus the relative body of work that hasn't seen application yet. Across all institutions, pure mathematics is an extremely small field; yet their papers have an (almost) infinite amount of time available to find an application.
success doesn't come on its own, one needs to put some aimed effort into it, or rely on luck.
I can say exactly the same about me, sometimes feeling like the biggest idiot around, with serious brain damage (really, not joking). But then I realize that I managed when working 100% to increase my paycheck 20x compared to my first Java 100% job right after school, doing roughly same job (it's a long story). Many kids being brighter than me, they now have "meh" jobs at best. For success, initial talent is just one of many properties.
Anyway, what we should strive for is happiness, with oneself, with job, people and world around us. Rest are just details.
I increased my paycheck by >3 times in the last five years. I can't talk about saiya-jin's 20x, but I assume it's partly to do with starting on a stupidly low salary in the first place, and then going to something above average.
For the criterion of the title,
i.e., "where are they now?", from
the article and more, sadly, it
sounds like a Math Olympiad
is not a very good way to do well
on the criterion.
Why not very good? Because it looks
like the students are given what, for
the criterion, is poor direction
and use of time and effort.
E.g., when I look at my education
in math and its use in applications,
most of the best of the education
had nothing to do with anything
like a contest, especially a
contest in the early and mid teens.
Instead, the best education was
well selected, well presented
material into the best work in
math, e.g., via the best authors --
Birkhoff, Halmos, Rudin, Royden,
Neveu, Tukey, Kelley, Suppes,
Simmons, etc.
Or, my impression of encouraging
kids in their early and mid teens
to pursue math is to
give them some enrichment
material such as Pascal's triangle,
some number theory, various
puzzle problems, a lot of
recreational math, etc. Why?
Because mostly the people
directing the efforts and
selecting the materials are
not very well educated in math.
In simple terms, to have kids
do well on the criterion in the
title is just to have the kids
proceed along the main line --
get through the standard
high school material in algebra
and geometry, with some applications
to high school chemistry and physics,
and then move on to a fairly
standard college major in math --
calculus, modern algebra,
linear algebra,
advanced calculus,
ordinary differential equations,
advanced calculus for applications,
partial differential equations,
point set topology,
measure theory,
functional analysis,
probability, statistics,
stochastic processes,
etc.
Then that background
promises to be good for a good
answer 30 years later for
the criterion "where are they now"?
It seems to me like plain regression effect. Also your argument makes no sense because most or virtually all of those people had all that standard math curriculum too. You'll find that "is a professor" is very well correlated with "wants to be a professor" among that set.
Edit: Pascal's triangle??? Sorry but if you give some mopper Pascal's triangle they'll have already heard about it or invented it themself.
The question was where
are the people now
who did well in Math Olympiad
in middle and high school.
Good grief: Lots of people in middle
and high school get pushed into
math competitions such as the Math
Olympiad but don't go on to be
math majors in college and grad school.
My point is that the middle and high
school Math Olympiad directions
basically don't help people be
good at math later in life or good
at anything later in life. So,
bingo, presto, people
who were good at Math Olympiad
show little or no good effect
later in life.
But if want to study some math that
has a chance of having a positive
effect later in life, then follow
what I outlined.
Net, sadly, Math Olympiad and other
middle and high school math
competitions are unpromising
for doing any good later in life.
Such middle and high school efforts
would, could, and should be
helpful but not by pursuing
recreational math, etc.
Seems totally obvious to me --
sorry you don't agree.
You proposed enrichment materials such as number theory puzzles and other recreational mathematics. That's already what the math olympiad contests consist of.
> Good grief: Lots of people in middle and high school get pushed into math competitions such as the Math Olympiad but don't go on to be math majors in college and grad school.
This isn't true at all. Most people in high end math contests are just smart and do no preparation. They just accidentally scored high on the AMC. A lot do some practice for fun, because the contests are fun. Possible exceptions are a handful of people at Philips Exeter++ and maybe Thomas Jefferson High School, I'm not familiar with that dynamic. Contests like the ARML are oriented around making local friends more than anything. At the most local levels it's like, some high school teachers will corral their students into doing a contest at the nearby college one afternoon.
For example some friends of mine in the Philadelphia area thought it would be neat to enter the Harvard/MIT math contest (which is way more fun and difficult than the Temple School of Actuarial Sciences contest or Drexel's). It was the kids telling their parents they were doing this, and deciding to practice for it. I think the same happened with a group of students in Albany.
++and they were recruited after performing well on middle and elementary school math contests that parents are completely unaware of.
> You proposed enrichment materials such as number theory puzzles and other recreational mathematics. That's already what the math olympiad contests consist of.
No, I did just the opposite: I said that
others proposed such materials;
I did not propose them but criticized such
proposals. Again, for a high school
student to be pushed into seeing
lots of tricky things in Pascal's
triangle, attacking number theory
exercises, doing recreational math
won't do the students much good in
later life -- so as in the title
of this thread, we won't expect to
see many such students successful in math
later in life. Instead, the efforts
there in middle and high school will
rarely come to anything.
Helping middle and high school students
do better in math than what is
in just the usual courses would, could,
and should be doable and done, but
the path is not recreational math, etc.
but, instead, as I outlined, just
proceeding with the main line of
math education -- rush through the
high school math of algebra and
geometry, likely also trig,
and then get a college calculus
book and dig in.
E.g., the high school math
teachers I had were eager to
say that we "were not ready for
calculus". Nonsense. It was
the teachers who didn't know
calculus.
I very much would
have rushed ahead had
I had some good guidance.
Eventually I knew enough to rush ahead
when I was
a college freshman: I'd had
four years of math, grades 9-12,
at a relatively good high school
but for my college freshman year
went to a state school that was
cheap and close enough for me to walk
there and back. They wouldn't let
me take calculus but pushed me into
a course beneath what I'd already done
in high school. So a girlfriend told
me when the tests were, and I showed
up for those. The teacher said I
was the best math student he'd ever had --
in no very real sense was I his student!
I asked to be permitted to start calculus
but was turned down. So I got a good
book and dug in. I did well (in high
school mostly I'd been teaching myself
from the books and sleeping in class anyway).
For my sophomore year I went to a good
college with an excellent college math
department, started on their sophomore
calculus, from the same text used at
Harvard, and did well.
I'm hot on this stuff about what
middle and high schools do to
good math students: My eighth grade
arithmetic teacher gave me a D and
took me aside one on one and fervently
urged me never to take anymore math.
In some significant ways,
my father was good at education
and just laughed.
That eighth grade teacher
didn't have a clue about what math was:
For the actual math, I'd done well
in her class. What I did poorly on was
multiplying two four digit numbers, and
the reasons were (1) my handwriting
was sloppy (common for boys), so sloppy
I didn't get the columns lined up and
then added wrong for the final answers,
(2) my clerical ability was just awful
(common for boys), (3) I understood the
ideas right away but never worked
on the clerical skills to
get correct answers, (4) my parents
never urged me to work to get right
answers (my father had some
high-end views of education
and just wanted me to learn,
especially out of interest,
and didn't care at all about grades,
and I did learn).
But in the ninth grade,
the teacher saw right away that
I was his best student, and he sent
me to the state math tournament.
In the summer after the 11th grade
I was sent to an NSF math and physics
program.
So, net, I was a good math student,
was very interested in math, was
eager to race ahead, but the school
did a poor job helping me.
There is a little more to the
story: When the SATs came back,
I was called out of study hall
to get my scores, from the same
teacher I'd had in the sixth grade.
Nice woman. Sweet. Ignorant!
She, along with most of the rest of
the teachers, thought I was
a dunce. So, she read me
the Verbal SAT score -- 538.
She said "Very good". BS!
It ...
> I didn't mention any activities at the college level.
Instead, you wrote:
> Harvard/MIT math contest (which is way more fun and difficult than the Temple School of Actuarial Sciences contest or Drexel's)
Sounds like college.
> It's clear you have no idea what you're taking about.
What's now fully clear is just that
you are angry about something
and want to attack me personally,
not my writing or thinking but just
me, personally. Not good.
Nope, not college. You don't know anything about high school math contests or the motivations of students that do them, or anything about that world at all, and amid irrelevant stories about your life you're just pretending that you do.
To clarify: I think you don't know what kinds of problems they ask, what their contest format is like, what students learn and practice when preparing for them, how it affects students' interest in research mathematics and how it develops their abilities there. Why do I think that? Because instead of talking about the USAMO or ARML or the NSA's mail-in contest or anything specific about whether these contests actually harm or distract students' attention from what you think they should be doing, instead of rebutting the well-known "calculus trap" opinion or showing that you have any awareness of it, instead of showing knowledge of what students do to prepare for these contests, you've talked in abstractions, and there is no glue tying the words you are saying to objective reality.
Why are you acting as if these contests do a poor job training kids? The OP never suggested that -- in fact, it seems nearly all of them grew up to be very successful. Your stance seems to be in direct contradiction of the evidence offered in the original article.
The article implied
that the Math Olympiad
work didn't much result
in significant results 30
years later
from the work in math for
the contest.
You are correct that the OP
didn't directly say that the
"training" of the "kids"
was poor.
My view of such high school
math competitions is that
they waste the time and
effort of the students
and, otherwise, with good
guidance, there would be
significant results
from the efforts 30 years later.
In this sense, then the
training was a "poor job".
But the OP did not say that
directly -- that's my
interpretation, heavily
from what I saw used
as efforts to have some
high school students
do better in math --
the efforts were a waste
of time for all concerned
and because the people
directing the efforts
didn't know enough math.
I'm still not following. In what way are there not significant results from the kids in the contest? They almost all seemed to have had significant success.
My view of efforts in grades 8-12 or so
to have students do well in math competitions,
e.g., the Math Olympiad, is that those
efforts are not promising
for helping those students do well
with math or anything else later in life.
Instead, mostly the efforts are just wasteful
for all concerned.
The main reason for the waste is that the
people, usually the high school math teachers,
running the programs don't know enough
math to give good direction to the students
that might want to do especially well in math,
well enough that 30 years later we would
commonly see significant positive effects
from that direction and the resulting
efforts of the students.
So, the direction that is commonly given
is to have the students study
relatively useless material,
say, recreational math, some
puzzle problems, every tricky
thing they can see in Pascal's
triangle (the binomial coefficients,
that is, the coefficients of
(x + y)^n with one row for each
value of positive integer n), and there are a lot
of such tricky things.
But, right,
the binomial coefficients have some
nice properties, some of which are
good to know, and an early volume
of D. Knuth's The Art of Computer
Programming has some good coverage.
But, net, mostly to heck with the
triangle and, if insist, then read
the relevant sections of Knuth --
but even there Knuth has a good reason
for presenting that material, and what's
important is the applications Knuth makes,
so, really, just read on in Knuth.
So, instead, students who do want to
do better in math, and I did,
should just proceed along the usual
topics, texts, etc. for a ugrad math
major. In particular, as a first goal,
ASAP get a
good college calculus text, get
the basic prerequisites, and dig in.
By the way, the time I looked
at the AP calculus materials
I didn't like them and
concluded that the people
who wrote the materials
didn't understand calculus
very well. Instead, just get
a good college calculus text.
For the rest I mentioned, about have
to have a relatively applied
ugrad math major to
understand: So, would need to know
some linear programming, network flows,
optimization, integer programming,
something about the question of P versus
NP, and more.
For Lagrangian relaxation,
that's a bit tricky and specialized
and not commonly well known even among
college math profs. I did give an
outline in
but it's not really easy reading.
But it's a cute topic with some
a good intuitive views, a bit
amazing in some ways,
and at times shockingly powerful.
E.g., the guy who first did the
many worlds interpretation of
quantum mechanics, a Ph.D.
physics student
of J. Wheeler at Princeton,
an expert in relativity theory,
was H. Everett, and at one time
he had a nice applied math
shop near DC doing resource
allocation for the US DoD
using a first, simple version
of Lagrangian relaxation.
What I outlined in
is more general and powerful
and makes some cute use of
some convexity -- that there is
convexity there is also surprising
but true and powerful and, with
the right setup, easy to prove.
I have to admit, when I started reading this I thought the places they'd be would be a lot different. Not sure why I had such weirdly high expectations.
Sounds like they're in pretty prestigious positions to me. Lots of professors publishing papers, and a couple working for the biggest name in finance; only one ordinary-sounding job ("web developer in Boston"). What were you expecting?
If anything I was surprised so many of them had done so well. I was on the UK squad for a few years and most of my friends from then seem to have ordinary jobs at companies you've never heard of (like me).
I felt exactly the same. I think its the illusion of the infinite promise of youth and subsequent disappointment. That and it reminds me of my own mortality.
If someone has a natural talent that they've developed into a very useful skill (mathematics, programming, selling vacuum cleaners, whatever) and they use that to get a cush 9 to 5 job, all the more power too them. Let them spend less time on politics, and more time on family, friends and hobbies. Who are we to say it was a bad decision to let someone else write the next great math paper?
If anything, this highlights that an aptitude for theoretical mathematics can be a launching point to a lot of different things, and that's refreshing. (I'd be less likely to encourage kids in math if Professorships were the only outcome)
There's a huge "Shit Happens" factor. One bad decision or major health problem, or just a loss of interest, and you're on a different path and it's hard as hell to fight your way back. Take that out 20 years, and you get the high fall-out rate. It's not that a lot of people fall off "the good track" for one reason; it's a bunch of reasons that affect small numbers of people.
Success is also hard to measure at an individual level. If you're 35 and making $130k as a full professor of mathematics, you're a star. If you're a consultant making $130k working specifically on what you want to work on, you're a success. If you're still a non-shot-caller making $130k at a Silicon Valley company, at 35, then you've failed. We don't really know how people are doing, though. The "web developer in Boston" might be able to pick his projects, or making a lot of money.
When I told John Tate I was thinking of switching out of mathematics, his response was in effect "Good heavens! If there's anything else you can imagine doing, don't stay in math!" I later read that Wittgenstein had famously said something similar about philosophy. (Context: I was 19 years old and finishing up my PhD in math at Harvard. In that small department, Tate was not one of the professors I knew well.)
In fact, I often feel pangs about not staying in academia. On the other hand, I've been able to carve out a life with a big dose of public intellectual -- for want of a better or less pretentious term -- so that's cool too. E.g., I'm repeatedly told that my blog is "required reading" for grad students in the relevant areas of computer science ...
> "Good heavens! If there's anything else you can imagine doing, don't stay in math"
I heard similar advice from people in many different lines of work including laws, finance, medicine, academia... usually they add it's not as interesting as it used to be.
It's a matter of compromise. I wish I could have the salary of banker, the freedom of a tenured professor and the prestige of a doctor.
Well, one disadvantage of academia is that you don't really get to choose where to live, even if your CV is quite good. Indeed, there are a number of academic couples that make crazy long-distance commuter marriages work because they weren't able to get jobs at the same institution (or in the same city).
It didn't used to be that way. The academic job market melted down in the 1980s and now commuter marriages are the norm. These days, there are too many PhDs and few full-time faculty positions. It's horrible, but academia doesn't have to be that way, and wasn't always.
At least at one time and maybe still,
there was a decently good living in
applied math within 100 miles of the
Washington Monument in DC.
There early in my career, once I sent
some resumes and in two weeks went
on seven interviews and got five offers.
Big topics were scientific engineering
programming, numerical analysis, especially
numerical linear algebra, standard
applied statistics -- hypothesis testing,
linear statistics, analysis of variance,
curve fitting, etc. --
the fast Fourier transform, digital
filtering, power spectral estimation,
deterministic optimal
control theory, linear and non-linear
programming, discrete and continuous
time stochastic processes, Monte-Carlo,
numerical solutions of ordinary and
partial differential equations, etc.
Some of these topics are part of
mathematical EE.
At one time I my annual salary was
six times what a new, high end
Camaro cost. My wife got to
pursue her Ph.D. in essentially
mathematical sociology as part
of her desire to save the world
while we had big times at
Thanksgiving and Christmas,
shopped in Georgetown for high
end French wine and cheese,
were regulars with good tables
at high end restaurants,
went to lots of concerts
and plays, got a good piano
for her and a good violin
for me,
got boxes of fancy French
pastries and sweet, sparkling
dessert wines and
on Saturday nights
pigged out watching old movies,
went to Shenandoah for
vacations, had plenty of
time for trips to families
in Tennessee and Indiana,
did some fancy cooking at home,
etc.
Of course the money was coming almost
entirely through the US Federal
Government, mostly for US national
security, e.g., via the US Navy,
DARPA, etc. But, still, there was
money via the DoE, NIST, and more.
I went ahead and got an applied
math Ph.D. at Johns Hopkins,
research on stochastic optimal control.
My startup has at its core some
applied math I derived based on
some advanced prerequisites I got mostly
at Hopkins. The math is an advantage.
If the startup works, then the math
will have been the key -- except for
the math and the basic business
idea, it's all routine, that is,
the math is the only thing really
special.
Math can be used to say how to
take data that is available and
manipulate it to get data that
is valuable. Now there is a lot
of data and means to communicate,
store, process, and display it,
infrastructure software to make
the rest much easier,
ads to run to get revenue,
etc.
Except possibly for coding theory
and cryptography,
I don't know if there are valuable
applications of algebraic topology,
algebraic geometry, algebraic
number theory, commutative algebra,
etc., but for the topics I mentioned
it would seem that there is some
practical value.
For some of what can be done
with applications, there is
a long dessert buffet
from the Hahn-Banach
theorem and more in
D. Luenberger, Optimization
by Vector Space Methods.
One heck of a toolkit.
Make just one good application
and retire, make a donation
to a university, and get your
name on a building.
Here's another possibility:
Attack integer linear programming
problems in business. Focus
on the problems that can
attack with min cost flows
on networks with integer arc
capacities. Why? Because it is
just linear programming;
the simplex algorithm becomes
really simple in that case;
in practice
the simplex algorithm is
fast beyond belief on problems
large beyond belief;
and get integer programming
for no extra effort --
if start with an integer basic
feasible solution, then the
simplex algorithm will maintain
an integer solution and,
if there is an optimal
solution then will get
an integer optimal solution.
Can use this in various
resource allocation problems.
For more, use it as a linear
approximation in nonlinear
problems and for still more
enhance it with some
Lagrangian relaxation.
Show that P = NP? Nope.
Make money? Maybe.
I'm a third year maths student at the moment and having a bit of an existential career choice crisis. I don't know if academia is for me, because I don't think it is worth it unless you are top tier, and having done an internship at a big bank neither does finance, mainly due to the stress. What kind of resources or subject areas do you think are valuable to study while I will have time? How lucrative is the industry in its current state and how good current employment opportunities are?
I've basically done half a cs major already. My part time job is some stats programming work for a mining consultancy company and I was a strategist intern at Goldman over the summer so that is very good to hear!
At one time I wrote Fisher Black
at GS and got a nice letter back
from him, I still have, saying
that he saw no opportunities for
applied math on Wall Street!
I didn't hear about James Simons
until much later.
For academics, for one possibility, look into
B-schools: At the research
universities, the B-school research requirement
seems to be to publish nearly anything.
Some of the journals popular in B-school
publishing have not very high standards for
quality.
The B-schools are basically
forced by some standards efforts
to teach some courses that
are essentially applied math.
One might guess that the B-schools
want to be professional or clinical
but, instead, they tend to want to
see themselves as applied social
science, and social science has
physics envy and wants to be
mathematical. Their favorite math
is linear statistics. So typically
there's a lot of such statistics
in the research at B-schools. And
typically the B-school profs are not
very good at math or statistics.
B-schools also want to dabble in
computing -- if you can teach some
courses in computing, even as electives,
then you might get liked for that.
No doubt some B-schools would like
someone who could teach courses
in financial engineering, say,
with Brownian motion, stochastic
integration, Black-Scholes
and generalizations -- basically
the Brownian motion solution to the
Dirichlet problem. So, if you
like mathematical finance, then
maybe teach it in a B-school.
DC may still be a good area for applied
math: So, submit a Civil Service application
and also send copies to the usual
suspects -- NIST, various DoE labs,
various DoD labs, various groups
interested in economics or mathematical
finance, that is, maybe the Federal
Reserve (also try the regional
Fed banks), the Social Security
Administration, etc. If you want,
try NSA, CIA, DIA, FBI, etc. -- no telling
what they might want to do with some
applied math. Also try the JHU/APL.
There have long been lots of
Beltway Bandit shops because
commonly Congress gives the
departments of the Federal Government
more money to spend than full time
head count to spend it on so that
a lot of the money and/or work goes through
companies. E.g., Edward Snowden
worked for Booz-Allen or some such.
For technical jobs around DC,
it used to be that the job ads in
The Washington Post were good
places to look.
There is a chance that Google, Facebook,
Yahoo and some others are interested
in some projects that might
be able to use some high end applied math.
A problem may be that the other workers there are
not very good at math and, really,
make a mess out of the work, the
opportunities, the positions, etc.,
and that can be bad for everyone.
It's not clear that everything
in mathematical finance is
a rat race. I doubt that James
Simons ran a rat race shop.
Houston has some important slots for
optimization, e.g., as in the work
of Princeton Chem Eng prof Floudas.
E.g., here's all the crude oil
inputs we have available and the costs,
and here's all the prices for
all the possible refinery products.
So, say what crude oil to buy and what
products to sell to maximize
earnings. That's an old problem,
but likely people are still working
on it.
The airlines have some very serious
problems in fleet and crew scheduling,
and first cut it's integer linear
programming. But with uncertainties,
if you want to handle them at all realistically,
the formulations and solutions can become
challenging in all respects.
A better solution can show
its value just on the computer
and get taken seriously. For
some airline spending $100
million a month just on jet fuel,
saving a few percent can pay for
some nice computing, work, etc.
Maybe look at what the operations
research people are doing these
days in scheduling for trucks,
airplanes, cargo ships, etc.
No doubt Amazon, Wal-Mart, etc.
have some good problems
that are generalization of
the old transportation
problem -- the one Kantorovich
got a Nobel prize for.
Now we regard that problem
as least cost flows on a network
and use the simplex algorithm
to solve it and where a basic
feasible solution corresponds to
a spanning tree of arcs on the network.
FedEx had candidate investor General
Dynamics (GD), and GD had sent two of their
guys, one aero engineer and one finance guy,
to help FedEx. FedEx needed the
GD investment.
At one point, the Board wanted some
revenue projections. I wasn't asked
to get involved and didn't want to,
but no one had any ideas
of how to do
the projections except just
draw a free hand line on a graph
based on hopes, dreams, intentions,
guesses, etc.
So, I thought: We know (1) the current
revenue. We know (2) the revenue
when the planned service to 90 US
cities with 33 airplanes is full.
So, the projections are roughly
how interpolate between (1) and (2).
For how the interpolation will go,
we have (3) current customers and
(4) the rest of the customers we
will have when we have (2) and are full.
Then say that the growth is
from current customers (3) talking
to the rest of the customers (4).
That is, the rate of growth is
directly proportional to the
number of (happy) customers (3)
talking to the rest of the
customers we will have (4).
So, let t denote time in days
with t = 0 corresponding to the
present. Let the revenue
at time t be y(t).
Let the revenue from (2), being full, be
b dollars a day. Then at time t,
the growth rate is y'(t),
that is, d/dt y(t). So,
we have that
y'(t) = k y(t) ( b - y(t) )
How 'bout that! Maybe it's
so simple it's almost just
a joke, but the rest of the
ideas floating around the office
are much worse!
So, sure, it's an initial value
problem for a first order,
linear ordinary differential
equation but, really, is so
simple can just use freshman
calculus directly and find the
closed form solution, right,
some exponentials I omit here.
So, on a Friday the Senior Vice
President (SVP) for Planning and I
got out some graph paper,
picked a value for k, and
drew the graph.
The next day, Saturday, I
was in my office, and at noon
I got a phone call asking if
I knew about the revenue projections.
Well, the SVP was traveling,
and, yes, I knew. So, I was
asked if I could come over to
the HQ offices and explain.
So, I got in my Camaro hot rod
and drove over.
When I got there, people
were unhappy. The two
GD guys were standing in
the hall with their bags
packed.
I was led to the graph,
given a time, and asked
to reproduce the value on the
graph at that time. So,
I had brought my HP scientific
calculator, punched the buttons,
and got the value on the graph.
I did that a few more times,
and people started to be happy
again.
It turned out that the Board meeting
had been that Saturday morning;
the graph had been presented;
and the two GD guys had asked how
it had been calculated.
Well, all the FedEx people there
tried to figure out how the
graph had been calculated and
went on for a few hours
without success, and
then GD guys lost patience,
got plane reservations back to
Texas, went to their rented
rooms and packed their bags,
and as a last chance returned
to the HQ offices to see if
FedEx had an explanation for the
graph. FedEx was about to die.
That's when I arrived and
explained the graph.
The FedEx guys stayed, and FedEx
was saved.
I never got thanked! Really
my impression was that the rest
of the top of FedEx had
concluded that I'd been
dangerous, maybe had 'hooked'
the company.
And I'd never been invited to the
Board meeting. So, I began to
conclude that my contribution
was so resented that the top
of FedEx would rather see
all of FedEx go under than
let me help.
The promised FedEx stock was
already 18 months late.
I was in Memphis while my
wife was in her Ph.D. program
at Johns Hopkins
near our home in Maryland.
So, to heck with FedEx --
I went home and
to Hopkins for my Ph.D.
So, that's an example of
applied math in US business!
E.g., I was the only one around
who still knew freshman calculus!
And, heck, I'd never even taken
freshman calculus, had studied it
on my own and started with
sophomore calc...
What kind of companies would be good to get practical experience in this area? In my experience the bridge between theoretically and actually apply mathematics is very hard to cross. Having started working mathematically recently it seems to me that even though I can propose a solution which isn't the best, they are usually better than what is currently available. Is that a common experience as a working mathematician.
> Having started working mathematically
recently it seems to me that even though I
can propose a solution which isn't the
best, they are usually better than what is
currently available. Is that a common
experience as a working mathematician.
Sure. For the politics, just report the
savings. The stuff about the optimal
solution, that is, its, say, moral or
ethical value, is lost on business
people. With some good common sense, the
business people just want to know how much
has been saved (or extra earned). If you
can also have, say, a lower bound on the
possible savings and get close to that,
then you have in practice a stopping
criterion. Or, what in practice is really
difficult about the NP-hard problems is
usually saving the very last possible tiny
fraction of one penny and proving that did
that. Instead, just concentrate on
saving, say, the first $5 million a month
on, say, the $100 million a month of jet
fuel for an airline.
For a while in optimization, having the
optimal solution was made a moral
objective with anything that wasted even
0.0001 pennies regarded as a sin.
Religious nonsense.
Really sloppy work is not good, but close
approximation usually is plenty good.
For not the best, that's the other time I
kept FedEx from going out of business:
The founder, Fred W. Smith, spent the
afternoon in his office trying to schedule
the fleet, staggered out exhausted, said
"We need a computer", and a college friend
of mine heard that plea and called me.
Sure, digging in, I heard a lot about
optimal solutions. But I knew that
first just get something that helps, say,
so can type in a schedule, have the
software check it out for, say, planes
being over loaded or having to fly too
far, get the arrival times, etc., and
print out all the details. So, I typed in
6000 lines of code to do that. Right,
great circle distances are from just the
law of cosines for spherical triangles!
Soon the Board wanted a schedule; biggie
Board members doubted that the scheduling
could even be done; biggie funding was
being held up; the company was at risk.
So, I got a call from the same guy who
called me to the Board meeting for the
revenue projections, Roger Frock (he wrote
a book about FedEx), and he and one short,
easy evening he and I used my software to
develop a schedule for the whole planned
system of 90 cities and 33 airplanes. It
printed out and looked nice.
At the next senior staff meeting, Fred's
reaction to the printout was "Amazing
document -- solved the most important
problem facing Federal Express". Our same
two guys from General Dynamics went over
the schedule carefully and announced "It's
a little tight in a few places but it's
flyable". The Board was pleased; the
funding was enabled. That was the first
time I saved FedEx.
So, I got into how to do better
scheduling. So, for one period from, say,
5 PM to midnight (again from 1 AM to, say,
8 AM) set up a table with one row for each
city to be served, 90 cities, and one
column for each candidate airplane tour
from Memphis and back. So, may have some
thousands of columns. Considering all
fine details want to account for, and take
expectations for random costs, find for
each of the candidate tours the cost
(first cut, assume that if a plane goes to
a city, then it completely serves that
city).
Then for the schedule, want to pick no
more than 33 columns from the 1000s so
that all 90 cities are served and total
cost is minimized.
So, each column covers some cities, and
want to pick columns that cover all 90
cities -- so have a set covering
problem. Right, it's in NP-complete.
Well, that table is essentially the
matrix in a 0-1 integer linear programming
least cost optimization problem. There
have one unknown variable for each of the
columns, and the variable is 1 if use that
column and 0 otherwise. If relax the 0-1
constraint, then have a fairly routine
linear programming problem but will li...
For what companies to go to to get going
on a career in such applied math, I can't
think of any. Mostly people and their
organizations would rather waste money
than get involved with mathematicians
doing work only mathematicians understand.
Again, if want to make math pay, just sell
the results, usually as a vendor outside
the customer.
If are inside the company, then likely no
good work will go unpunished. Basically
to be successful, you will need the strong
personal support of the CEO and hopefully
the Board -- literally. And the CEO will
routinely have to go around and break 2 x
4s over the heads of recalcitrant middle
managers to see if they are just asleep or
really dead -- otherwise those middle
managers can sabotage you with high
determination and creativity.
E.g., twice I saved FedEx from going out
of business and had much more that would
have saved FedEx maybe tens of millions of
dollars a year, but I got flack and push
back and contempt and none of the promised
stock.
Such are some of the challenges in getting
stuff done. But successful people have
been overcoming challenges, some much
worse than I faced at FedEx, for many
centuries.
Now that you know about the possibilities
of some of the absurd challenges, don't be
surprised. Really, one of the best rules
is just to keep your good technical work
essentially a secret until you have a
solution that is just too darned good to
refuse, and then pull back the curtain,
let people see the results and also see
that it's too late to fight with you.
Better still, just provide such results as
a business and own the business. E.g.,
in some cases, just do the work for free,
show the results, and then announce "You
can have this for your real operations for
a fee of only 15% of the savings."
Once there was an in-house research group
that was successful. It was for WestVaco
paper -- think specialty papers, e.g.,
coated papers for milk cartons. So, they
had Ph.D. chem eng guys running their
plants in jungles, etc., that is, where
the trees were, but it was a family owned
business with HQ in NYC.
The research shop was in Maryland, and I
got an explanation from them: Each year
Research went to NYC for their budget.
Always the CEO, family guy, offered much
more money for the Research shop than the
shop was willing to accept (first-cut,
good budget situation!).
Why? The rules: Research picks their own
projects. Projects tend to run from a few
months to a few years. One project in 10
gets to operations. When Research has a
project ready for operations, they
approach the relevant operational
managers. In the first three years,
savings are allocated half to Research and
half to the operational unit. After three
years, all the savings are allocated to
the operational unit. With this
accounting, Research was returning $3 to
the company for each $1 in their budget.
So, sure, HQ wanted to increase the
budget, but Research didn't want to have
to find ways to return $3 for each extra
budget $1 so didn't want a bigger budget!
It is just crucial that the CEO is
supporting such in-house innovation. The
rules are also important.
Ah, while I was in grad school, to make
enough money to support my wife and I
while we both finished our Ph.D.s, I took
a job at a DC area think tank working
for the US Navy.
At one point our problem sponsor group
wanted an analysis of a special, worrisome
scenario of global nuclear war but
limited to sea -- they wanted to know how
long the US SSBN fleet could survive if it
was just held in reserve and didn't shoot
its missiles with nuke warheads. And they
wanted the results in two weeks.
Gee, two weeks to model global nuclear war
limited to sea! How generous! Good that
they were not in a real hurry! Besides it
was good timing since the day after the
due date my wife had already gotten us
reservations for a vacation at Shenandoah!
Well, there was a guy B. Koopman who in
WWII had written a report OEG-56 on
finding things at sea. So, gi...
Thanks. I really appreciate you taking the time to write so much. Honestly, this seems to be the best description of potential careers that I have ever gotten and I feel so much more assured about my choice to study Maths now. It is also really interesting to hear about how colourful such a career could be! You sound like, despite the politics, you have had a really interesting career and have done the kind of things that I would exactly like to do!
Again, thanks so much. One last question, what are you doing with your life right now? It seems like you have had a pretty broad career and I am interested in the end game. If you are retired, what is your lifestyle like and in particular how has your career impacted your life as a retiree.
For now, sure, I know some math and can
stir up some more, and I know some
computing and can learn more.
So, I've got a 1.8 GHz AMD single core
processor, Windows XP Professional SP3 (I
have an official copy of Windows 7
Professional on DVD but have seen no great
reason to go to all the trouble to install
it and rebuild all my software
environment yet), three hard disks, 100
million files, .NET Framework 4.0, a good
text editor (KEdit), Visual Basic .NET
(comes with the .NET Framework), ASP.NET,
ADO.NET, IIS (low level Web server), SQL
Server Express, etc.
I thought, why not go after about 2/3rds
of Internet search, the safe for work
part served at best poorly by looking for
keywords/phrases?
So, how to do that? Not with just routine
software! And not with anything commonly
talked about for search!
I stirred up some math, typed the theorems
and proofs into Knuth's TeX, worked up a
scalable architecture, and typed in the
software.
Maybe I should have used Redis, but
instead I thought that writing my own
session state server would be faster than
even understanding Redis. So, my session
state server is single threaded ("Look,
Ma, no concurrency problems"), for faster
lookup rates is trivial to run as several
instances as in sharding, and is just
some TCP/IP sockets, some de/serialization
of instances of my session state class,
and uses two instances of a .NET
collection class. Simple.
It's fast! A server for less than $1500
should be able to keep session state for
an hour of inactivity for each user and do
the session state work for sending 5000+
Web pages a second. Two standard racks
should be able to handle session state for
the world.
The rest of the software is also readily
scalable also from just simple sharding.
Currently I have one bug in one Web page
-- it's not handling session state just
right! But I have the fix in code in
another Web page and should copy it over
today!
My interactive development environment
is just KEdit with about 100 macros and
some careful use of file system
directories.
I'm using SQL Server only to record the
data from the users and for the results of
some batch computations; at one point I
actually do make use of a transaction;
the data for the searches is drawn from
SQL Server with a batch program (run it
maybe once a day); some solid state drives
(write rarely, read thousands of times a
second) should do wonders for the data for
the searches.
I was about to fix the bug in the Web page
but took some opportunities to gather some
good initial data. The site will start
focused and only slowly grow to be
comprehensive -- right, at first do some
things that don't scale and please some
niche group of users a lot instead of
trying to please 2+ billion users a
little.
Currently the database has only some
meaningless data I put in for first
testing of the software. It's about time
to load in some of the good initial data I
have. Then give a critical review, go
live, etc.
It's getting there.
All the work uniquely mine has been fast,
fun, and easy, but the whole project has
taken far, far, far too long. Why? I
worked through about a cubic foot of books
and 6000+ Web pages of documentation of
Windows, .NET, Visual Basic .NET, ASP.NET,
ADO.NET, SQL Server, etc.
The main problems: (1) Badly written,
obscure documentation (worst bottleneck in
the future of computing); (2) computer
viruses; (3) SQL Server installation bugs
destroying my boot partition requiring
rebuilding starting with the XP DVD
(barbed wire enema with an unanesthetized
upper molar root canal procedure); (4) SQL
Server management and administration
(e.g., a week of throwing stuff against a
wall to see what sticks just to get a SQL
Server connection string that will let
code for a server side Web page connect
with SQL Server); (5) clean, smooth means
of system backup and recovery (including
for both user data and bootable
partitions); (6) Sony DVD d...
Writing this as a reply rather than an email because there's no contact info on graycat's profile.
Hi graycat,
I'm an applied math & CS undergrad at Caltech and I love all of your posts, especially the ones about how data center scale computing really needs to embrace statistics. The FedEx stories are also excellent. I'm personally interested in high performance computing and machine learning, but I'm also interested in solving "real problems" and like how you seem to focus on the actual value of the applications. I love the feeling of engineering solutions to mathematical problems, and this seems to be something that you also enjoy.
I'd really like to hear more about your career, research, and also the startup that you're working on. I'd be very happy if you shoot me an email at eric@ericmart.in
I really enjoy your posts and your writing style. Thanks much. I am currently working on some math problems as a part of a business that makes accessible to Indians who do not have access to structured banking services. Would love to discuss it with you, if you are inclined. My email is takenottie at google's mail. Thanks.
Well, in part we aimed too high: My wife
was brilliant, quite broadly -- yes,
Valedictorian, Summa Cum Laude, Woodrow
Wilson, PBK, piano, clarinet, voice,
prizes in cooking, sewing, raising
chickens. But her family had her try to
be perfect and to dedicate herself to
saving the world.
She wanted a Ph.D. in mathematical
sociology to do social engineering of
social change to save the world and,
thus, get her praise, acceptance, and
emotional and financial security. And
those concerns filled her plate.
Well, at the roots there some anxiety,
from nature and/or nurture, was involved,
then some perfectionism and some fear of
criticism from powerful, influential
people -- net, call it a special case of
social phobia. That brought stress
which can bring depression. That slows
the work and in her case caused more
stress and more depression then clinical
depression. She was in a clinical
depression the day she got her Ph.D. She
never recovered and, net, didn't make it.
Took me a while to reinvent and learn the
basics of clinical psychology to
understand what was going on and what to
do about it. I did learn but nearly
always too late.
In trying to have a stable job so that I
could take care of her, I took a job at
IBM's Watson lab, in what we called
artificial intelligence (AI) to do
monitoring and management of large server
farms and networks.
Then IBM got sick and the lab phone book
went from 4500 full time names down to
1000 or so. The guy who hired me, a big
star who was deliberately ignored by the
higher ups, left for greener pastures --
eventually ended up with a nice place in
Malibu.
IBM Watson Research was run by a clique
of people who stuck together and blocked
out everyone else. At one point it
appeared that the company HQ in Armonk
tried to correct that situation.
Instead of or better than the AI, I had
some ideas: Detecting a problem in a
server farm or network is necessarily
essentially a statistical hypothesis test
with Type I (false alarm) and Type II
(missed detection) errors. So, want
hypothesis tests that keep down the
rates of the errors.
Since there's a lot of data readily
available and want to exploit it to keep
down the rates of the errors, also want
multi-variate inputs -- nearly all
hypothesis tests have only univariate
inputs. Also for such multi-variate data
can't hope to know any of the probability
distributions so need tests that are
distribution-free.
I don't think there were any such tests,
so I dreamed up some. I used some group
theory, summed used the classic S. Ulam,
guy on the left in
result LeCam called tightness (see P.
Billingsley, Convergence of Probability
Measures), and was able to permit
selecting false alarm rate in advance and
then getting that rate exactly. Asking
for all of the Neyman-Pearson result would
take more data than we had any chance of
having, but there was a somewhat useful
sense in which my work gave
asymptotically, for any selected false
alarm rate, the highest possible detection
rate with that false alarm rate.
I cooked up some synthetic data that was
challenging -- the critical region was
something like the red squares on a
checkerboard in several dimensions. My
work did fine.
I got some data from a cluster of
computers at Allstate, wrote some
prototype software, and confirmed the
false alarm rate empirically. And I
cooked up an algorithm to make the
computations nicely fast (in part I used
k-D trees -- reinvented those -- a few
years earlier and k-D trees would have
been mine).
Politics: A guy in the clique up there
didn't like me. But it took two levels of
management to fire someone, so he
reorganized to put me under a wuss who
would g...
> So, that's an example of applied math in US business! E.g., I was the only one around who still knew freshman calculus! And, heck, I'd never even taken freshman calculus, had studied it on my own and started with sophomore calculus!
I work for an extremely large defense contractor, and it never ceases to amaze me how many of my fellow engineers have forgotten the mathematics they were taught in high school! I know many of them took AP courses and learned calculus there, but they have completely and totally forgotten it.
Attack integer linear programming problems in business
Yep, this can be lucrative if you find a company with a genuine need for it. The company I work for has recently seized an opportunity to apply a mixed integer linear algorithm to a specific problem in the logistics industry, and I think it will soon become one of our most successful products.
I played a similar game some time ago. I looked up people from a nationwide (French) elite math program. Most people where either university professor in Math, CS or theoretical Physics, or they were working in finance. Two opposite environments it seems.
I wonder how their lives is in the financial industry. I imagine finance as a demanding and stressful environment, driven by big egos, commercial types. How do these math geniuses fit there? what level of freedom do they have in their job? in term of schedule, things they work on and so on...
Not really, or at least it doesn't have to be. Some ambitious and competitive persons may work very hard to publish more papers than their colleague. But for those that don't feel like playing this game and are immune to the comparison with more prolific researchers, it is possible to work very little in a low profile institution.
As an aside, I find it curious that so many mathematicians, programmers and others associated with logical professions be interested in music. I was relieved to see Olympiad guys from before my birth showing similar traits.
In case anyone else is interested but didn't make it all the way through the comments on the blog page, a commenter notes that Miller Puckette of Max/PD music software fame was one of their coaches a year or two later.
I participated in the IMO around 2000. Now I work as a software engineer, but I've never lost my love of mathematics. In fact, I still consider myself a mathematician, but instead of seeking inspiration from the physical world (physics) I seek inspiration from the practice of computer programming. This approach has already yielded homotopy type theory, but I think that this is the tip of the iceberg.
102 comments
[ 3.0 ms ] story [ 150 ms ] threadAre you kidding me? This is huge; this is what prevented the financial crisis, as bad as it was, from turning into something much worse (it's "bad" for a bubble to pop, but it's worse for it to keep growing and pop later). This guy made the difference between billions of dollars going into houses that people couldn't afford and weren't worth the cost, and that same money going into productive investments. He probably contributed more to humanity than the rest of the list put together, with the possible exception of the other Goldman guy.
I know everyone thinks their own field is the most important, and I love academic maths, but goddam the snobbery towards the people who did something more directly useful with their talents is irritating here.
Andrew says: March 18, 2015 at 9:41 am
Nick:
You ask why I said “That’s too bad” after learning that Gregg is a vice president at Goldman Sachs.
Perhaps the best analogy is, what if you knew someone who was one of the top high school basketball prospects in the country, but instead of ending up in the NBA he made a career as a sports hustler, using trick bets to make money off suckers. This is not a perfect analogy, as this sort of sports betting is illegal, I guess, but the point is that it’s sad to see a great talent that is being used neither in its pure form or for scientific progress or the public good.
So, if you're a talented mathematician not working in that industry... it's reasonably likely you've made a conscious decision not to work there - perhaps because you don't agree that it's directly useful to society.
There's a reason why the financial industry has to pay the highest salaries. It's surely not out of generosity.
I kind of surprised how those things are frequently mixed together. The math is a science while various industry modelling is an application of the known/established tools/skills. It is like designing a drill vs. actually drilling a hole. (note: i have an MS in Math (3.8 GPA) from one of the top Russian schools)
They are one of few who can afford to pay higher, and give all kinds of perks. Everybody wants the best of breed to work for them, don't we? On the other side, while searching for a job, the lure of neat cashflow is a strong persuator for many.
Plus there is plenty of pretty clever people in banking crowd (maybe some of them don't use their talent to max or in a way they would find fulfilling, but that's another topic).
That said, I too dislike his tone about finance. Why? Why is it bad to work in one industry over another?
Baker? Sure, making cookies adds value to the world. Software? Yeah, writing programs adds something to the world. Holding onto money, lending some of it out, and taking a cut? Not so much.
That's slowly starting to happen in the financial industry, with robo-advisors and cheap index funds taking trillions of dollars out of the hands of overpaid Wall Street firms, but that process takes time. Hopefully we'll start to see the same thing with Lyft and Sidecar eating into Uber's nonsensical valuations.
When I pay him off, slowly, over time, and somewhat enrich him, nothing of value has been added to the world.
It's counterintuitive, maybe, but any sort of trade adds value to the world even though all that's happening is that goods and/or money are being moved from one place to another. What enables this is that different people have different preferences; in the case of your mortgage, it's differences in how you value future money relative to present money -- you want money now and having less money later is a cost you're willing to pay, whereas your mortgagee prefers to have more money later and doesn't mind having less now.
And so you make the trade -- in this case, you exchange a pile of money for some obligations to make future payments adding up to somewhat more money -- and both parties consider themselves better off than before. If that isn't adding value to the world, I don't know what is.
It's not a very glamorous kind of added value: it's less exciting than inventing a new kind of machine, or shaping raw materials into beautiful sculptures. But added value it is.
(Assuming, of course, that you and the mortgagee aren't badly wrong about your preferences. If it turns out that you're going to be spectacularly unable to repay the mortgage, so that you lose your home and the mortgagee loses the money, then maybe everyone loses. See, e.g., 2008. But on the whole it seems like the existence of mortgages is a considerable net benefit.)
Nothing new is created when ownership changes hands, as in the case of me possessing the money to buy a house, where before I didn't.
A change of ownership, or a new contract that two people have signed, or a promise to do something, none of that adds value to the world. The value is in the doing.
Hey, I hate the anti-bank sentiment as much as anybody, but you do realize that every asset they sold by Goldman required a buyer, right? So, it prevented nothing, just moved the risk onto someone other institution's balance sheet.
Unfortunately, I'm a guy with a very bad impostor sindrome, and a) whenever I see guys like him in worse/intellectually less challeging jobs than me I think I must be just lucky b) whenever I see guys like him I think I must be useless. :)
Have you considered that he might be perfectly happy where he is as he can basically relax at his job as most of the tasks that come up , even the difficult ones must be a cakewalk for him.
Success has a very narrow definition these days.
Here's an interesting story : http://heuna.tumblr.com/post/2415073898/kim-ung-yong-a-child...
Myself, while I was not an Olympian in Math or Physics (the US fields some very competitive teams full of people who had better resources than me), I consider myself accomplished for what I had & the environment I grew in. I ultimately abandoned academia, in part because of the uncertainty, difficult work, and because my caring for friends won out - attempting to become wealthy enough to support friends in their endeavors, and giving them the resources I never had mattered more to me. I am perfectly happy being a lead developer, mentoring people to success & building products that help people's lives. I have no regrets.
It should also be mentioned that being someone who is supremely accomplished in competitions does not mean that the person is the smartest - there may be others smarter than him/her, and just did not have the fortune of growing up in an environment that could best foster that intelligence. Many of the smartest realize this, and don't hang their hats on any particular competition achievement.
It's not clear to me that he would have contributed more to the world had he become some high-status academic.
Different goals lead to different outcomes, regardless of innate ability.
And I'm a big believer of luck and randomness. Since I've got a kid I wonder how do we even survive, life is so risky, starting from the birth itself.
Anyway, I think perspective gives you more happiness. At least it did for me.
Indeed. Like, for example, being born with a high IQ?
Would you mind explaining this a little bit more, please? I would imagine that significant or groundbreaking mathematical discoveries qualify as "great things" by someone's reckoning.
Most work in pure mathematics -- even outstandingly brilliant work done by outstandingly brilliant people -- is never useful outside pure mathematics. If you regard pure mathematics as very valuable in its own right, that's no problem. But if you care mostly about practical benefits, you might think that someone with the mental characteristics needed to win a Fields Medal would have done better to use that wonderful brain for something that benefits the world more.
(For my part, I'm glad that some people with wonderful brains use them to enrich pure mathematics, because I happen to love pure mathematics. But I think those more practically minded people have a point.)
Perhaps this is untrue on a timescale of decades or centuries.
The list is expansive if you care to look for it.
The question that's relevant here, though, is what fraction of pure mathematics is ever useful outside pure mathematics.
I think this would be difficult to measure. My guess: only a small fraction, but some of that turns out to be incredibly useful.
I can say exactly the same about me, sometimes feeling like the biggest idiot around, with serious brain damage (really, not joking). But then I realize that I managed when working 100% to increase my paycheck 20x compared to my first Java 100% job right after school, doing roughly same job (it's a long story). Many kids being brighter than me, they now have "meh" jobs at best. For success, initial talent is just one of many properties.
Anyway, what we should strive for is happiness, with oneself, with job, people and world around us. Rest are just details.
Why not very good? Because it looks like the students are given what, for the criterion, is poor direction and use of time and effort.
E.g., when I look at my education in math and its use in applications, most of the best of the education had nothing to do with anything like a contest, especially a contest in the early and mid teens. Instead, the best education was well selected, well presented material into the best work in math, e.g., via the best authors -- Birkhoff, Halmos, Rudin, Royden, Neveu, Tukey, Kelley, Suppes, Simmons, etc.
Or, my impression of encouraging kids in their early and mid teens to pursue math is to give them some enrichment material such as Pascal's triangle, some number theory, various puzzle problems, a lot of recreational math, etc. Why? Because mostly the people directing the efforts and selecting the materials are not very well educated in math.
In simple terms, to have kids do well on the criterion in the title is just to have the kids proceed along the main line -- get through the standard high school material in algebra and geometry, with some applications to high school chemistry and physics, and then move on to a fairly standard college major in math -- calculus, modern algebra, linear algebra, advanced calculus, ordinary differential equations, advanced calculus for applications, partial differential equations, point set topology, measure theory, functional analysis, probability, statistics, stochastic processes, etc.
Then that background promises to be good for a good answer 30 years later for the criterion "where are they now"?
When I view my posts at HN, my browser flows the lines ignoring my end of line indications except does honor blank lines.
I thought that such text flowing was arranged by some internal HN server logic.
Edit: Pascal's triangle??? Sorry but if you give some mopper Pascal's triangle they'll have already heard about it or invented it themself.
Good grief: Lots of people in middle and high school get pushed into math competitions such as the Math Olympiad but don't go on to be math majors in college and grad school.
My point is that the middle and high school Math Olympiad directions basically don't help people be good at math later in life or good at anything later in life. So, bingo, presto, people who were good at Math Olympiad show little or no good effect later in life.
But if want to study some math that has a chance of having a positive effect later in life, then follow what I outlined.
Net, sadly, Math Olympiad and other middle and high school math competitions are unpromising for doing any good later in life. Such middle and high school efforts would, could, and should be helpful but not by pursuing recreational math, etc.
Seems totally obvious to me -- sorry you don't agree.
Difference of opinion is what makes horse races.
> Good grief: Lots of people in middle and high school get pushed into math competitions such as the Math Olympiad but don't go on to be math majors in college and grad school.
This isn't true at all. Most people in high end math contests are just smart and do no preparation. They just accidentally scored high on the AMC. A lot do some practice for fun, because the contests are fun. Possible exceptions are a handful of people at Philips Exeter++ and maybe Thomas Jefferson High School, I'm not familiar with that dynamic. Contests like the ARML are oriented around making local friends more than anything. At the most local levels it's like, some high school teachers will corral their students into doing a contest at the nearby college one afternoon.
For example some friends of mine in the Philadelphia area thought it would be neat to enter the Harvard/MIT math contest (which is way more fun and difficult than the Temple School of Actuarial Sciences contest or Drexel's). It was the kids telling their parents they were doing this, and deciding to practice for it. I think the same happened with a group of students in Albany.
++and they were recruited after performing well on middle and elementary school math contests that parents are completely unaware of.
No, I did just the opposite: I said that others proposed such materials; I did not propose them but criticized such proposals. Again, for a high school student to be pushed into seeing lots of tricky things in Pascal's triangle, attacking number theory exercises, doing recreational math won't do the students much good in later life -- so as in the title of this thread, we won't expect to see many such students successful in math later in life. Instead, the efforts there in middle and high school will rarely come to anything.
Helping middle and high school students do better in math than what is in just the usual courses would, could, and should be doable and done, but the path is not recreational math, etc. but, instead, as I outlined, just proceeding with the main line of math education -- rush through the high school math of algebra and geometry, likely also trig, and then get a college calculus book and dig in.
E.g., the high school math teachers I had were eager to say that we "were not ready for calculus". Nonsense. It was the teachers who didn't know calculus.
I very much would have rushed ahead had I had some good guidance. Eventually I knew enough to rush ahead when I was a college freshman: I'd had four years of math, grades 9-12, at a relatively good high school but for my college freshman year went to a state school that was cheap and close enough for me to walk there and back. They wouldn't let me take calculus but pushed me into a course beneath what I'd already done in high school. So a girlfriend told me when the tests were, and I showed up for those. The teacher said I was the best math student he'd ever had -- in no very real sense was I his student!
I asked to be permitted to start calculus but was turned down. So I got a good book and dug in. I did well (in high school mostly I'd been teaching myself from the books and sleeping in class anyway). For my sophomore year I went to a good college with an excellent college math department, started on their sophomore calculus, from the same text used at Harvard, and did well.
I'm hot on this stuff about what middle and high schools do to good math students: My eighth grade arithmetic teacher gave me a D and took me aside one on one and fervently urged me never to take anymore math. In some significant ways, my father was good at education and just laughed.
That eighth grade teacher didn't have a clue about what math was: For the actual math, I'd done well in her class. What I did poorly on was multiplying two four digit numbers, and the reasons were (1) my handwriting was sloppy (common for boys), so sloppy I didn't get the columns lined up and then added wrong for the final answers, (2) my clerical ability was just awful (common for boys), (3) I understood the ideas right away but never worked on the clerical skills to get correct answers, (4) my parents never urged me to work to get right answers (my father had some high-end views of education and just wanted me to learn, especially out of interest, and didn't care at all about grades, and I did learn).
But in the ninth grade, the teacher saw right away that I was his best student, and he sent me to the state math tournament. In the summer after the 11th grade I was sent to an NSF math and physics program.
So, net, I was a good math student, was very interested in math, was eager to race ahead, but the school did a poor job helping me.
There is a little more to the story: When the SATs came back, I was called out of study hall to get my scores, from the same teacher I'd had in the sixth grade. Nice woman. Sweet. Ignorant! She, along with most of the rest of the teachers, thought I was a dunce. So, she read me the Verbal SAT score -- 538. She said "Very good". BS! It ...
I didn't mention any activities at the college level. It's clear you have no idea what you're taking about.
Instead, you wrote:
> Harvard/MIT math contest (which is way more fun and difficult than the Temple School of Actuarial Sciences contest or Drexel's)
Sounds like college.
> It's clear you have no idea what you're taking about.
What's now fully clear is just that you are angry about something and want to attack me personally, not my writing or thinking but just me, personally. Not good.
To clarify: I think you don't know what kinds of problems they ask, what their contest format is like, what students learn and practice when preparing for them, how it affects students' interest in research mathematics and how it develops their abilities there. Why do I think that? Because instead of talking about the USAMO or ARML or the NSA's mail-in contest or anything specific about whether these contests actually harm or distract students' attention from what you think they should be doing, instead of rebutting the well-known "calculus trap" opinion or showing that you have any awareness of it, instead of showing knowledge of what students do to prepare for these contests, you've talked in abstractions, and there is no glue tying the words you are saying to objective reality.
You are correct that the OP didn't directly say that the "training" of the "kids" was poor.
My view of such high school math competitions is that they waste the time and effort of the students and, otherwise, with good guidance, there would be significant results from the efforts 30 years later. In this sense, then the training was a "poor job".
But the OP did not say that directly -- that's my interpretation, heavily from what I saw used as efforts to have some high school students do better in math -- the efforts were a waste of time for all concerned and because the people directing the efforts didn't know enough math.
Sorry not to be clear.
My view of efforts in grades 8-12 or so to have students do well in math competitions, e.g., the Math Olympiad, is that those efforts are not promising for helping those students do well with math or anything else later in life. Instead, mostly the efforts are just wasteful for all concerned.
The main reason for the waste is that the people, usually the high school math teachers, running the programs don't know enough math to give good direction to the students that might want to do especially well in math, well enough that 30 years later we would commonly see significant positive effects from that direction and the resulting efforts of the students.
So, the direction that is commonly given is to have the students study relatively useless material, say, recreational math, some puzzle problems, every tricky thing they can see in Pascal's triangle (the binomial coefficients, that is, the coefficients of (x + y)^n with one row for each value of positive integer n), and there are a lot of such tricky things.
But, right, the binomial coefficients have some nice properties, some of which are good to know, and an early volume of D. Knuth's The Art of Computer Programming has some good coverage. But, net, mostly to heck with the triangle and, if insist, then read the relevant sections of Knuth -- but even there Knuth has a good reason for presenting that material, and what's important is the applications Knuth makes, so, really, just read on in Knuth.
So, instead, students who do want to do better in math, and I did, should just proceed along the usual topics, texts, etc. for a ugrad math major. In particular, as a first goal, ASAP get a good college calculus text, get the basic prerequisites, and dig in.
By the way, the time I looked at the AP calculus materials I didn't like them and concluded that the people who wrote the materials didn't understand calculus very well. Instead, just get a good college calculus text.
For the rest I mentioned, about have to have a relatively applied ugrad math major to understand: So, would need to know some linear programming, network flows, optimization, integer programming, something about the question of P versus NP, and more.
For Lagrangian relaxation, that's a bit tricky and specialized and not commonly well known even among college math profs. I did give an outline in
https://news.ycombinator.com/item?id=8919311
but it's not really easy reading. But it's a cute topic with some a good intuitive views, a bit amazing in some ways, and at times shockingly powerful.
E.g., the guy who first did the many worlds interpretation of quantum mechanics, a Ph.D. physics student of J. Wheeler at Princeton, an expert in relativity theory, was H. Everett, and at one time he had a nice applied math shop near DC doing resource allocation for the US DoD using a first, simple version of Lagrangian relaxation. What I outlined in
https://news.ycombinator.com/item?id=8919311
is more general and powerful and makes some cute use of some convexity -- that there is convexity there is also surprising but true and powerful and, with the right setup, easy to prove.
If anything I was surprised so many of them had done so well. I was on the UK squad for a few years and most of my friends from then seem to have ordinary jobs at companies you've never heard of (like me).
While the hedge fund landing zone is very typical of the math grifted grads, I expected a higher % in PHD/professor world then were listed.
If someone has a natural talent that they've developed into a very useful skill (mathematics, programming, selling vacuum cleaners, whatever) and they use that to get a cush 9 to 5 job, all the more power too them. Let them spend less time on politics, and more time on family, friends and hobbies. Who are we to say it was a bad decision to let someone else write the next great math paper?
If anything, this highlights that an aptitude for theoretical mathematics can be a launching point to a lot of different things, and that's refreshing. (I'd be less likely to encourage kids in math if Professorships were the only outcome)
There's a huge "Shit Happens" factor. One bad decision or major health problem, or just a loss of interest, and you're on a different path and it's hard as hell to fight your way back. Take that out 20 years, and you get the high fall-out rate. It's not that a lot of people fall off "the good track" for one reason; it's a bunch of reasons that affect small numbers of people.
Success is also hard to measure at an individual level. If you're 35 and making $130k as a full professor of mathematics, you're a star. If you're a consultant making $130k working specifically on what you want to work on, you're a success. If you're still a non-shot-caller making $130k at a Silicon Valley company, at 35, then you've failed. We don't really know how people are doing, though. The "web developer in Boston" might be able to pick his projects, or making a lot of money.
In fact, I often feel pangs about not staying in academia. On the other hand, I've been able to carve out a life with a big dose of public intellectual -- for want of a better or less pretentious term -- so that's cool too. E.g., I'm repeatedly told that my blog is "required reading" for grad students in the relevant areas of computer science ...
I heard similar advice from people in many different lines of work including laws, finance, medicine, academia... usually they add it's not as interesting as it used to be.
It's a matter of compromise. I wish I could have the salary of banker, the freedom of a tenured professor and the prestige of a doctor.
Well, one disadvantage of academia is that you don't really get to choose where to live, even if your CV is quite good. Indeed, there are a number of academic couples that make crazy long-distance commuter marriages work because they weren't able to get jobs at the same institution (or in the same city).
There early in my career, once I sent some resumes and in two weeks went on seven interviews and got five offers.
Big topics were scientific engineering programming, numerical analysis, especially numerical linear algebra, standard applied statistics -- hypothesis testing, linear statistics, analysis of variance, curve fitting, etc. -- the fast Fourier transform, digital filtering, power spectral estimation, deterministic optimal control theory, linear and non-linear programming, discrete and continuous time stochastic processes, Monte-Carlo, numerical solutions of ordinary and partial differential equations, etc.
Some of these topics are part of mathematical EE.
At one time I my annual salary was six times what a new, high end Camaro cost. My wife got to pursue her Ph.D. in essentially mathematical sociology as part of her desire to save the world while we had big times at Thanksgiving and Christmas, shopped in Georgetown for high end French wine and cheese, were regulars with good tables at high end restaurants, went to lots of concerts and plays, got a good piano for her and a good violin for me, got boxes of fancy French pastries and sweet, sparkling dessert wines and on Saturday nights pigged out watching old movies, went to Shenandoah for vacations, had plenty of time for trips to families in Tennessee and Indiana, did some fancy cooking at home, etc.
Of course the money was coming almost entirely through the US Federal Government, mostly for US national security, e.g., via the US Navy, DARPA, etc. But, still, there was money via the DoE, NIST, and more.
I went ahead and got an applied math Ph.D. at Johns Hopkins, research on stochastic optimal control.
My startup has at its core some applied math I derived based on some advanced prerequisites I got mostly at Hopkins. The math is an advantage. If the startup works, then the math will have been the key -- except for the math and the basic business idea, it's all routine, that is, the math is the only thing really special.
Math can be used to say how to take data that is available and manipulate it to get data that is valuable. Now there is a lot of data and means to communicate, store, process, and display it, infrastructure software to make the rest much easier, ads to run to get revenue, etc.
Except possibly for coding theory and cryptography, I don't know if there are valuable applications of algebraic topology, algebraic geometry, algebraic number theory, commutative algebra, etc., but for the topics I mentioned it would seem that there is some practical value.
For some of what can be done with applications, there is a long dessert buffet from the Hahn-Banach theorem and more in D. Luenberger, Optimization by Vector Space Methods. One heck of a toolkit. Make just one good application and retire, make a donation to a university, and get your name on a building.
Here's another possibility: Attack integer linear programming problems in business. Focus on the problems that can attack with min cost flows on networks with integer arc capacities. Why? Because it is just linear programming; the simplex algorithm becomes really simple in that case; in practice the simplex algorithm is fast beyond belief on problems large beyond belief; and get integer programming for no extra effort -- if start with an integer basic feasible solution, then the simplex algorithm will maintain an integer solution and, if there is an optimal solution then will get an integer optimal solution. Can use this in various resource allocation problems. For more, use it as a linear approximation in nonlinear problems and for still more enhance it with some Lagrangian relaxation. Show that P = NP? Nope. Make money? Maybe.
At one time I wrote Fisher Black at GS and got a nice letter back from him, I still have, saying that he saw no opportunities for applied math on Wall Street!
I didn't hear about James Simons until much later.
The B-schools are basically forced by some standards efforts to teach some courses that are essentially applied math.
One might guess that the B-schools want to be professional or clinical but, instead, they tend to want to see themselves as applied social science, and social science has physics envy and wants to be mathematical. Their favorite math is linear statistics. So typically there's a lot of such statistics in the research at B-schools. And typically the B-school profs are not very good at math or statistics.
B-schools also want to dabble in computing -- if you can teach some courses in computing, even as electives, then you might get liked for that.
No doubt some B-schools would like someone who could teach courses in financial engineering, say, with Brownian motion, stochastic integration, Black-Scholes and generalizations -- basically the Brownian motion solution to the Dirichlet problem. So, if you like mathematical finance, then maybe teach it in a B-school.
DC may still be a good area for applied math: So, submit a Civil Service application and also send copies to the usual suspects -- NIST, various DoE labs, various DoD labs, various groups interested in economics or mathematical finance, that is, maybe the Federal Reserve (also try the regional Fed banks), the Social Security Administration, etc. If you want, try NSA, CIA, DIA, FBI, etc. -- no telling what they might want to do with some applied math. Also try the JHU/APL.
There have long been lots of Beltway Bandit shops because commonly Congress gives the departments of the Federal Government more money to spend than full time head count to spend it on so that a lot of the money and/or work goes through companies. E.g., Edward Snowden worked for Booz-Allen or some such.
For technical jobs around DC, it used to be that the job ads in The Washington Post were good places to look.
There is a chance that Google, Facebook, Yahoo and some others are interested in some projects that might be able to use some high end applied math. A problem may be that the other workers there are not very good at math and, really, make a mess out of the work, the opportunities, the positions, etc., and that can be bad for everyone.
It's not clear that everything in mathematical finance is a rat race. I doubt that James Simons ran a rat race shop.
Houston has some important slots for optimization, e.g., as in the work of Princeton Chem Eng prof Floudas. E.g., here's all the crude oil inputs we have available and the costs, and here's all the prices for all the possible refinery products. So, say what crude oil to buy and what products to sell to maximize earnings. That's an old problem, but likely people are still working on it.
The airlines have some very serious problems in fleet and crew scheduling, and first cut it's integer linear programming. But with uncertainties, if you want to handle them at all realistically, the formulations and solutions can become challenging in all respects. A better solution can show its value just on the computer and get taken seriously. For some airline spending $100 million a month just on jet fuel, saving a few percent can pay for some nice computing, work, etc.
Maybe look at what the operations research people are doing these days in scheduling for trucks, airplanes, cargo ships, etc.
No doubt Amazon, Wal-Mart, etc. have some good problems that are generalization of the old transportation problem -- the one Kantorovich got a Nobel prize for. Now we regard that problem as least cost flows on a network and use the simplex algorithm to solve it and where a basic feasible solution corresponds to a spanning tree of arcs on the network.
My view is that broadly...
FedEx had candidate investor General Dynamics (GD), and GD had sent two of their guys, one aero engineer and one finance guy, to help FedEx. FedEx needed the GD investment.
At one point, the Board wanted some revenue projections. I wasn't asked to get involved and didn't want to, but no one had any ideas of how to do the projections except just draw a free hand line on a graph based on hopes, dreams, intentions, guesses, etc.
So, I thought: We know (1) the current revenue. We know (2) the revenue when the planned service to 90 US cities with 33 airplanes is full. So, the projections are roughly how interpolate between (1) and (2).
For how the interpolation will go, we have (3) current customers and (4) the rest of the customers we will have when we have (2) and are full.
Then say that the growth is from current customers (3) talking to the rest of the customers (4). That is, the rate of growth is directly proportional to the number of (happy) customers (3) talking to the rest of the customers we will have (4).
So, let t denote time in days with t = 0 corresponding to the present. Let the revenue at time t be y(t). Let the revenue from (2), being full, be b dollars a day. Then at time t, the growth rate is y'(t), that is, d/dt y(t). So, we have that
y'(t) = k y(t) ( b - y(t) )
How 'bout that! Maybe it's so simple it's almost just a joke, but the rest of the ideas floating around the office are much worse!
So, sure, it's an initial value problem for a first order, linear ordinary differential equation but, really, is so simple can just use freshman calculus directly and find the closed form solution, right, some exponentials I omit here.
So, on a Friday the Senior Vice President (SVP) for Planning and I got out some graph paper, picked a value for k, and drew the graph.
The next day, Saturday, I was in my office, and at noon I got a phone call asking if I knew about the revenue projections. Well, the SVP was traveling, and, yes, I knew. So, I was asked if I could come over to the HQ offices and explain. So, I got in my Camaro hot rod and drove over.
When I got there, people were unhappy. The two GD guys were standing in the hall with their bags packed.
I was led to the graph, given a time, and asked to reproduce the value on the graph at that time. So, I had brought my HP scientific calculator, punched the buttons, and got the value on the graph. I did that a few more times, and people started to be happy again.
It turned out that the Board meeting had been that Saturday morning; the graph had been presented; and the two GD guys had asked how it had been calculated.
Well, all the FedEx people there tried to figure out how the graph had been calculated and went on for a few hours without success, and then GD guys lost patience, got plane reservations back to Texas, went to their rented rooms and packed their bags, and as a last chance returned to the HQ offices to see if FedEx had an explanation for the graph. FedEx was about to die.
That's when I arrived and explained the graph.
The FedEx guys stayed, and FedEx was saved.
I never got thanked! Really my impression was that the rest of the top of FedEx had concluded that I'd been dangerous, maybe had 'hooked' the company.
And I'd never been invited to the Board meeting. So, I began to conclude that my contribution was so resented that the top of FedEx would rather see all of FedEx go under than let me help.
The promised FedEx stock was already 18 months late. I was in Memphis while my wife was in her Ph.D. program at Johns Hopkins near our home in Maryland. So, to heck with FedEx -- I went home and to Hopkins for my Ph.D.
So, that's an example of applied math in US business! E.g., I was the only one around who still knew freshman calculus! And, heck, I'd never even taken freshman calculus, had studied it on my own and started with sophomore calc...
That story about FedEx is hilarious!
> Having started working mathematically recently it seems to me that even though I can propose a solution which isn't the best, they are usually better than what is currently available. Is that a common experience as a working mathematician.
Sure. For the politics, just report the savings. The stuff about the optimal solution, that is, its, say, moral or ethical value, is lost on business people. With some good common sense, the business people just want to know how much has been saved (or extra earned). If you can also have, say, a lower bound on the possible savings and get close to that, then you have in practice a stopping criterion. Or, what in practice is really difficult about the NP-hard problems is usually saving the very last possible tiny fraction of one penny and proving that did that. Instead, just concentrate on saving, say, the first $5 million a month on, say, the $100 million a month of jet fuel for an airline.
For a while in optimization, having the optimal solution was made a moral objective with anything that wasted even 0.0001 pennies regarded as a sin. Religious nonsense.
Really sloppy work is not good, but close approximation usually is plenty good.
For not the best, that's the other time I kept FedEx from going out of business: The founder, Fred W. Smith, spent the afternoon in his office trying to schedule the fleet, staggered out exhausted, said "We need a computer", and a college friend of mine heard that plea and called me.
Sure, digging in, I heard a lot about optimal solutions. But I knew that first just get something that helps, say, so can type in a schedule, have the software check it out for, say, planes being over loaded or having to fly too far, get the arrival times, etc., and print out all the details. So, I typed in 6000 lines of code to do that. Right, great circle distances are from just the law of cosines for spherical triangles!
Soon the Board wanted a schedule; biggie Board members doubted that the scheduling could even be done; biggie funding was being held up; the company was at risk.
So, I got a call from the same guy who called me to the Board meeting for the revenue projections, Roger Frock (he wrote a book about FedEx), and he and one short, easy evening he and I used my software to develop a schedule for the whole planned system of 90 cities and 33 airplanes. It printed out and looked nice.
At the next senior staff meeting, Fred's reaction to the printout was "Amazing document -- solved the most important problem facing Federal Express". Our same two guys from General Dynamics went over the schedule carefully and announced "It's a little tight in a few places but it's flyable". The Board was pleased; the funding was enabled. That was the first time I saved FedEx.
So, I got into how to do better scheduling. So, for one period from, say, 5 PM to midnight (again from 1 AM to, say, 8 AM) set up a table with one row for each city to be served, 90 cities, and one column for each candidate airplane tour from Memphis and back. So, may have some thousands of columns. Considering all fine details want to account for, and take expectations for random costs, find for each of the candidate tours the cost (first cut, assume that if a plane goes to a city, then it completely serves that city).
Then for the schedule, want to pick no more than 33 columns from the 1000s so that all 90 cities are served and total cost is minimized.
So, each column covers some cities, and want to pick columns that cover all 90 cities -- so have a set covering problem. Right, it's in NP-complete.
Well, that table is essentially the matrix in a 0-1 integer linear programming least cost optimization problem. There have one unknown variable for each of the columns, and the variable is 1 if use that column and 0 otherwise. If relax the 0-1 constraint, then have a fairly routine linear programming problem but will li...
For what companies to go to to get going on a career in such applied math, I can't think of any. Mostly people and their organizations would rather waste money than get involved with mathematicians doing work only mathematicians understand.
Again, if want to make math pay, just sell the results, usually as a vendor outside the customer.
If are inside the company, then likely no good work will go unpunished. Basically to be successful, you will need the strong personal support of the CEO and hopefully the Board -- literally. And the CEO will routinely have to go around and break 2 x 4s over the heads of recalcitrant middle managers to see if they are just asleep or really dead -- otherwise those middle managers can sabotage you with high determination and creativity.
E.g., twice I saved FedEx from going out of business and had much more that would have saved FedEx maybe tens of millions of dollars a year, but I got flack and push back and contempt and none of the promised stock.
Such are some of the challenges in getting stuff done. But successful people have been overcoming challenges, some much worse than I faced at FedEx, for many centuries.
Now that you know about the possibilities of some of the absurd challenges, don't be surprised. Really, one of the best rules is just to keep your good technical work essentially a secret until you have a solution that is just too darned good to refuse, and then pull back the curtain, let people see the results and also see that it's too late to fight with you.
Better still, just provide such results as a business and own the business. E.g., in some cases, just do the work for free, show the results, and then announce "You can have this for your real operations for a fee of only 15% of the savings."
Once there was an in-house research group that was successful. It was for WestVaco paper -- think specialty papers, e.g., coated papers for milk cartons. So, they had Ph.D. chem eng guys running their plants in jungles, etc., that is, where the trees were, but it was a family owned business with HQ in NYC.
The research shop was in Maryland, and I got an explanation from them: Each year Research went to NYC for their budget. Always the CEO, family guy, offered much more money for the Research shop than the shop was willing to accept (first-cut, good budget situation!).
Why? The rules: Research picks their own projects. Projects tend to run from a few months to a few years. One project in 10 gets to operations. When Research has a project ready for operations, they approach the relevant operational managers. In the first three years, savings are allocated half to Research and half to the operational unit. After three years, all the savings are allocated to the operational unit. With this accounting, Research was returning $3 to the company for each $1 in their budget. So, sure, HQ wanted to increase the budget, but Research didn't want to have to find ways to return $3 for each extra budget $1 so didn't want a bigger budget!
It is just crucial that the CEO is supporting such in-house innovation. The rules are also important.
Ah, while I was in grad school, to make enough money to support my wife and I while we both finished our Ph.D.s, I took a job at a DC area think tank working for the US Navy.
At one point our problem sponsor group wanted an analysis of a special, worrisome scenario of global nuclear war but limited to sea -- they wanted to know how long the US SSBN fleet could survive if it was just held in reserve and didn't shoot its missiles with nuke warheads. And they wanted the results in two weeks.
Gee, two weeks to model global nuclear war limited to sea! How generous! Good that they were not in a real hurry! Besides it was good timing since the day after the due date my wife had already gotten us reservations for a vacation at Shenandoah!
Well, there was a guy B. Koopman who in WWII had written a report OEG-56 on finding things at sea. So, gi...
Again, thanks so much. One last question, what are you doing with your life right now? It seems like you have had a pretty broad career and I am interested in the end game. If you are retired, what is your lifestyle like and in particular how has your career impacted your life as a retiree.
For now, sure, I know some math and can stir up some more, and I know some computing and can learn more.
So, I've got a 1.8 GHz AMD single core processor, Windows XP Professional SP3 (I have an official copy of Windows 7 Professional on DVD but have seen no great reason to go to all the trouble to install it and rebuild all my software environment yet), three hard disks, 100 million files, .NET Framework 4.0, a good text editor (KEdit), Visual Basic .NET (comes with the .NET Framework), ASP.NET, ADO.NET, IIS (low level Web server), SQL Server Express, etc.
I thought, why not go after about 2/3rds of Internet search, the safe for work part served at best poorly by looking for keywords/phrases?
So, how to do that? Not with just routine software! And not with anything commonly talked about for search!
I stirred up some math, typed the theorems and proofs into Knuth's TeX, worked up a scalable architecture, and typed in the software.
Maybe I should have used Redis, but instead I thought that writing my own session state server would be faster than even understanding Redis. So, my session state server is single threaded ("Look, Ma, no concurrency problems"), for faster lookup rates is trivial to run as several instances as in sharding, and is just some TCP/IP sockets, some de/serialization of instances of my session state class, and uses two instances of a .NET collection class. Simple.
It's fast! A server for less than $1500 should be able to keep session state for an hour of inactivity for each user and do the session state work for sending 5000+ Web pages a second. Two standard racks should be able to handle session state for the world.
The rest of the software is also readily scalable also from just simple sharding.
Currently I have one bug in one Web page -- it's not handling session state just right! But I have the fix in code in another Web page and should copy it over today!
My interactive development environment is just KEdit with about 100 macros and some careful use of file system directories.
I'm using SQL Server only to record the data from the users and for the results of some batch computations; at one point I actually do make use of a transaction; the data for the searches is drawn from SQL Server with a batch program (run it maybe once a day); some solid state drives (write rarely, read thousands of times a second) should do wonders for the data for the searches.
I was about to fix the bug in the Web page but took some opportunities to gather some good initial data. The site will start focused and only slowly grow to be comprehensive -- right, at first do some things that don't scale and please some niche group of users a lot instead of trying to please 2+ billion users a little.
Currently the database has only some meaningless data I put in for first testing of the software. It's about time to load in some of the good initial data I have. Then give a critical review, go live, etc.
It's getting there.
All the work uniquely mine has been fast, fun, and easy, but the whole project has taken far, far, far too long. Why? I worked through about a cubic foot of books and 6000+ Web pages of documentation of Windows, .NET, Visual Basic .NET, ASP.NET, ADO.NET, SQL Server, etc.
The main problems: (1) Badly written, obscure documentation (worst bottleneck in the future of computing); (2) computer viruses; (3) SQL Server installation bugs destroying my boot partition requiring rebuilding starting with the XP DVD (barbed wire enema with an unanesthetized upper molar root canal procedure); (4) SQL Server management and administration (e.g., a week of throwing stuff against a wall to see what sticks just to get a SQL Server connection string that will let code for a server side Web page connect with SQL Server); (5) clean, smooth means of system backup and recovery (including for both user data and bootable partitions); (6) Sony DVD d...
Hi graycat,
I'm an applied math & CS undergrad at Caltech and I love all of your posts, especially the ones about how data center scale computing really needs to embrace statistics. The FedEx stories are also excellent. I'm personally interested in high performance computing and machine learning, but I'm also interested in solving "real problems" and like how you seem to focus on the actual value of the applications. I love the feeling of engineering solutions to mathematical problems, and this seems to be something that you also enjoy.
I'd really like to hear more about your career, research, and also the startup that you're working on. I'd be very happy if you shoot me an email at eric@ericmart.in
Best, Eric
I really enjoy your posts and your writing style. Thanks much. I am currently working on some math problems as a part of a business that makes accessible to Indians who do not have access to structured banking services. Would love to discuss it with you, if you are inclined. My email is takenottie at google's mail. Thanks.
Well, in part we aimed too high: My wife was brilliant, quite broadly -- yes, Valedictorian, Summa Cum Laude, Woodrow Wilson, PBK, piano, clarinet, voice, prizes in cooking, sewing, raising chickens. But her family had her try to be perfect and to dedicate herself to saving the world.
She wanted a Ph.D. in mathematical sociology to do social engineering of social change to save the world and, thus, get her praise, acceptance, and emotional and financial security. And those concerns filled her plate.
Well, at the roots there some anxiety, from nature and/or nurture, was involved, then some perfectionism and some fear of criticism from powerful, influential people -- net, call it a special case of social phobia. That brought stress which can bring depression. That slows the work and in her case caused more stress and more depression then clinical depression. She was in a clinical depression the day she got her Ph.D. She never recovered and, net, didn't make it.
Took me a while to reinvent and learn the basics of clinical psychology to understand what was going on and what to do about it. I did learn but nearly always too late.
In trying to have a stable job so that I could take care of her, I took a job at IBM's Watson lab, in what we called artificial intelligence (AI) to do monitoring and management of large server farms and networks.
Then IBM got sick and the lab phone book went from 4500 full time names down to 1000 or so. The guy who hired me, a big star who was deliberately ignored by the higher ups, left for greener pastures -- eventually ended up with a nice place in Malibu.
IBM Watson Research was run by a clique of people who stuck together and blocked out everyone else. At one point it appeared that the company HQ in Armonk tried to correct that situation.
Instead of or better than the AI, I had some ideas: Detecting a problem in a server farm or network is necessarily essentially a statistical hypothesis test with Type I (false alarm) and Type II (missed detection) errors. So, want hypothesis tests that keep down the rates of the errors.
Since there's a lot of data readily available and want to exploit it to keep down the rates of the errors, also want multi-variate inputs -- nearly all hypothesis tests have only univariate inputs. Also for such multi-variate data can't hope to know any of the probability distributions so need tests that are distribution-free.
I don't think there were any such tests, so I dreamed up some. I used some group theory, summed used the classic S. Ulam, guy on the left in
http://www-history.mcs.st-and.ac.uk/BigPictures/Ulam_Feynman...
result LeCam called tightness (see P. Billingsley, Convergence of Probability Measures), and was able to permit selecting false alarm rate in advance and then getting that rate exactly. Asking for all of the Neyman-Pearson result would take more data than we had any chance of having, but there was a somewhat useful sense in which my work gave asymptotically, for any selected false alarm rate, the highest possible detection rate with that false alarm rate.
I cooked up some synthetic data that was challenging -- the critical region was something like the red squares on a checkerboard in several dimensions. My work did fine.
I got some data from a cluster of computers at Allstate, wrote some prototype software, and confirmed the false alarm rate empirically. And I cooked up an algorithm to make the computations nicely fast (in part I used k-D trees -- reinvented those -- a few years earlier and k-D trees would have been mine).
Politics: A guy in the clique up there didn't like me. But it took two levels of management to fire someone, so he reorganized to put me under a wuss who would g...
I work for an extremely large defense contractor, and it never ceases to amaze me how many of my fellow engineers have forgotten the mathematics they were taught in high school! I know many of them took AP courses and learned calculus there, but they have completely and totally forgotten it.
Yep, this can be lucrative if you find a company with a genuine need for it. The company I work for has recently seized an opportunity to apply a mixed integer linear algorithm to a specific problem in the logistics industry, and I think it will soon become one of our most successful products.
I wonder how their lives is in the financial industry. I imagine finance as a demanding and stressful environment, driven by big egos, commercial types. How do these math geniuses fit there? what level of freedom do they have in their job? in term of schedule, things they work on and so on...
The same is true of academia, from what I understand, just less money.
I don't know who Dan Scales is but if you can't google someone nowadays then they might just work for the government ;-)