Hi HN peeps, I did a write-up on the major lessons I learned doing ML research at Google and PathAI, including my time working with Samy Bengio and Ian Goodfellow.
The main questions I'm curious to answer are (1) why do so many good-seeming ideas fail? and (2) what should we do about it? So I'm curious to know y'alls answers on those questions too.
Thanks for the write-up! The "empathize with the model" line is great. :)
I think the problem is really that people (in ML especially) come up with solutions before they've properly framed the problem that the solution is trying to solve. We should be finding solutions to actual problems, observed by looking at the data and past model performance on that data.
And once we've got a good problem, there's usually more than one way to address it. Your initial 'good idea' is typically just one, and often not the easiest to implement or evaluate.
So, when faced with an observable problem, I formulate a number of possible causes (hypotheses), and then list two or three ways that I might solve the problem if that hypothesis is correct. I then pick the /simplest/ solution for each hypothesis, implement them, and see how each one does. Some will have more effect than others; this helps update my prior on which hypothesis might really be driving the problem, at which point I might consider revisting some of the more complicated solutions. Assuming it hasn't already been fixed!
You deserve some credit for raising the question, but in some sense I'm not sure what you'd expect?
I don't mean that in a hostile way, I just mean that the overlap between reality and good ideas is very low almost by necessity? This is why we have science, after all.
In fact, I think the idea that people who stumble on things that work somehow do so because they're smarter or more talented than those who don't is somewhat dangerous. Certainly, bad ideas are bad. But at some point you reach a set of ideas that are all equally probable given existing information, and you're literally at the mercy of luck, almost by definition. Maybe there's some bit of information here or there that can tilt the scales one way or another, and maybe some are better at that than others, or just have access to additional information through experience or resources, but at some point you are just left with the need for raw experimentation.
I think the only way to do this really is through as much information as possible, which is why I think transparency in data is key -- so you can make use of what others have done already as much as possible.
Maybe there's some way of bootstrapping to a sort of meta-modeling? What I mean is, maybe you can develop a model/AI system that learns how to learn and becomes sort of auto-developing, but it seems like we're not quite there yet.
Yeah this is good point, honestly hadn't thought about this.
Yeah I guess in any field the low-hanging fruit will be yanked. One situation we could end up in is where we feel like we have more possibilities but counter-intuitively they don't work (otherwise, by definition I guess we could say, they would be low-hanging fruit that had already been yanked). This is the situation you may be describing. Let's call this situation A.
Alternatively, we could end up in a place where there's no more low-hanging fruit and we also can't even come up with ideas that seem promising. I guess at that point the field will just kind of die and people will move on to other pursuits. Let's call this situation B.
I guess a field should hit A at some point and may continue to progress to B eventually or splinter into sub-fields.
Yeah so from a very meta perspective I guess I should expect that things that I expect to work should fail. But at the same time I can't ignore the fact that I honestly felt surprised for at least a few of my ideas that they failed (others felt more like a gamble so it's no surprise they failed). So in some sense I should expect to be surprised but I am still trying to get to the bottom of why I felt surprised in the individual situations...
Your point does raise some other psychological questions. Like maybe we're wired to delusionally believe our likely-to-fail ideas are likely-to-succeed. If we get rid of those delusional beliefs, will we become unmotivated to try new ideas, or will we just seek out actually promising ideas?
Your experience rings true to me. It's one of my biggest frustrations with ML at the moment. There are so many ideas I'd like to try, but I know only a small fraction of them will work, and discovering which of the ideas fail and which succeed is a herculean task. Your conclusion that empathy helps may also be true, but I have a different take.
It currently takes way too much time to explore ML-based ideas. I compare this to the early days of computer programming where programmers needed to manually fill out punch cards, and doing anything took days of full time work. There is lots of room for improvement along every step of the ML pipeline, from data wrangling, model choice, training, and evaluation. Good ML tooling will likely bring huge gains in the field.
It gets better with practice as you develop more accurate intuition. Eventually, most new ideas that you try will work, but it takes a lot of failures to get to that level.
Maybe… but the article had an anecdote of asking Ian Goodfellow for ideas and none of those ideas working. I would assume Ian Goodfellow would have the requisite amount of experience for sufficient intuition.
> I would assume Ian Goodfellow would have the requisite amount of experience for sufficient intuition.
Well, maybe. It isn't entirely clear just when that anecdote happened (he was at Google first as an intern, then as a research scientist, then he was at OpenAI before going back to Google).
In any case, as important as a developed intuition is, it is no guarantee that it will provide the necessary insight after hearing a description of the problem rather than experiencing and digging into it himself.
Yep I think you hit another nail on the head (likely the bigger nail).
There is a whole 'nother category of ideas I wasn't thinking about which are the ideas that seem promising but just like way too much effort to even test.
Those ideas just get completely lost when it's hard to test ideas.
This writeup is to be commended for courage and initiative, but to this reader falls terribly in the wisdom category
> attempt and inability to empathize with the model
> watch the tree frog jump with excitement
> intimacy with the model ?!?
my feedback on this essay is that I needed many, many years of psychological study, mindfulness practice and real-life relationship experience to untangle my youthful spirit of life from my intellectual interests. I see both joy and tragedy in mixing your maths and your hormones. The metaphors of insight here are just not mixable, and I believe you are not alone in that.
Maybe using empathy for logical problems is akin to using GPU for computations. Maybe there are some problems it just solves well even though it wasn’t necessarily designed/evolved for that initially. See also people who solve problems in their dreams/subconscious/downtime.
When learning, extending, or fixing software, I generally work by building an understanding of software's principles, implementation, and runtime behavior, so I can reason about the current state and the effects of potential changes. Though sometimes I tweak code to see the effects and better understand the problem (shh we don't talk about tweaking things until they "work on my machine" and we're done).
Understanding runtime behavior is easier in software with high debuggability and transparency (which is orthogonal or even opposed to performance, since logging slows programs down and optimizations impede debugging). Bryan Cantrill's "Platform as a Reflection of Values" talk (https://vimeo.com/230142234) has a list of software values.
I think "observability" is another good word for what I want, but it tends to be thrown around in cloud-scale software rather than the human-scale software I prefer. Perhaps "empathy" is a workable word for developing an intuition (and less so for building a mental model).
Maintaining someone else's code, I'm equal parts mind reader, software archeologist, detective (forensics), and grumpy old man. When I've had to maintain ML stuff, I spent most of my time wrestling with the tools and chaotic mutant one-off instances of "data pipelines", contorting myself into the mindset of the creators.
So, ya, "empathy" works for me.
FWIW, I found the tools like PyTorch, Spark, and such to be laughable bad. I pity the people who have to do this work on the daily. Jupyter is cool though.
“ I’d go so far as to estimate the failure rate of good-seeming ideas to be at least 90%. “
I’m a scientist in materials research and this rate of failure holds as well. The reasons are multifaceted and kind of interesting. The nice thing about ML is that it doesn’t take 5 years to know if it has failed yet :) (Pharma is worse, it can be greater than ones entire career).
I'm surprised that someone who actually works with AI would personify their models. Is that common? Does it help or is it just something we find hard to avoid doing?
16 comments
[ 3.2 ms ] story [ 40.9 ms ] threadThe main questions I'm curious to answer are (1) why do so many good-seeming ideas fail? and (2) what should we do about it? So I'm curious to know y'alls answers on those questions too.
I think the problem is really that people (in ML especially) come up with solutions before they've properly framed the problem that the solution is trying to solve. We should be finding solutions to actual problems, observed by looking at the data and past model performance on that data.
And once we've got a good problem, there's usually more than one way to address it. Your initial 'good idea' is typically just one, and often not the easiest to implement or evaluate.
So, when faced with an observable problem, I formulate a number of possible causes (hypotheses), and then list two or three ways that I might solve the problem if that hypothesis is correct. I then pick the /simplest/ solution for each hypothesis, implement them, and see how each one does. Some will have more effect than others; this helps update my prior on which hypothesis might really be driving the problem, at which point I might consider revisting some of the more complicated solutions. Assuming it hasn't already been fixed!
I don't mean that in a hostile way, I just mean that the overlap between reality and good ideas is very low almost by necessity? This is why we have science, after all.
In fact, I think the idea that people who stumble on things that work somehow do so because they're smarter or more talented than those who don't is somewhat dangerous. Certainly, bad ideas are bad. But at some point you reach a set of ideas that are all equally probable given existing information, and you're literally at the mercy of luck, almost by definition. Maybe there's some bit of information here or there that can tilt the scales one way or another, and maybe some are better at that than others, or just have access to additional information through experience or resources, but at some point you are just left with the need for raw experimentation.
I think the only way to do this really is through as much information as possible, which is why I think transparency in data is key -- so you can make use of what others have done already as much as possible.
Maybe there's some way of bootstrapping to a sort of meta-modeling? What I mean is, maybe you can develop a model/AI system that learns how to learn and becomes sort of auto-developing, but it seems like we're not quite there yet.
Yeah I guess in any field the low-hanging fruit will be yanked. One situation we could end up in is where we feel like we have more possibilities but counter-intuitively they don't work (otherwise, by definition I guess we could say, they would be low-hanging fruit that had already been yanked). This is the situation you may be describing. Let's call this situation A.
Alternatively, we could end up in a place where there's no more low-hanging fruit and we also can't even come up with ideas that seem promising. I guess at that point the field will just kind of die and people will move on to other pursuits. Let's call this situation B.
I guess a field should hit A at some point and may continue to progress to B eventually or splinter into sub-fields.
Yeah so from a very meta perspective I guess I should expect that things that I expect to work should fail. But at the same time I can't ignore the fact that I honestly felt surprised for at least a few of my ideas that they failed (others felt more like a gamble so it's no surprise they failed). So in some sense I should expect to be surprised but I am still trying to get to the bottom of why I felt surprised in the individual situations...
Your point does raise some other psychological questions. Like maybe we're wired to delusionally believe our likely-to-fail ideas are likely-to-succeed. If we get rid of those delusional beliefs, will we become unmotivated to try new ideas, or will we just seek out actually promising ideas?
It currently takes way too much time to explore ML-based ideas. I compare this to the early days of computer programming where programmers needed to manually fill out punch cards, and doing anything took days of full time work. There is lots of room for improvement along every step of the ML pipeline, from data wrangling, model choice, training, and evaluation. Good ML tooling will likely bring huge gains in the field.
Well, maybe. It isn't entirely clear just when that anecdote happened (he was at Google first as an intern, then as a research scientist, then he was at OpenAI before going back to Google).
In any case, as important as a developed intuition is, it is no guarantee that it will provide the necessary insight after hearing a description of the problem rather than experiencing and digging into it himself.
There is a whole 'nother category of ideas I wasn't thinking about which are the ideas that seem promising but just like way too much effort to even test.
Those ideas just get completely lost when it's hard to test ideas.
> attempt and inability to empathize with the model
> watch the tree frog jump with excitement
> intimacy with the model ?!?
my feedback on this essay is that I needed many, many years of psychological study, mindfulness practice and real-life relationship experience to untangle my youthful spirit of life from my intellectual interests. I see both joy and tragedy in mixing your maths and your hormones. The metaphors of insight here are just not mixable, and I believe you are not alone in that.
Understanding runtime behavior is easier in software with high debuggability and transparency (which is orthogonal or even opposed to performance, since logging slows programs down and optimizations impede debugging). Bryan Cantrill's "Platform as a Reflection of Values" talk (https://vimeo.com/230142234) has a list of software values.
I think "observability" is another good word for what I want, but it tends to be thrown around in cloud-scale software rather than the human-scale software I prefer. Perhaps "empathy" is a workable word for developing an intuition (and less so for building a mental model).
So, ya, "empathy" works for me.
FWIW, I found the tools like PyTorch, Spark, and such to be laughable bad. I pity the people who have to do this work on the daily. Jupyter is cool though.
All great problems of ML can never be solved; they can only be outgrown.
EDIT: mentat learning>>machine learning??!!
I’m a scientist in materials research and this rate of failure holds as well. The reasons are multifaceted and kind of interesting. The nice thing about ML is that it doesn’t take 5 years to know if it has failed yet :) (Pharma is worse, it can be greater than ones entire career).