As some of you probably know, I spent a number of years working with GE, which included plenty of time at the legendary GE Research lab in Niskayuna. So it was a special thrill to see this piece connecting Answer.AI to the long history of R&D labs. Our ideas about R&D are kind of out of fashion in our current age, but I hope this says more about the age than it does about the quality of the ideas. Happy to answer questions in this thread, if there are any fellow R&D-lab-enjoyers here.
I've never been a big fan of patents as they've historically applied to software. But this is a public policy issue, not something an individual company can fix. So I've always been content to follow patent best practices, even when they aren't the most logical.
In terms of tech transfer, Jeremy has quite a bit of experience with collaborating with the best academic labs, especially in machine learning, so I expect we will do a lot of that kind of partnering.
Makes sense. We have to work within the system as we find it. Would be great to see the system evolve in a way that would support efforts like yours though!
Jeremy from Answer.AI here -- Answer.AI is, to some extent, the subject of this article. Perhaps worth noting here that we didn't pay the author, Eric Gilliam; it's his independent analysis based on years of study of 19th & 20th century R&D labs.
I'm hoping that not only will Answer.AI be successful, but that Eric's words in the conclusion will come to pass:
> "If their USD10 million experiment works, it has the chance to spark a rush of emboldened researchers and engineers to found small research firms, leveraging the models of the once-great dragons of American industrial R&D."
I'm happy to answer any questions you have about our experiment, and my co-founder Eric Ries (@eries) is around too. (Note that there are 2 Erics mentioned here -- that can get confusing!) By way of background, Eric Ries is the creator of the "Lean Startup", coined the term "Minimal Viable Product", and created the Long Term Stock Exchange, and I'm the co-founder of fast.ai, Enlitic, Kaggle, FastMail, and Optimal Decisions Group.
Currently we're working on low-resource fine-tuning research (i.e. train larger models on smaller cheaper GPUs), and looking into applications in the education and legal spaces. We're also starting to look at intersections between long-tail image recognition and language models.
I don't think we'll want to patent anything, since that's something which keeps people out -- and we want to let people in! The only reason I could think of that we'd need a patent portfolio would be for defensive purposes.
That's different from how things worked for Edison. But similar to how they worked for Luther Burbank.
I'll be rooting for you to succeed in this endeavor. I used to visit IBM ARC when I was a kid, and to my mind the academic environment is not quite the same. But as you know we really have not seen corporate R and D recover from the peaks it attained over fifty years ago.
Jeremy, you've taken a bold, open initiative. Thank you for calling on all the small, innovative teams. To better understand the types of projects or applications that excite you and Eric most, could you share examples of companies or applications that Answer.AI currently supports or plants to invest in? Additionally, if yo u have any published articles detailing these ventures please share the link. Best wishes with your venture.
Could you also elaborate on how companies can reach out to you and Eric? What are the key criteria or 'table stakes' you look for in a project or team?
Fast.AI is awesome and how I first learned ML. Can’t than you enough for that great project and its impact on my career! How do you see Answer.AI as supporting other commercial research efforts that have some overlap with the open source community? I’m thinking of things like Unsloth, which is trying to take a wild swing at efficient fine-tuning of LLMs without getting scooped by big tech.
That’s great to hear! I can’t think of better stewards for this kind of work than a team you’ve put together, especially given everything you’ve provided for the community.
@publicdaniel asked below about the relevance of openai’s Ken Stanley of “greatness cannot be planned” fame. @jph00 Can you comment on the similarities and difference of your approach as you understand it, given that Ken is or was a research manager at OpenAI?
I like the very idea of applying this transformative AI technology to make values. I have been doing research and always been a member of research team ever since I started the ML journey. But, I've always valued applications that can solve problems and found myself inclined to building apps. I would love to work at Answer.AI. How can I apply?
My experience is that many large companies have problems that could be solved/improved with small fine-tuned AI services, but usually they are extremely cagey about disclosing which parts of their workflow are slow/expensive and would, therefore, be the best targets for automation and/or improvements coming from AI research. The cliche example here would be to automate processing of some paper forms. And it's cliche precisely because for most companies it would help them become faster and more competitive.
How do you plan to research potential problems to solve?
Would it be possible to publish and discuss those to-be-solved problems, so that hobbyists can also try their luck at solving them?
Initially, I thought that Kaggle competitions would be a great way to get the industry's problems into the hands of motivated problem-solvers. But I don't think I've ever seen a large car manufacturer (for example) as Kaggle competition host. So it looks like something is holding companies back.
“At first sight, when one comes upon it in its surprisingly rural setting, the Bell Telephone Laboratories’ main New Jersey site looks like a large and up-to-date factory, which in a sense it is. But it is a factory for ideas, and so its production lines are invisible.” — Arthur C. Clarke, science fiction writer, in his 1958 book, Voice Across the Sea.
Prescient, really. Almost all of us knowledge workers today work in factories where the production lines are invisible. It's why we struggle so much with concepts (like work-in-progres inventory or product development flow) that are relatively easy to understand when there is a physical factory to look at.
@Jeremy, have you ever encountered Ken Stanley’s book “Why Greatness Cannot be Planned: The Myth of the Objective”?
If you’re not familiar with him, he was the guy that invented the NEAT algorithm, Novelty Search, etc.
In his book he talks about stepping stones and following what’s “interesting” to individuals. It seems like Answer.AI is focused on applied engineering, but any thoughts on this type of approach from a Research perspective?
This is funnily relevant because as far as I understand Ken is or was a research manager at OpenAI, and as outlined in the article, Answer.AI is trying not to be OpenAI.
I should caveat this by saying I've only read summaries of the book, not the book itself. My understanding of it is that they view setting ambitious objectives as potentially limiting progress, and instead promote shorter-term novelty-seeking approaches.
This is certainly how we do things at Answer.AI -- we hire people that are passionate tinkerers, and encourage a playful and spontaneous approach. That doesn't mean there's no coordination or long-term goal, but rather that we view these short-term approaches as being a good way to make progress.
Thanks! That nicely dovetails with my understnding. BTW, the two case studies at the end of the book I found to be helpful in analyzing the main ideas!
Hey! It’s Eric (the author). To the author of this comment, I’d say that the use of “long leash within a narrow fence” (which directed Langmuir’s work) or “circumscribed freedom” (which directed much Bell work) is generally compatible with researcher interests.
As I talk about in the article, Langmuir was able to pick from a bunch of problems in his wheelhouse…there were just conditions. And those conditions meant whatever area he picked might come with a high willingness to spend from GE! And if his work yielded results GE would have a way to quickly deploy the knowledge. All of which is great. To put his work in a box with what the Coolidge-types did is probably unfair. He was following curiosity under constraints, that’s all.
That is not to say all basic research roles in the world should look like that. But it makes sense given most basic researchers don’t fully understand which of all the problems they could happily pursue are actually most useful to industry. MIT professors of the early 1900s used to source research problems somewhat similarly.
The classic on the early electrical manufacturers is "The electrical manufacturers 1875-1900".[1] This is the Harvard Business School take on Edison. The business plan for his first power plant is in there. (He way underestimated the payback period, but it was only a few years, so it worked out.) There's also "Men and Volts"[2], a history of the early days of General Electric. Which, I think, is where this article got their info on tungsten.
The big corporate labs were part of far larger manufacturing companies. It's not clear that a standalone lab can function in the same way.
Hey it’s Eric (the author). I talk quite a bit in the article about how to make the lessons of places like Bell and GE work even if you’re not attached to a large lab.
Repurposing Bell-style systems engineers to isolate white hot problems is one example. Another is the general BBN-style approach which worked for them as a stand-alone R&D firm. I have stand alone pieces where I explore both of those quite a bit.
In general, I don’t think the concept of the playbooks of the great industrial R&D labs not working in firms un-attached to large firms holds much water.
I'm going to have to look at some of your other material.
Experimentation is different than other research or engineering but I think anything that amounts to a "system" can benefit from a systems problem solver who can perform as needed across the whole system.
Any organization is fortunate to have a person who can go without routine or typical workdays and do this.
It does seem like the bigger companies which have the most clearly defined roles can eventually get by on momentum without having anyone at all in a creative problem-solving position.
OTOH a small outfit can deploy unique problem-solving ability using only a handful of people that many corporations have none of. So the "very, very best people" may be what can get the momentum going to begin with given the right opportunity.
You wouldn't want it to function the same way as a modern corporate lab. Those are already making as much progress as they can by now.
Something worthwhile to do in a smaller organization is things that bigger labs need but don't do.
If you do it "right" a single-handed individual can make more promising creative progress than a whole team of corporate researchers can often end up with, but you need some pretty good breadth of innovation.
If you look carefully at the historical illustration, you may notice a pipe organ is the central device in the picture.
In the natural science lab I always liked to have different instruments that attract the attention of different types of visitors at different times.
And some instruments that are only developed or used in the presence of no visitors at all.
The best of luck to you guys, and excellent article by Eric. I recently came across his freaktales.com site, and was delighted to find another fan of the bygone age of R&D.
We also just launched Unbelievable Labs (unbelievablelabs.com), with the ambition to combine Physics-oriented venture building with applied research. It would be great to form a network of these to emulate some of the scale that Bell Labs had due to monopoly funding.
I am a researcher working in an industrial research lab who is in the process of changing careers (with research being more of a hobby rather than something that’s subject to performance reviews) due to my disagreements with the short-term focus of upper management and the constant pressure to “deliver value” on a regular basis. This post from Answer.ai was quite timely for me, and I am rooting for the lab to succeed! We need more research institutions that are free from either short-term business demands or the “publish or perish” system of academic research. Such freedom will encourage more ambitious, riskier research that could potentially have a greater payoff for our field and for society in general. I hope that Answer.ai will prove to be a successful model that balances business concerns with the risk-taking and exploration that comes with great research. If Answer.ai is successful, this will give me hope in the future of long-term research.
> Legally, Answer.AI is a company. But in practice, it might hover somewhere between a lab and a normal “profit-maximizing firm” — as was the case with Edison’s lab. The founders seem perfectly content to pursue high-risk projects that might lead to failures or lack of revenue for quite a while. In saying this, I do not mean to imply they are content to light money on fire doing research with no chance of a return. Rather, they hope to fund a body of research projects that ideally have positive ROI in the long term. They are just not overly concerned with short-term revenue creation.
This may be the biggest weakness. One things the greatest R&D labs had was a big cash cow that paid the bills. Bell Labs was funded by AT&T’s massive phone monopoly. PARC was funded by Xerox’ massive business. Lockheed Martin’s Skunkworks were paid by massive DoD contracts.
One benefit answer.ai has over some of those labs is that it's building software, so expenses are mostly just salaries and GPU time - no need for massive capex on fabrication equipment or anything like that. And since we're focused on efficient fine-tuning and inference rather than foundation model training, even the GPU spend is likely to be extremely modest compared to some other players :)
The name is bad and the name is everything in tech. I thought it was some answers.com AI spinoff. This is far too generic and a google search of it is a nightmare. You don't buy your books on books.com, you buy them on amazon.com, you don't buy your pet supplies on pets.com, you buy them on chewy.com.
This makes me nostalgic for something that I never had and doesn't seem to exist anymore, labs unburdened by either the red tape of government or academia, or alternately, the pressure for results and profits that many startups seem to slaved to. Is there a 21st century equivalent of Bell Labs out there somewhere? Asking for a friend.
I believe Microsoft Research is the closest thing to the old Bell Labs or the old Xerox PARC that exists today, but I’ve heard that even MSR has changed (though this is just hearsay). I’ve heard historically researchers at MSR had a wide degree of freedom, and there has been many interesting projects coming out of there, my favorites being Singularity and Midori. Once again I don’t know how free the lab culture is, and I also don’t know the lab’s performance review criteria.
I’ve been reading lots of history books and interesting narratives. It seems like some sort of result-oriented pressure or a resource-based constraint has nearly always been the case with R&D and inventions. If Bell Labs is about laid back experimentation, I guess it’s more of an exception that proves the rule in these things. Even Google tends to feel the pressure of interest rates going up and lots of the more laid back stuff being thrown off as ”zero interest rate phenomenon” moonshot bets.
Eric's depiction of Edison and his rapid experimentation sounds very similar to the descriptions of one of Edison's contemporaries, Luther Burbank, as told in The Garden of Invention by Jane S. Smith. I recommend this book to the author (if he hasn't already read it himself) as I do to anybody else who enjoyed his essay.
There's a lot to be said for just exploring the combinatorial space of possibility in even a naive or brute force manner.
55 comments
[ 0.16 ms ] story [ 101 ms ] threadHow are you thinking about the role of patenting at Answer.AI? How about handling tech transfer issues with universities?
In terms of tech transfer, Jeremy has quite a bit of experience with collaborating with the best academic labs, especially in machine learning, so I expect we will do a lot of that kind of partnering.
Appreciate your writing by the way!
I'm hoping that not only will Answer.AI be successful, but that Eric's words in the conclusion will come to pass:
> "If their USD10 million experiment works, it has the chance to spark a rush of emboldened researchers and engineers to found small research firms, leveraging the models of the once-great dragons of American industrial R&D."
I'm happy to answer any questions you have about our experiment, and my co-founder Eric Ries (@eries) is around too. (Note that there are 2 Erics mentioned here -- that can get confusing!) By way of background, Eric Ries is the creator of the "Lean Startup", coined the term "Minimal Viable Product", and created the Long Term Stock Exchange, and I'm the co-founder of fast.ai, Enlitic, Kaggle, FastMail, and Optimal Decisions Group.
I think it would be awesome to be able to use GPUs from second-hand crypto mining rigs for cheaper fine-tuning research.
Good luck! As an enthusiast who loves The Lean Startup and who recently started playing around with LLMs, I'm super stoked about Answer.AI
I'll be rooting for you to succeed in this endeavor. I used to visit IBM ARC when I was a kid, and to my mind the academic environment is not quite the same. But as you know we really have not seen corporate R and D recover from the peaks it attained over fifty years ago.
Here is Ken talking about the concept
https://www.youtube.com/watch?v=lhYGXYeMq_E
https://www.youtube.com/watch?v=dXQPL9GooyI
“ collaboration can sometimes thwart innovation by tacitly forcing its participants into an objective-driven mindset.”
How do you plan to research potential problems to solve?
Would it be possible to publish and discuss those to-be-solved problems, so that hobbyists can also try their luck at solving them?
Initially, I thought that Kaggle competitions would be a great way to get the industry's problems into the hands of motivated problem-solvers. But I don't think I've ever seen a large car manufacturer (for example) as Kaggle competition host. So it looks like something is holding companies back.
If you’re not familiar with him, he was the guy that invented the NEAT algorithm, Novelty Search, etc.
In his book he talks about stepping stones and following what’s “interesting” to individuals. It seems like Answer.AI is focused on applied engineering, but any thoughts on this type of approach from a Research perspective?
This is certainly how we do things at Answer.AI -- we hire people that are passionate tinkerers, and encourage a playful and spontaneous approach. That doesn't mean there's no coordination or long-term goal, but rather that we view these short-term approaches as being a good way to make progress.
As I talk about in the article, Langmuir was able to pick from a bunch of problems in his wheelhouse…there were just conditions. And those conditions meant whatever area he picked might come with a high willingness to spend from GE! And if his work yielded results GE would have a way to quickly deploy the knowledge. All of which is great. To put his work in a box with what the Coolidge-types did is probably unfair. He was following curiosity under constraints, that’s all.
That is not to say all basic research roles in the world should look like that. But it makes sense given most basic researchers don’t fully understand which of all the problems they could happily pursue are actually most useful to industry. MIT professors of the early 1900s used to source research problems somewhat similarly.
Hopefully all of that helps a bit!
The big corporate labs were part of far larger manufacturing companies. It's not clear that a standalone lab can function in the same way.
[1] https://www.degruyter.com/document/doi/10.4159/harvard.97806...
[2] https://archive.org/details/menandvoltsstory00hammrich
Repurposing Bell-style systems engineers to isolate white hot problems is one example. Another is the general BBN-style approach which worked for them as a stand-alone R&D firm. I have stand alone pieces where I explore both of those quite a bit.
In general, I don’t think the concept of the playbooks of the great industrial R&D labs not working in firms un-attached to large firms holds much water.
I'm going to have to look at some of your other material.
Experimentation is different than other research or engineering but I think anything that amounts to a "system" can benefit from a systems problem solver who can perform as needed across the whole system.
Any organization is fortunate to have a person who can go without routine or typical workdays and do this.
It does seem like the bigger companies which have the most clearly defined roles can eventually get by on momentum without having anyone at all in a creative problem-solving position.
OTOH a small outfit can deploy unique problem-solving ability using only a handful of people that many corporations have none of. So the "very, very best people" may be what can get the momentum going to begin with given the right opportunity.
You wouldn't want it to function the same way as a modern corporate lab. Those are already making as much progress as they can by now.
Something worthwhile to do in a smaller organization is things that bigger labs need but don't do.
If you do it "right" a single-handed individual can make more promising creative progress than a whole team of corporate researchers can often end up with, but you need some pretty good breadth of innovation.
If you look carefully at the historical illustration, you may notice a pipe organ is the central device in the picture.
In the natural science lab I always liked to have different instruments that attract the attention of different types of visitors at different times.
And some instruments that are only developed or used in the presence of no visitors at all.
We also just launched Unbelievable Labs (unbelievablelabs.com), with the ambition to combine Physics-oriented venture building with applied research. It would be great to form a network of these to emulate some of the scale that Bell Labs had due to monopoly funding.
This may be the biggest weakness. One things the greatest R&D labs had was a big cash cow that paid the bills. Bell Labs was funded by AT&T’s massive phone monopoly. PARC was funded by Xerox’ massive business. Lockheed Martin’s Skunkworks were paid by massive DoD contracts.
https://www.microsoft.com/en-us/research/
There's a lot to be said for just exploring the combinatorial space of possibility in even a naive or brute force manner.