While there is no doubt that 'CD Index' has been declining over time, it is debatable whether it truly indicates a drop in 'disruption'.
Let's take AI for example. Research papers that were published in 2019-20 have become obsolete within a year or less, and consequently get cited less and less, as more recent and derivative papers introduced better models. Does that make the older papers any less 'disruptive'? No!
The first half of the 20th century had a lot of "low hanging fruit" in physics and chemistry that basically came out of the enlightenment. Things like electromagnetism, quantum mechanics. I wouldn't guess that's always the steady state for research.
The war may have had something to do with it too, I know a lot of advances (like radio / microwave engineering) have roots in the war efforts. But I suspect it's more of just a dry spell because we pushed forward in the 18th-20th century and ran out of ideas
Wouldn't we be presented with another bout of "low hanging fruits" in the second half of 21st century when tectonic shifts happen in quantum computing, superconductivity, ai, ar/vr/mr etc.?
Good article. Interesting finding at the very end:
>Although the proportion of disruptive research dropped significantly between 1945 and 2010, the number of highly disruptive studies has remained about the same.
So... maybe this is because having more people doing scientific research doesn't yield more breakthroughs. More people doing scientific research DOES yield more incremental achievements, as the study found, i.e. word frequency analysis showing more occurrences of "improve", "expand", "extend".
The number of highly disruptive studies remaining the same seems impressive enough to me. I expect diminishing returns to be the long-term norm in science. The low hanging fruit get picked first, and every breakthrough means one more concept that the next generation of researchers have to learn and incorporate into their work.
Alan Kay wrote an essay on how large scientific breakthroughs where done [1], further discussed in [2]. The funders and the systems of funding of science are to blame for the deciline, for example they won't fund problem finding anymore.
I blame the grant funding process. What gets published is what gets funded when written up as a grant proposal. What gets written up as a grant proposal is what is called for by grant funding agencies for their own research initiatives. If you want disruptive science, fund it then. That's all everyone wants to do in science, but the bills need to be payed somehow, so you play the game the funding agencies want you to play.
Plus, there is a large degree of "orthodoxy" in these grants, even most of them proclaim that they are "high-risk, high-gain". In the end, it seems that if you are too much on the "risk" scale, you will not get the funding since committees prefer "a little more of the same, please" to more wild ideas. I realise that such ideas are of course hard to classify in terms of their feasibility, but that's the main purpose of science, right? Venturing boldly into the unknown and all that; sometimes you come back with the treasure, sometimes you come back with lessons learned.
The current small-independent-grant culture is flat. This means that all projects are small and there are is almost no hierarchical 'higher level' science. PIs are happy to publish minor descriptions of the clouds, and journals are happy to publish them , in lack of something more impressive. So much value is lost in creating processes and bureaucracy.
The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
It almost seems as a new model would take the world by storm, but it doesnt seem to arise. Even private research often falls for the same pitfalls, like how Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward.
That's an apt way of describing the voter base :). Or the market at scale.
That's the major problem humanity faces. Individuals wielding concentrated power are dangerous, but they're also at least capable of thinking about something other than their immediate short-term interest. The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
If what you are saying was true then NIMBYism wouldn't be a problem, and we also wouldn't have any large "open spaces" left.
The problem is more parochialism than it is short-term thinking. And both sets of entities (powerful individuals and aggregate masses) are capable of parochialism.
> The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
I think this is a dangerous delusion:
Historically, centralized control has been a disaster because those individuals wielding power believed they could think of something greater… but actually lacked the skill. So we ended up with simplistic concepts that failed at scale, ideas that never could work because they missed key details, etc. By contrast, society gets on with the business of building the future in spite of and not due to that meddling by centralized power.
> The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
I think there's plenty enough of competition in academia. And competition alone doesn't help, because the metrics over which one loses or wins the race are not correlated with short or long-term worthiness. And while we can describe, even if somewhat vaguely, what makes worthy science, there's no way I know of to have a stable system in which that metric drives funding.
> Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward
This research at least has some feedback: Google funds it because it expects to make money off using its results in practice, so the research has to be at least somewhat correlated with reality. Unfortunately, this structure also means that "open ecosystem" isn't pursued.
If you're really smart, enough to make disruptive scientific discoveries, why bother doing that when you could learn programming and work at a FAANG for $400k, or work in the finance sector and make possibly more? If you stay in science, you'll earn peanuts and not even have any career stability since the academic track is so bad these days.
"why bother doing that when you could learn programming and work at a FAANG for $400k, or work in the finance sector and make possibly more?"
Because it's boring and depressing. And one thing money can't buy, unless you're solely interested in accumulating money, is satisfaction with your life.
From my training as a computer scientist, I am used to the idea that some problems are easy and some are plainly intractable. Perhaps that's also true in science and technology, and we are starting to exhaust all the easy directions.
Still, many less techy problems remain, at least climate change and wealth inequality come to mind. Perhaps we'll see the biggest innovations in solving those in the future.
As a software developer, I’d want to see a rewrite to clean up the legacy bodges before we conclude we can’t add more.
Also, wealth inequality isn’t a problem: how wealthy the people at the bottom are is. Wealth inequality is associated with fewer poor people and a higher standard of living because wealth inequality requires a wealthy society.
I'm not suggesting we can't add more, rather, that I find it conceivable that there is a barrier of technological progress that we can approach, but never reach. So the pace of progress will slow down.
Concerning wealth inequality, I do think it is a fundamental problem of society if a fraction of a percentage controls majority of the available resources even if the lowest half has their immediate needs met, but we can disagree on that all day and arrive nowhere.
It's indeed not an easy issue, which is why I think there could still be lots of innovations to made in that space in terms of economics, politics, social sciences etc.
The GINI index in absolute monarchies is close to 1.
Wealth inequality implies that some people disproportionately control the life production of other people. If you're in favor of a "great man" idea of progress this is a benefit. If you're in favor of individual power to direct production and innovation (along the lines of de Tocqueville or it's left-wing equivalent) this is a detraction.
Climate change seems like one of the techiest of problems.
I think wealth inequality is too ill defined to even be a problem. If anything i would argue it is only a problem if class mobility is low and that is where the focus should be.
I wonder if this is to do with major vs minor breakthrough.
Take physics. As I read it, modern fundamental physics is mostly some combination of general relativity and quantum mechanics, which are both early 20th century inventions.
They weren't necessarily genius in particular in-depth analysis, rather they were genius new concepts that created a whole new blank canvas to fill in.
What did the field look like just before them? I always forget the details, but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done? I wonder if it looked similarly uninnovative. The canvas they had was full, and they needed a new canvas for true innovation.
Now the thing is, I think true disruption, like GR and quantum physics, is not limited by the number of postdocs, but by truly genius insights. Einstein wasn't even working as an academic when he came up with Special Relativity. So the number of papers written isn't a proxy for pace of revolution.
Not to mention, since number of published papers has become a target, it has become useless as a measure of progress.
I agree, I think we're discounting genius insight as the primary driver of innovation and ground breaking science, and we can't expect genius insight to occur in some predictable time frame or a time frame even within a lifespan.
I think perhaps we've also partially lost the meaning of ground breaking research given the constant news articles on "ground breaking science" and the constant overinflation and exaggeration of one's papers and achievements.
A lot of the obvious questions on the table were answered, even without an answer we at least have the question. Today true, groundbreaking innovation needs someone to identify the new questions. It's like doing science in a bubble and you need someone to take a massive new one and merge it into our own. Right now we're not opening wide doors, but just refining and adding almost imperceptibly to an already huge bubble. We're grinding in the game because we can't find the boss on the map or can't beat it yet.
Besides the current structure of research and the known weaknesses (misaligned incentives, funding issues, etc.) science in general today ran out of low hanging fruit. The last unexplored or unanswered riddles may need prerequisites that are outside of the current boundaries so someone needs to open the door do a vast new bubble that has more relatively low hanging fruit.
Also humans probably (don't quote me on that :) ) can't come up with any new questions or answers unless they are already at the edge of that bubble. And they each see and understand a smaller patch of that surface as it expanded. Stumbling onto new thinks is exponentially harder.
This is what those revolutionary discoveries did for us in the past: Opened the door to a new world where the hard answers were suddenly a bit easier, they made sense.
> we can't expect genius insight to occur in some predictable time frame
Yes, and we can’t expect it to occur in a socially confined over-structured environment either. Academia has changed a lot since then, with peer review hell, publishing pressure, etc. I mean, are we really pretending to be surprised that a system of incrementalist incentives produces incrementalist results?
We get around this in part by not understanding (or at least remembering our understanding of) many of the basics. Such as this guy who looked things up in a textbook: https://news.ycombinator.com/item?id=34236889
It also seems unlikely that a patent clerk, even one with a PhD in physics, would be allowed to publish in a physics journal today. The gatekeeping has gotten a lot more strict.
I’m not sure it filters out kook theories, given the replication crisis.
Rather, journals promote well-connected kooks at the expense of less well-connected kooks and happen to occasionally publish some science as a byproduct.
So "peer review" isn't holding back kook theories at all on its own. At best, journal prestige is bolstered by it, which the journal can then translate into holding them back. The track record is a bit unclear, though.
I view peer review as respecting the time of the journal subscriber. Someone filters out the things that are obviously flawed before wasting everyone else's time.
Journal aren't just a magic upload into scientific consciousness; it takes time and energy to read and process them.
It's doing a rather bad job at filtering out obvious flaws, given the reproducibility crisis we've got on our hands. Everyone says peer review is very important, but when it comes to reviewers putting in the hard and boring work of actually checking the raw data, almost none do, so in truth, peer review is NOT important to most scientists.
And we nevertheless pay the price of adhering closer to orthodoxy for it, since effort being put in is a rather superficial "this paper sounds reasonable to me" filter, perhaps with some boxes that must be checked like p-value or sample size.
A peer review young earth creationist journal would still gatekeep anything that did not come from a young earth perspective.
>Papers can be in any scientific, or social scientific, field, but must be from a young-earth perspective and aim to assist the development of the creation model of origins
The problem is that inherently peer review is designed to act as a brake:
>Peer review, in which a scientist's paper is scrutinized by a group of colleagues, is designed to find errors and weed out half-baked ideas.
But in blocking you are going to reject ideas.
>And although some have criticized peer review for rejecting new ideas just because they're too radical for the establishment to stomach
How much of a problem and how many revolutionary ideas are stopped, I don't know. I am more concerned about the reproducibility problem in science.
Sorry, bad phrasing on my part. I didn't mean that peer reviewing doesn't lead to gatekeeping, but that simply making something peer reviewed isn't enough, because ludicrously bad attempts at science can be peer reviewed.
Peer review is also obviously a bad solution for the reproducibility problem, given that we've got the problem now, concurrent with generations of peer reviewed manuscripts.
This article mentions Einstein being offended that his paper was submitted to peer review by Physical Review, and submitting it elsewhere instead:
"We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere."
which argues, not unconvincingly IMO, that peer review is a failed experiment which tries to stop bad science at the price of also stopping brilliant, unconventional science, where instead we would profit more from letting the scientific public do the sorting instead of hand-picked specialists. The author phrases it as science being as strong as its strongest links, not as weak as the weakest one, and we should do everything to get the brilliant crazy ideas.
In brief, London 'man off the street' solves 20k year Ice Age drawings mystery, published in Cambridge Archaeological Journal via contact with Durham professor. (And he is first author.)
I don't say that to disagree with you by the way - it's undoubtedly a rare example.
These days Einstein could just put his paper on arXiv and worry about publication later. The physics community goes nuts about arXiv results all the time. There's zero review if you have a university email (or national lab etc).
I tend to agree that peer review has flaws. But given the ease with which someone can publish anything with no review whatsoever it seems reasonable that peer review might shift to fill a niche that is slower and less disruptive.
I disagree, at least in math there have been many instances recently where laymen (with a PhD) found some new and exciting results[0] and got to publish them, and I don't see why it would be any other way in physics.
> but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done?
This is apparently a common misconception regarding a lecture given by Lord Kelvin in 1900 concerning "two clouds" in physics.[1] In fact, Max Planck solved the ultraviolet catastrophe, also in 1900, by assuming electromagnetic radiation can only be emitted or absorbed in discrete packets of energy called quanta. Albert Einstein also solved the ultraviolet catastrophe in a paper he published in 1905, for which he was awarded the Nobel Prize in 1922, by, as always, standing on the shoulders of giants, hypothesizing that Planck's quanta were actual particles. These particles are known today by a word coined by physical chemist Gilbert Newton Lewis in a letter to Nature in 1926, namely, photons.
I think that the expectation that "ground-breaking" discoveries in science should follow some linear or predictable timetable is not reasonable. Is the time period between "ground-breaking" discoveries supposed to be every 5 years, every 10 years?
The volume of papers has increased significantly, and publish or perish kinda stinks, but it's not just an issue of funding, but some (many?) researchers publish and exaggerate about the importance and difficulty of their research to receive an "I am smart" badge on social media. Although, despite this increase in volume of crap tier papers, the article seems to think it's not correlated:
"Declines in disruptiveness are also not attributable to changing publication, citation or authorship practices..."
What you're pointing to is far more insidious than I think is usually recognized.
People can point to incentive structures like grants etc, and to the implications in terms of the meaning of metrics etc but the real harm is in how it shifts mental focus and attention collectively.
It's not ok to let minor things or things you think are obvious go anymore to focus on bigger picture issues, that might be "riskier" but nonobvious. The way that plays out too is incredibly dependent on your institutional social environment etc.
So much of this is difficult to easily quantify in ways that can be easily studied. I loved this paper for example, but it's easy for me to think of papers in my own field that would look "disruptive" in terms of citation networks but are really the same content. There's these weird shifts I've seen happen in reading old literature and during my career, where big shifts in who is being cited will happen for quixotic social reasons. Usually it's basically politics or ignorance of big parts of a field, who will get introduced to an idea by a particular author or group. Lots of chaotic citation patterns and feedback loops.
Add in shifts in what's motivating grant and paper topics and it really damages authentic scientific discourse. So much is driven by looking like a brilliant scientist and not by scientific progress. People aren't dumb either and they are very very good at looking like brilliant scientists.
Higgs, who hypothesized the Higgs boson back in the 70s, has only published 2-3 papers in the decades since. Before it was announced that he won the Nobel prize, his University was deciding between forcing him out or gambling on him potentially winning the Nobel prize which would bring more reknown to the department. He has commented recently saying if he were a young, freshly minted PhD today he would not have gotten tenure with his publication record. That seems insane to me and a bitter critique of today's incentive structures created by the funding system we have that rewards incremental work over real breakthroughs.
This reminds me of a comment from an academic nearing retirement that I heard second-hand. It was something to the effect that applicants for Assistant Professor (or Lecturer) positions have CVs that are as good as or better than what applications for tenure looked like 20-30 years ago.
- A "Pain point" focus would tend to be more incremental than truly disruptive
On the good side there may be some decrease of disruptive inventions and discoveries because we are also focusing on eliminating the externalities of current and previous inventions and discoveries. It's a lot easier to make wildly new things if you aren't worried about their side effects.
This is a big problem. PARC, Bell Labs, IBM Research are the first that come to mind. Same is true for mostly government-funded initiatives like the one where Marc Andreessen invented Mosaic browser, that eventually became Netscape Navigator. I don't recall the project name. Do we even do that anymore?
Patents went from real innovation to Legalese BS and market protection in the past decades.
Also I suspect most innovation are being done outside of papers, especially when you have "reviewer 2" ready to be nitpicky at every minor detail, so you make it more bland and maybe skip one juicy part or another
I get that in this context "patents" should be seen as the announcement of a new innovation, a surrogate of the innovation itself if you will, but setting that aside for a moment I'm struggling to imagine how patents could ever do anything but slow down disruption.
Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does. Shouldn't the former always be more disruptive of the status quo? I can imagine this is an oversimplification though so I'm open to hearing counterintuitive examples where the opposite is true.
And to be clear, either way I'm not saying this is good or bad in and of itself - in some cases it might slow down essential improvements, in others it might be good to have speed limits for change so that things don't break.
You could argue that there is more economic incentive to invent new things as discoveries are worth more if you alone have the right to use them. This in turn could lead to more investments in research that lead to more inventions that increase disruption.
I know that that has been one argument in favor of patents since forever. But that ignores the part where competing parties are not freely allowed to build on another person's patent, which is essential for making good improvements.
And you can see this play out in the history of invention. For example, Edison stopped other people from doing research on the light bulb. Meanwhile the Netherlands was a country without IP laws at the time so Phillips quickly ended up producing the superior and cheaper product. No patent laws, bigger disruption. Similarly the Wright brothers were notorious for holding back aviation with their patents.
James Watt and his patents on steam engine are also an interesting case. Efficiency gains basically stopped for duration of the main patent patent and exploded later.
On the other hand it is also case for more innovation due to necessity, e.g. his patent on beam connection to a shaft (linear-to-rotary movement) required others to come up with a different solutions (e.g. sun and planet gear).
- There are as many revolutionary discoveries with and without patents
- Without patents, discoveries would be freely avalaible
As far as I know, two (related) arguments are generally made for patents:
- Patents create an indirect (by preventing the competition from using your invention) or direct (by licensing it) monetary return to innovation, potentially leading to more innovation
- If a company wants competitors not to copy their innovation, they can 1. keep it secret or 2. disclose it and patent it; without patent the choice is between 1. keep it secret or 2. disclose it and have everybody copy them. In this case, patents lead to more innovation being made freely avalaible (with a delay!).
Whether patents lead to more or less innovation is, as far as I know, contentious.
> There are as many revolutionary discoveries with and without patents
I explicitly don't: I mentioned the patent stops others from innovating on top of the patent. The necessity of patents to be public is one I had not considered though, being a force against trade secrets is a good counterpoint.
> Whether patents lead to more or less innovation is, as far as I know, contentious.
Honestly, I really have trouble accepting this. Not that I have an answer, but with the idea that researchers haven't been able to find some sensible way of measuring this by now. I already mentioned the example of light bulbs. Surely there are enough similar historical scenarios available to analyze where one can make use of global differences in IP laws and other variables to simulate control groups?
"Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. [...]"
You take the situation where there _is_ a revolutionary discovery, with or without patents, and then wonder about the effect of patents on the next innovations. In doing that you do not consider that may be a revolutionary discovery with patents, and none without.
Extruded plastic took off before resin printers because the patents expired a few years earlier.
As far as I can tell, we could have had the 3D printing revolution in the 90s or early 2000s, but instead we locked the technology away to only be used in a few esoteric commercial applications until the patents expired.
Given the abundance of cheap eink book readers, price tags etc on the market, cost doesn't seem to be a problem. OTOH the limitations of the technology - in particular, the contrast and the refresh rate - and how expensive large screens get, prevents it from expanding into more niches.
Perhaps if there were no patent, there'd be more third-party research to improve all these things faster. But as far as adoption of the existing stuff goes, I don't see how license costs are a direct blocker.
The original idea of a patent is that it must reveal the "secrets" behind an invention, in return for a "temporary monopoly". It's supposed to help diffusion of technology by publishing trade secrets, not to "incentivise" research.
I think what is actually happening is that patents are made obtuse/opaque containing little meaningful information. They are then used as playing cards in patent licensing deals between big corporations.
There is no longer any hope for patent clerks to do meaningful inspection of patents to determine if they are legitimate. Instead, it's left to lawyers/attorneys to play something out in court.
Patents have always been written in precise but vague terms. If you take that as the language and understand the structure of them, they can be quite readable, although you will often need to use outside knowledge and critical thinking to determine how to actually replicate the invention.
I lost a lot of faith in scientific research when I had to spend time reviewing patents in an early job.
The fact that I was looking into battery materials (a notoriously over-hyped sector) certainly didn't help, but some of the approved parents were laughable. They included core materials which had been historically known to be useful in batteries (and hence shouldn't qualify for protection), but then also every possible element-wise substitution of the same structure - many of the resulting materials would not even have been stable, never mind viable as a battery component. There were also instances of multiple patents covering the same material, suggesting that any prior-art searches were not being done rigorously.
> EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does.
Everyone is free to innovate on top of any pre-existing, current patent. They simply can't profit on their new innovation without licensing the patent. However what they can do with their new innovation is patent the innovation itself. This newly patented innovation prevents the original patent holder from profiting off of the new innovation, and thus incentivizes the original patent holder to license their patent to the new patent holder (or vice versa).
When I worked at IBM, we were given generous cash incentives for filing patents. I am named as an inventor on about 40 patents. To be honest, only one of them was a genuine invention in my opinion.
Its just the evolution vs revolution periods...
In any field 99% of the time advancements are slow and incremental, than a new idea sparks a revolution and you get a lot of advancements in a short period of time. This is just the nature of scientific progress.
In the early 20 century new ideas in physics and mathematics sparked a revolution and we are slowly unraveling the benefits of it ever since.
For 40 years we had evolution in the AI research field, now we have a revolution, I do not know when it will die down but after a while we will just start to build over the ideas we discovered and a long period will pass before we do another jump.
Same with psychology, economics, biology...
A big part is that science has become very “sanitized”. Work on the wrong issues and you won’t get funding or will even get ridiculed out of a career.
It’s not even just going against headline consensus issues that will get you in trouble. Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
More than that though. Modern science research is a highly hierarchal and managed enterprise. It's led, or at least its funding is controlled by, the risk averse managerial class who can only think it terms of returns on investment.
9/11 (believe it or not) had a lot to do with that in IT/computer engineering research (and likely other fields) with government funding. After the attacks, the US Federal government moved to stressing development more than blue sky research. That drew all the funding away from fundamental science to projects that could be made into a product in 3-5 years and focused on addressing some issue of the War on Terror.
Would you say this is a natural consecuence of human emotions (e.g ego), economic interests, powerful players that have corrupted the game etc? I'm genuintely curious. As an ignorant on science and its processes in academia or markets, I cannot feel nothing other than dissapointment and loss of hope everytime I heard this kind of things about a matter I've always considered quite rational and focused on pursuing the truth over anything else. Naive me, I guess...
It’s always been a problem but I’d say these factors make it worse:
1. It’s harder for individuals or small groups to make breakthroughs. We need larger groups and more expertise as well as expensive equipment to continue making discoveries.
More hands in the pie make more gatekeepers and less risk taking.
2. In general our entire society is “circling the wagons”. I’m not sure why but we’re more tribal than normal. Science isn’t immune from the trend.
This is a very natural consequence of the higher difficulty of maintaining one's career in academia. What is a natural consequence of the increased competition for those jobs.
> Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
Hasn't that always been the case?
It's said science advances one grave at a time.
I suspect that there isn't a single cause but a cluster of them.
One possible issue is that high level science needs more and more energy (like LHC) and more and more intelligence to crack (most science prizes are teams).
Alternative theory: we've pretty much nailed the basics and it takes a PhD before people can even understand the frontier of most scientific fields. These frontiers are manifold, highly technical, and extremely boring to the layperson. Number theory is a very clear example of this: go read the Birch and Swinnerton-Dyer conjecture. Come back in a decade after you understand the Tate–Shafarevich group. (hahahaha just kidding, if you understood that you'd have proved the conjecture)
I’ve heard this too about number theory. Perhaps in that field, the paradigm shift lies in reducing incidental complexity? Or even managing it in a way that is better for humans?
> it takes a PhD before people can even understand the frontier of most scientific fields
It’s interesting that we have this implicit view that hyper-specialization is the only way to advance. I always thought so too, but why really? If you think about it, specialization is very close to incrementalism. Groundbreaking paradigm-shift stuff can often be written down on a napkin, and rarely does it require a PhD.
That said, to find treasure you have to go down a LOT of wrong paths before you (or someone else) ends up in the right one.
This was considered in the paper: "Some point to a dearth of ‘low-hanging fruit’ as the readily available productivity-enhancing innovations have already been made19,27. Others emphasize the increasing burden of knowledge; scientists and inventors require ever more training to reach the frontiers of their fields, leaving less time to push those frontiers forward18,28. "
Doesn't it just stand to reason that there's less to discover, that what there is mostly isn't the low-hanging fruit (and unlike actual fruit, it's not regrowing), over time?
That's what I was thinking, too. Perhaps we already know majority of what there is to know, when it comes to fundamental concepts? Probably plenty of work left in improving our tools and engineering solutions, like machine learning-based software, as well as in understanding the intricates of biology. Maybe mathematics, too. Maybe improvements in these will cause another golden age of discovery. Like, understanding biology enough to gain significantly extended lifespan, which means much more expertise can be built by one individual.
On the paper front, I have a number of published scientific papers with citations. My 3 most important papers—that will go down as turning points in the history of computer science—have a combined 0 citations, and 2 of them not only were not in a journal, but they were even flagged and removed from arxiv.
On the patent front, well patents are for parasites.
> The intuition is that if a paper or patent is disruptive, the subsequent work that cites it is less likely to also cite its predecessors
This intuition seems to overlook the culture of citations in the patent space. There, the incentive is for applicants to be over inclusive with citations. This is combined with the increasing ease with which references can be found.
There’s an intuition that recent products and inventions are more incremental and that historical innovations are more revolutionary- in the sense of lacking precedence.
But this is an effect of the simplification of history, and when you read the detailed technical histories you can see how remarkably incremental the old innovations were.
We can easily see the precedents of recent innovations but not for older ones.
This is pointed out in the book I am reading on the development of the turbojet, which only looks more unprecedented than it really was.
I've heard another interesting argument, at least for disruptive science/research from industry.
Lower corporate taxation incentivizes the hoarding of cash. Prior, corporations were incentivized to allocate more funding towards their industry research (IBM, Bell Labs, etc).
171 comments
[ 3.0 ms ] story [ 248 ms ] threadLet's take AI for example. Research papers that were published in 2019-20 have become obsolete within a year or less, and consequently get cited less and less, as more recent and derivative papers introduced better models. Does that make the older papers any less 'disruptive'? No!
The war may have had something to do with it too, I know a lot of advances (like radio / microwave engineering) have roots in the war efforts. But I suspect it's more of just a dry spell because we pushed forward in the 18th-20th century and ran out of ideas
https://www.nature.com/articles/d41586-022-04577-5
>Although the proportion of disruptive research dropped significantly between 1945 and 2010, the number of highly disruptive studies has remained about the same.
So... maybe this is because having more people doing scientific research doesn't yield more breakthroughs. More people doing scientific research DOES yield more incremental achievements, as the study found, i.e. word frequency analysis showing more occurrences of "improve", "expand", "extend".
[1] http://www.vpri.org/pdf/Kay_How.pdf
[2] https://m.youtube.com/watch?v=j9ZGFaIHegE
The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
It almost seems as a new model would take the world by storm, but it doesnt seem to arise. Even private research often falls for the same pitfalls, like how Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward.
That's the major problem humanity faces. Individuals wielding concentrated power are dangerous, but they're also at least capable of thinking about something other than their immediate short-term interest. The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
The problem is more parochialism than it is short-term thinking. And both sets of entities (powerful individuals and aggregate masses) are capable of parochialism.
I think this is a dangerous delusion:
Historically, centralized control has been a disaster because those individuals wielding power believed they could think of something greater… but actually lacked the skill. So we ended up with simplistic concepts that failed at scale, ideas that never could work because they missed key details, etc. By contrast, society gets on with the business of building the future in spite of and not due to that meddling by centralized power.
I think there's plenty enough of competition in academia. And competition alone doesn't help, because the metrics over which one loses or wins the race are not correlated with short or long-term worthiness. And while we can describe, even if somewhat vaguely, what makes worthy science, there's no way I know of to have a stable system in which that metric drives funding.
> Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward
This research at least has some feedback: Google funds it because it expects to make money off using its results in practice, so the research has to be at least somewhat correlated with reality. Unfortunately, this structure also means that "open ecosystem" isn't pursued.
All these geniuses... people are giving them jobs!
Because it's boring and depressing. And one thing money can't buy, unless you're solely interested in accumulating money, is satisfaction with your life.
Obviously not true for everyone, but we've got 8.8 million "scientists" (I'm not sure what they mean by the word). https://sciencebusiness.net/news/number-scientists-worldwide...
Most scientists don't work in academia. Some of those who do have stable jobs as staff instead of faculty positions.
And this guy did it in his spare time while working at FAANG: https://news.ycombinator.com/item?id=34236889
Still, many less techy problems remain, at least climate change and wealth inequality come to mind. Perhaps we'll see the biggest innovations in solving those in the future.
Also, wealth inequality isn’t a problem: how wealthy the people at the bottom are is. Wealth inequality is associated with fewer poor people and a higher standard of living because wealth inequality requires a wealthy society.
What exactly are you “fixing”?
Concerning wealth inequality, I do think it is a fundamental problem of society if a fraction of a percentage controls majority of the available resources even if the lowest half has their immediate needs met, but we can disagree on that all day and arrive nowhere.
My concern with your view is that in trying to “squash the gap”, you end up doing so by making everyone poorer — and society worse overall.
Happy to agree to disagree; hopefully we can consider each other’s perspectives in crafting policies.
Wealth inequality implies that some people disproportionately control the life production of other people. If you're in favor of a "great man" idea of progress this is a benefit. If you're in favor of individual power to direct production and innovation (along the lines of de Tocqueville or it's left-wing equivalent) this is a detraction.
I think wealth inequality is too ill defined to even be a problem. If anything i would argue it is only a problem if class mobility is low and that is where the focus should be.
Take physics. As I read it, modern fundamental physics is mostly some combination of general relativity and quantum mechanics, which are both early 20th century inventions.
They weren't necessarily genius in particular in-depth analysis, rather they were genius new concepts that created a whole new blank canvas to fill in.
What did the field look like just before them? I always forget the details, but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done? I wonder if it looked similarly uninnovative. The canvas they had was full, and they needed a new canvas for true innovation.
Now the thing is, I think true disruption, like GR and quantum physics, is not limited by the number of postdocs, but by truly genius insights. Einstein wasn't even working as an academic when he came up with Special Relativity. So the number of papers written isn't a proxy for pace of revolution.
Not to mention, since number of published papers has become a target, it has become useless as a measure of progress.
I think perhaps we've also partially lost the meaning of ground breaking research given the constant news articles on "ground breaking science" and the constant overinflation and exaggeration of one's papers and achievements.
Besides the current structure of research and the known weaknesses (misaligned incentives, funding issues, etc.) science in general today ran out of low hanging fruit. The last unexplored or unanswered riddles may need prerequisites that are outside of the current boundaries so someone needs to open the door do a vast new bubble that has more relatively low hanging fruit.
Also humans probably (don't quote me on that :) ) can't come up with any new questions or answers unless they are already at the edge of that bubble. And they each see and understand a smaller patch of that surface as it expanded. Stumbling onto new thinks is exponentially harder.
This is what those revolutionary discoveries did for us in the past: Opened the door to a new world where the hard answers were suddenly a bit easier, they made sense.
Yes, and we can’t expect it to occur in a socially confined over-structured environment either. Academia has changed a lot since then, with peer review hell, publishing pressure, etc. I mean, are we really pretending to be surprised that a system of incrementalist incentives produces incrementalist results?
It’s amazing upright apes can figure out the speed of light or postulate and find the Higgs Boson in the first place.
How much physics do we expect the smartest dog in the world to do? What if we 10x’d its lifespan?
At some point either:
1. There is no more physics
2. We build beings that can do physics that we can’t
3. We’re stopped dead in our tracks at the limit of our ability to comprehend things that apes weren’t built to comprehend
A bunch of blind people could theoretically construct a model of an elephant, if they work together well. https://en.wikipedia.org/wiki/Blind_men_and_an_elephant
It would be cool to have a brain that sees time, just as an incidental consequence of what’s happening in higher dimensions.
Rather, journals promote well-connected kooks at the expense of less well-connected kooks and happen to occasionally publish some science as a byproduct.
So "peer review" isn't holding back kook theories at all on its own. At best, journal prestige is bolstered by it, which the journal can then translate into holding them back. The track record is a bit unclear, though.
Journal aren't just a magic upload into scientific consciousness; it takes time and energy to read and process them.
And we nevertheless pay the price of adhering closer to orthodoxy for it, since effort being put in is a rather superficial "this paper sounds reasonable to me" filter, perhaps with some boxes that must be checked like p-value or sample size.
The problem is that inherently peer review is designed to act as a brake: >Peer review, in which a scientist's paper is scrutinized by a group of colleagues, is designed to find errors and weed out half-baked ideas.
But in blocking you are going to reject ideas. >And although some have criticized peer review for rejecting new ideas just because they're too radical for the establishment to stomach
How much of a problem and how many revolutionary ideas are stopped, I don't know. I am more concerned about the reproducibility problem in science.
Peer review is also obviously a bad solution for the reproducibility problem, given that we've got the problem now, concurrent with generations of peer reviewed manuscripts.
This article mentions Einstein being offended that his paper was submitted to peer review by Physical Review, and submitting it elsewhere instead:
"We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere."
I just recently ended up there via this blog: https://experimentalhistory.substack.com/p/the-rise-and-fall...
which argues, not unconvincingly IMO, that peer review is a failed experiment which tries to stop bad science at the price of also stopping brilliant, unconventional science, where instead we would profit more from letting the scientific public do the sorting instead of hand-picked specialists. The author phrases it as science being as strong as its strongest links, not as weak as the weakest one, and we should do everything to get the brilliant crazy ideas.
Also currently on the front page: https://news.ycombinator.com/item?id=34257150
In brief, London 'man off the street' solves 20k year Ice Age drawings mystery, published in Cambridge Archaeological Journal via contact with Durham professor. (And he is first author.)
I don't say that to disagree with you by the way - it's undoubtedly a rare example.
I tend to agree that peer review has flaws. But given the ease with which someone can publish anything with no review whatsoever it seems reasonable that peer review might shift to fill a niche that is slower and less disruptive.
Most recent example: https://www.quantamagazine.org/long-out-of-math-an-ai-progra...
This is apparently a common misconception regarding a lecture given by Lord Kelvin in 1900 concerning "two clouds" in physics.[1] In fact, Max Planck solved the ultraviolet catastrophe, also in 1900, by assuming electromagnetic radiation can only be emitted or absorbed in discrete packets of energy called quanta. Albert Einstein also solved the ultraviolet catastrophe in a paper he published in 1905, for which he was awarded the Nobel Prize in 1922, by, as always, standing on the shoulders of giants, hypothesizing that Planck's quanta were actual particles. These particles are known today by a word coined by physical chemist Gilbert Newton Lewis in a letter to Nature in 1926, namely, photons.
[1] https://arxiv.org/pdf/2106.16033.pdf
The volume of papers has increased significantly, and publish or perish kinda stinks, but it's not just an issue of funding, but some (many?) researchers publish and exaggerate about the importance and difficulty of their research to receive an "I am smart" badge on social media. Although, despite this increase in volume of crap tier papers, the article seems to think it's not correlated:
"Declines in disruptiveness are also not attributable to changing publication, citation or authorship practices..."
People can point to incentive structures like grants etc, and to the implications in terms of the meaning of metrics etc but the real harm is in how it shifts mental focus and attention collectively.
It's not ok to let minor things or things you think are obvious go anymore to focus on bigger picture issues, that might be "riskier" but nonobvious. The way that plays out too is incredibly dependent on your institutional social environment etc.
So much of this is difficult to easily quantify in ways that can be easily studied. I loved this paper for example, but it's easy for me to think of papers in my own field that would look "disruptive" in terms of citation networks but are really the same content. There's these weird shifts I've seen happen in reading old literature and during my career, where big shifts in who is being cited will happen for quixotic social reasons. Usually it's basically politics or ignorance of big parts of a field, who will get introduced to an idea by a particular author or group. Lots of chaotic citation patterns and feedback loops.
Add in shifts in what's motivating grant and paper topics and it really damages authentic scientific discourse. So much is driven by looking like a brilliant scientist and not by scientific progress. People aren't dumb either and they are very very good at looking like brilliant scientists.
- Tenure requirements
- Job duties (both requirements and limitations)
- Elimination of corporate skunk works like PARC
- Credentialism
- A "Pain point" focus would tend to be more incremental than truly disruptive
On the good side there may be some decrease of disruptive inventions and discoveries because we are also focusing on eliminating the externalities of current and previous inventions and discoveries. It's a lot easier to make wildly new things if you aren't worried about their side effects.
This is a big problem. PARC, Bell Labs, IBM Research are the first that come to mind. Same is true for mostly government-funded initiatives like the one where Marc Andreessen invented Mosaic browser, that eventually became Netscape Navigator. I don't recall the project name. Do we even do that anymore?
A variety of DOE national labs and core facilities (including NCSA where Mosaic was developed) still exist and are funded.
We may need more to take up the slack of the corporate research facilities.
My mind is genuinely struggling...
Certainly not Curie, Einstein, Fleming worked in a hospital.
Maybe the problem is...
Undergraduates!
Also I suspect most innovation are being done outside of papers, especially when you have "reviewer 2" ready to be nitpicky at every minor detail, so you make it more bland and maybe skip one juicy part or another
Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does. Shouldn't the former always be more disruptive of the status quo? I can imagine this is an oversimplification though so I'm open to hearing counterintuitive examples where the opposite is true.
And to be clear, either way I'm not saying this is good or bad in and of itself - in some cases it might slow down essential improvements, in others it might be good to have speed limits for change so that things don't break.
And you can see this play out in the history of invention. For example, Edison stopped other people from doing research on the light bulb. Meanwhile the Netherlands was a country without IP laws at the time so Phillips quickly ended up producing the superior and cheaper product. No patent laws, bigger disruption. Similarly the Wright brothers were notorious for holding back aviation with their patents.
On the other hand it is also case for more innovation due to necessity, e.g. his patent on beam connection to a shaft (linear-to-rotary movement) required others to come up with a different solutions (e.g. sun and planet gear).
- There are as many revolutionary discoveries with and without patents
- Without patents, discoveries would be freely avalaible
As far as I know, two (related) arguments are generally made for patents:
- Patents create an indirect (by preventing the competition from using your invention) or direct (by licensing it) monetary return to innovation, potentially leading to more innovation
- If a company wants competitors not to copy their innovation, they can 1. keep it secret or 2. disclose it and patent it; without patent the choice is between 1. keep it secret or 2. disclose it and have everybody copy them. In this case, patents lead to more innovation being made freely avalaible (with a delay!).
Whether patents lead to more or less innovation is, as far as I know, contentious.
I explicitly don't: I mentioned the patent stops others from innovating on top of the patent. The necessity of patents to be public is one I had not considered though, being a force against trade secrets is a good counterpoint.
> Whether patents lead to more or less innovation is, as far as I know, contentious.
Honestly, I really have trouble accepting this. Not that I have an answer, but with the idea that researchers haven't been able to find some sensible way of measuring this by now. I already mentioned the example of light bulbs. Surely there are enough similar historical scenarios available to analyze where one can make use of global differences in IP laws and other variables to simulate control groups?
"Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. [...]"
You take the situation where there _is_ a revolutionary discovery, with or without patents, and then wonder about the effect of patents on the next innovations. In doing that you do not consider that may be a revolutionary discovery with patents, and none without.
Extruded plastic took off before resin printers because the patents expired a few years earlier.
As far as I can tell, we could have had the 3D printing revolution in the 90s or early 2000s, but instead we locked the technology away to only be used in a few esoteric commercial applications until the patents expired.
How much is the license cost?
Perhaps if there were no patent, there'd be more third-party research to improve all these things faster. But as far as adoption of the existing stuff goes, I don't see how license costs are a direct blocker.
The original idea of a patent is that it must reveal the "secrets" behind an invention, in return for a "temporary monopoly". It's supposed to help diffusion of technology by publishing trade secrets, not to "incentivise" research.
I think what is actually happening is that patents are made obtuse/opaque containing little meaningful information. They are then used as playing cards in patent licensing deals between big corporations.
There is no longer any hope for patent clerks to do meaningful inspection of patents to determine if they are legitimate. Instead, it's left to lawyers/attorneys to play something out in court.
The fact that I was looking into battery materials (a notoriously over-hyped sector) certainly didn't help, but some of the approved parents were laughable. They included core materials which had been historically known to be useful in batteries (and hence shouldn't qualify for protection), but then also every possible element-wise substitution of the same structure - many of the resulting materials would not even have been stable, never mind viable as a battery component. There were also instances of multiple patents covering the same material, suggesting that any prior-art searches were not being done rigorously.
Everyone is free to innovate on top of any pre-existing, current patent. They simply can't profit on their new innovation without licensing the patent. However what they can do with their new innovation is patent the innovation itself. This newly patented innovation prevents the original patent holder from profiting off of the new innovation, and thus incentivizes the original patent holder to license their patent to the new patent holder (or vice versa).
In the early 20 century new ideas in physics and mathematics sparked a revolution and we are slowly unraveling the benefits of it ever since. For 40 years we had evolution in the AI research field, now we have a revolution, I do not know when it will die down but after a while we will just start to build over the ideas we discovered and a long period will pass before we do another jump. Same with psychology, economics, biology...
It’s not even just going against headline consensus issues that will get you in trouble. Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
It’s always been a problem but I’d say these factors make it worse:
1. It’s harder for individuals or small groups to make breakthroughs. We need larger groups and more expertise as well as expensive equipment to continue making discoveries. More hands in the pie make more gatekeepers and less risk taking.
2. In general our entire society is “circling the wagons”. I’m not sure why but we’re more tribal than normal. Science isn’t immune from the trend.
I could go on. It’s a big topic.
Hasn't that always been the case? It's said science advances one grave at a time.
I suspect that there isn't a single cause but a cluster of them.
One possible issue is that high level science needs more and more energy (like LHC) and more and more intelligence to crack (most science prizes are teams).
[0] https://en.wikipedia.org/wiki/Planck%27s_principle?wprov=sfl...
> it takes a PhD before people can even understand the frontier of most scientific fields
It’s interesting that we have this implicit view that hyper-specialization is the only way to advance. I always thought so too, but why really? If you think about it, specialization is very close to incrementalism. Groundbreaking paradigm-shift stuff can often be written down on a napkin, and rarely does it require a PhD.
That said, to find treasure you have to go down a LOT of wrong paths before you (or someone else) ends up in the right one.
How to slow down scientific progress According to Leo Szilard
https://rootsofprogress.org/szilard-on-slowing-science
Discussed in https://news.ycombinator.com/item?id=34264436
Major breakthrough that disrupts the paradime. Followed by normal science with diminishing returns until the next paradime.
On the patent front, well patents are for parasites.
This intuition seems to overlook the culture of citations in the patent space. There, the incentive is for applicants to be over inclusive with citations. This is combined with the increasing ease with which references can be found.
But this is an effect of the simplification of history, and when you read the detailed technical histories you can see how remarkably incremental the old innovations were.
We can easily see the precedents of recent innovations but not for older ones.
This is pointed out in the book I am reading on the development of the turbojet, which only looks more unprecedented than it really was.
Lower corporate taxation incentivizes the hoarding of cash. Prior, corporations were incentivized to allocate more funding towards their industry research (IBM, Bell Labs, etc).
Would love some counterarguments, here!