I've been thinking about this too: why aren't we able to run safe political experiments?
We're missing guardrails to allow safe experimentation and we're missing institutions to provide affordances.
I think the difficult bit is figuring out how to seperate the goodness of centralized decision shaping and the badness of centralized power accumulation.
The article gives examples of situations where projects have failed, and states a solution.
> The [solution] is to organize the work into the smallest possible learnable chunks and continuously alternate between doing and learning.
It would be more convincing if the article gave examples of where that solution had been tried and succeeded. I mean this must have been tried somewhere, surely.
I think back to the first covid lockdowns. All of society was reorganized in a matter of weeks! Oil is worth negative money if they don't make us consume it! Work from home became mainstream, it turns out all those people with anxiety disorders or pain management issues could in fact participate in society! Generous payments were made for people who couldn't work, with no concern about who will pay for it!
The harsh reality is that all governments could solve most problems easily, they just couldn't be bothered.
Governments do run experiments sometimes, quite a lot of experiments on UBI have been run for example and we have good knowledge on whether its introduction would improve society. But I don't feel like leadership particularly cares about evidence and the right thing, they are far more idealogical than that and tend to gravitate towards policy based evidence from thinktanks and other powerful sources that produce bad science but the results they want to see.
The populace doesn't have much in the way of alternative choices for politicians that would follow the results of actual experiments nor fund them, its not really an option being offered, I think partly because its a tough sell compared to "we will do X". "We will test a variety of options and then do the best" requires more trust and its a low trust environment.
I think the UBI experiments were one-sided. They may have proven that the receivers have profited. But did they show that the givers profited as well, or at least did not suffer enough to significantly reduce their economic output and therefore risk the financing of UBI?
You'd need to check something like take a group of high-earners, increase their tax rate by say 100%, increase their cost of living, and validate that they neither leave the experiment nor work significantly less.
The fundamental issue with this is that many problems have a time/energy/financial threshold for success. Trying to tackle such a problem with incremental iterative solutions will consistently fail, as each individual iteration will fail.
This is most obvious when network effects are present (e.g. local immunisation efforts vs country-wide immunisation), but it's surprisingly common in other government-related areas like welfare, childcare, social security etc.
Edit: Another comment has reminded me that affordable public transport is the perfect example of this: Incrementally building out a public transport system will almost always fail, as the initial lines (be they buses, light rail, etc) will typically not be successful enough to justify the cost of building the line. If, instead, a system is built out universally and simultaneously, the utility (and thus income) of each line increases due to the interconnected nature of the network.
It doesn't work because politicians are not allowed to admit failure. A politician who admits that they tried something, no matter how small, and it didn't work, is fish food. This means there is no difference in downside, for the politician, between going all in on a big decision with huge risk versus taking baby steps. The politician is dead no matter what. They either keep claiming the bad decision was a good one in the hope that something turns up, or eventually resign or get voted out. But if they ever dare to publicly change their point of view in the presence of new evidence, they are accused of the worst crime a politician can commit. They are a "flip-flopper."
This is why I left The Netherlands. They have been working on laws for both self employment and unrealized capital gains for the past ten years and in the meanwhile have implemented some unworkable “temporary laws”.
Just implement good laws, do it right once. Yes it will hurt to not have a smooth transition but it’s better than having meetings about it for ten years.
It's weird that this article doesn't even attempt to grapple with the reason why governments can't run big policies as experiments and cancel them if they fail:
Every time the government hires a group of people to do X, that automatically creates a class of people who:
1. Depend on the government continuing to do X for their livelihoods
2. Are experts in X and know far more about it than any government official
This creates an automatic constituency that will fight tooth and nail to keep X going no matter what. And step one of that fighting will be to make sure that the official report answering the original research question "Did X work?" will never be a clear "No."
And God help any politician that ignores the official report and cancels X anyway. Now the problem that X was intended to solve is entirely their fault, and there's an army of X experts running to every media outlet in the country making sure the general public knows it!
Your argument hinges on deciding what "works" means.
You're saying it "works" for some people but maybe not in a way or scale that was originally intended for it to "work".
But isn't that how all learning in a complex system takes place? Doesn't mean you need to rush to cancel something?
What's wrong in saying it doesn't do what we thought it would but it does do something useful? Maybe adjust further investment based on results obtained. Ignore sunk cost.
Please give examples of these agencies. What you list seems more like how business captures control via 'knowing more than government officials' then how technocrats do. At least that's what conservative politicians tell us as they defer all oversight/domain knowledge to private industry and private industry 'best' practices. Are you also saying that government should stop that practice and that is is also a trap of some kind? Or is it just a trap when it's for the betterment of society?
this is based more on assumption than anything else i think. this is something i hear folks yell very loudly about all the time but actual examples seem to be harder to come by.
The most obvious (to me) example is the San Francisco anti-homelessness program. The city government of San Francisco understood that
1. Homelessness is a massive problem, far larger here than most cities
2. For political reasons, the obvious solution of "Double the number of homes in the city to take people off the streets at the margin" is off the table.
As a result, they started giving massive contracts to a wide variety of NGOs that promised a wide variety of novel solutions to the homelessness tragedy.
As it turned out, the effect of basically all of those experiments was either nothing or counterproductive. But the NGOs ended up being so numerous that they have political power of their own, and are now at the same level as "teachers" as a major expense and powerful constituency for the city.
If you're one of those people who needs an opposite-political-valence example, a good one is the TSA. Totally worthless at stopping terrorists, but totally skilled at keeping itself funded.
Just because an 'automatic consistency' for something exists doesn't mean they automatically get their way on everything. And for most X that group is not big enough to direct federal policy.
Yes Amtrak exist and people inside Amtrak want it to continue to exist, and yet tons of reports the government does shows issues with Amtrak. Not to mention many groups outside of Amtrak, civil society, journalists and so on.
Many government institutions have existed for 100s of years have no managed to capture government to such an extend as to make them be able to monopolize any analysis of them. And there are plenty of government programs that don't exist anymore as well.
You are only looking at 1 side of this issue. You have given 0 consideration if X was actually solved or not.
Do you remember the movie "Good Will Hunting". The Therapist says "You can never do anything because you see every possible bad thing down the road" but that also misses also the opportunity for all possible good things.
In Switzerland we have SBB, a almost universally popular government owned institution that has existed for well over 100 years and has solved a problem about as good as anybody in the world. So X was solved, and we still have very objective discussion on improving X and we get reports about issues with X and so on.
In your view it would be good if Switzerland canceled the SBB? No matter if reports indicate that its beneficial or not?
I don't see how your attitude is not just defeatism and depression.
The thing is, there is no hive mind called "society". Everybody works in their own interest. Always, and in every "society".
Individual workers work on failing initiatives as long as they get paid. The outcome of the initiative may be bad for "society". But was it good for the individuals working on it? Maybe they got paid well. Maybe they enjoyed the work? Maybe the work was easy because they knew that it would fail anyway, so they didn't have to put much effort into it?
Maybe it was also good for the management or politicians? Maybe it was a step up in their career. And maybe, if they could jump ship before the failure became obvious, they could climb up the ladder to get to an even better position? You can always blame your successors for ruining the project.
And maybe it was good for whoever ordered it? If it's a local project, maybe they got subsidies from federal government bodies, and they don't even care whether it succeeds, as long as it created employment and the illusion of progress? Or if it's a private project, maybe they just tried a moon shot that, if it fails, was useful as a tax write-off?
In real life, there are so many layers to a 'failed' project. It can be a failure for some and a success for others. And those for whom it is a success will defend it, maybe even deceive to keep it running.
1. An airplane crashes, everyone dies. Clearly a bad thing. Not so quick. For the funeral industry this means additional business, a good thing. And this is true for a LOT [1] of things, they are not good or bad, right or wrong, their judgment depends on perspective and personal preferences.
2. Which means that there is generally no policy that makes everyone happy. So you need a party with a program that aims at finding compromises that are acceptable for everyone.
3. But nobody will vote for such a party. Why would you vote for a party that gives you 50 % of what you want if there is a different party that is more aligned with your views and preferences and promises to give you 90 % of what you want?
4. In consequence the political direction tends to hop between extremes instead of settling on compromises. One group gets really unhappy with the current situation, shows up for elections, votes their party into power, moves the situation into the direction of a different extreme, until others get unhappy enough to start the process all over again.
5. Even in political systems where [sometimes] a coalition of parties exercises the power and they are forced to compromise, the outcome is all but ideal. Things move slowly because finding compromises is hard if you do not really want to compromise. Voters look down on the party they voted for because they are not delivering what they promised but only compromises.
I guess the moral of the story is that the voters have to realize that their view is not the only valid one and that voting for compromises would probably yield better outcomes than voting for extremes and either going in that direction for some time until turning around or maybe arriving at a forced compromise that no one voted for.
[1] Exercise for the reader, find something politically relevant that does not depend on perspective and personal preferences.
Plus the simple fact that, by the same logic, you can try to both-sides plain murder on the ground that the murderer wants to murder (and, again, that the undertakers and coroners and such must be kept in business). Clearly, whilst every policy will have supporters and detractors, some detractions are more worthy of consideration than others.
The point of the example is that something that would generally be judged as obviously bad often turns out to not be bad from all perspectives. The same obviously also holds for good things. But I did not intend to imply any economics. The crash investigation might uncover an issue that gets fixed and prevents future crashes, also in this sense the crash had a positive effect. The better outcome in some overall sense would probably still have been for the airplane not to crash and the families spending the money on something better than funerals.
[...] some detractions are more worthy of consideration than others.
This is probably quite hard to justify in general.
I want zero cents of my money to be wasted on climate change, I live now and I want to enjoy my life as much as possible. I do not care if the planet gets burned to a crisp in a hundred years when I am dead.
One can certainly have a different perspective and vehemently disagree, especially if you have children that will have to live through that future, but otherwise? How would you argue that this position is somehow less valid than any other?
Democracies solve the succession problem. Autocracies often don't have peaceful transition of power and swing far more extreme. Sometimes even civil war.
Are you seriously suggesting that there are different acceptable perspectives on the first point? I thought we still had a semblance of society, where we had a common perspective on that one, i.e that loss of life should be avoided at all times.
What do you prefer, a plane crash or a nuclear war? But that does not matter, I did not say a airplane crashing is preferable over something, I said it is not a bad event for everyone. It is bad for the people on board and their friends and families but there are also people that benefit from a crash. That still does not mean that people benefiting from the event are in favor of more airplanes crashing. And for the majority of people it will essentially be neutral, half the planet might never even hear about this particular crash and for many more it will be a thirty second news segment they once saw.
I agree with the problem, and I think it stems from a few core issues:
1. Societies are heterogenous, and some groups are always benefitting from the status quo, which will then rebel against any potential changes.
For instance, in Germany (and multiple other European countries), forming a GmbH (LLC) requires in-person sessions with expensive notaries, who will read the entire paperwork out loud.
This made sense when it was invented in 1892 because company formation was extremely rare, always required large facilities and upfront purchases, etc.
Today, this requirement only benefits notaries, as I reckon extremely few people would voluntarily hire a notary to form their company.
But of course notaries would be against making their services legally required! And of course, if anyone suggests loosening requirements, they will spell out all of the terrible fraud that would happen if company formation didn't require a thorough legal review.
That's an example for something relatively obvious. It gets much harder with genuinely complex topics (pensions? tax reform? healthcare?).
2. Many initiatives to improve society fail because they're viewed through a prism of administration/policy, not the actual people impacted.
Take modern urbanism, which cleanly separates where people live (residential areas), where they work (commercial districts), where they shop (malls), and where they spend their free time (recreational centers, etc.)
This beautifully serves people who plan and administer because it makes their work massively easier vs. a tangle of apartment buildings with a coffee shop and clothing boutique on the ground floor and a playground for the kids in front.
It checks the boxes of daily life, but the people living through it feel alienated from their community with these single-purpose urban areas.
3. We have a bias towards actionism.
We tend to think that big, complicated problems need big, complicated solutions, and politicians (who are evaluated on perception, not results) need to be seen designing and implementing those big, complicated solutions.
Massive amounts of infant mortality were prevented by simply making doctors wash their hands, and Semmelweis (who came up with the hypothesis) was so ridiculed for that idea that he died in an insane asylum.
It's sadly often unacceptable to say "we're going to make a little tweak that'll have a ton of downstream effects and solve this massive problem in 5 years".
> It checks the boxes of daily life, but the people living through it feel alienated from their community with these single-purpose urban areas.
these rules exist for good reason -- regs are written in blood -- and ware why your air and water are cleaner than ever: you're not right next to the places where they're leaching into the ground.
one of the original zoning rules was after a giant tank of hot molasses exploded and killed 12 people, burying a chunk of Boston in sticky-sweet tar. people were sitting at home minding their own business before being scalded and drowning in poorly stored food product. and that ain't got nothin on industrial toxins...
I think there's a big difference between putting people's houses next to the local paint factory and saying that you could have mixed-use neighborhoods where you have local businesses and recreational activities close to where you live vs. needing to go to the designated area for it.
I'm not arguing for zero zoning laws, but I'm saying that much of modernist urban planning primarily benefits the planner.
The purpose of a system is what it does. You're never going to have success optimizing a system towards goals that are not the goals of the system. You need to first assess what the system is optimizing towards in the first place and now to change the incentives, otherwise you're going to misdiagnose competence as incompetence.
39 comments
[ 4.4 ms ] story [ 52.4 ms ] threadWe're missing guardrails to allow safe experimentation and we're missing institutions to provide affordances.
I think the difficult bit is figuring out how to seperate the goodness of centralized decision shaping and the badness of centralized power accumulation.
> The [solution] is to organize the work into the smallest possible learnable chunks and continuously alternate between doing and learning.
It would be more convincing if the article gave examples of where that solution had been tried and succeeded. I mean this must have been tried somewhere, surely.
The harsh reality is that all governments could solve most problems easily, they just couldn't be bothered.
The populace doesn't have much in the way of alternative choices for politicians that would follow the results of actual experiments nor fund them, its not really an option being offered, I think partly because its a tough sell compared to "we will do X". "We will test a variety of options and then do the best" requires more trust and its a low trust environment.
You'd need to check something like take a group of high-earners, increase their tax rate by say 100%, increase their cost of living, and validate that they neither leave the experiment nor work significantly less.
This is most obvious when network effects are present (e.g. local immunisation efforts vs country-wide immunisation), but it's surprisingly common in other government-related areas like welfare, childcare, social security etc.
Edit: Another comment has reminded me that affordable public transport is the perfect example of this: Incrementally building out a public transport system will almost always fail, as the initial lines (be they buses, light rail, etc) will typically not be successful enough to justify the cost of building the line. If, instead, a system is built out universally and simultaneously, the utility (and thus income) of each line increases due to the interconnected nature of the network.
Just implement good laws, do it right once. Yes it will hurt to not have a smooth transition but it’s better than having meetings about it for ten years.
Every time the government hires a group of people to do X, that automatically creates a class of people who:
1. Depend on the government continuing to do X for their livelihoods
2. Are experts in X and know far more about it than any government official
This creates an automatic constituency that will fight tooth and nail to keep X going no matter what. And step one of that fighting will be to make sure that the official report answering the original research question "Did X work?" will never be a clear "No."
And God help any politician that ignores the official report and cancels X anyway. Now the problem that X was intended to solve is entirely their fault, and there's an army of X experts running to every media outlet in the country making sure the general public knows it!
You're saying it "works" for some people but maybe not in a way or scale that was originally intended for it to "work".
But isn't that how all learning in a complex system takes place? Doesn't mean you need to rush to cancel something?
What's wrong in saying it doesn't do what we thought it would but it does do something useful? Maybe adjust further investment based on results obtained. Ignore sunk cost.
1. Homelessness is a massive problem, far larger here than most cities 2. For political reasons, the obvious solution of "Double the number of homes in the city to take people off the streets at the margin" is off the table.
As a result, they started giving massive contracts to a wide variety of NGOs that promised a wide variety of novel solutions to the homelessness tragedy.
As it turned out, the effect of basically all of those experiments was either nothing or counterproductive. But the NGOs ended up being so numerous that they have political power of their own, and are now at the same level as "teachers" as a major expense and powerful constituency for the city.
If you're one of those people who needs an opposite-political-valence example, a good one is the TSA. Totally worthless at stopping terrorists, but totally skilled at keeping itself funded.
Yes Amtrak exist and people inside Amtrak want it to continue to exist, and yet tons of reports the government does shows issues with Amtrak. Not to mention many groups outside of Amtrak, civil society, journalists and so on.
Many government institutions have existed for 100s of years have no managed to capture government to such an extend as to make them be able to monopolize any analysis of them. And there are plenty of government programs that don't exist anymore as well.
You are only looking at 1 side of this issue. You have given 0 consideration if X was actually solved or not.
Do you remember the movie "Good Will Hunting". The Therapist says "You can never do anything because you see every possible bad thing down the road" but that also misses also the opportunity for all possible good things.
In Switzerland we have SBB, a almost universally popular government owned institution that has existed for well over 100 years and has solved a problem about as good as anybody in the world. So X was solved, and we still have very objective discussion on improving X and we get reports about issues with X and so on.
In your view it would be good if Switzerland canceled the SBB? No matter if reports indicate that its beneficial or not?
I don't see how your attitude is not just defeatism and depression.
Individual workers work on failing initiatives as long as they get paid. The outcome of the initiative may be bad for "society". But was it good for the individuals working on it? Maybe they got paid well. Maybe they enjoyed the work? Maybe the work was easy because they knew that it would fail anyway, so they didn't have to put much effort into it?
Maybe it was also good for the management or politicians? Maybe it was a step up in their career. And maybe, if they could jump ship before the failure became obvious, they could climb up the ladder to get to an even better position? You can always blame your successors for ruining the project.
And maybe it was good for whoever ordered it? If it's a local project, maybe they got subsidies from federal government bodies, and they don't even care whether it succeeds, as long as it created employment and the illusion of progress? Or if it's a private project, maybe they just tried a moon shot that, if it fails, was useful as a tax write-off?
In real life, there are so many layers to a 'failed' project. It can be a failure for some and a success for others. And those for whom it is a success will defend it, maybe even deceive to keep it running.
2. Which means that there is generally no policy that makes everyone happy. So you need a party with a program that aims at finding compromises that are acceptable for everyone.
3. But nobody will vote for such a party. Why would you vote for a party that gives you 50 % of what you want if there is a different party that is more aligned with your views and preferences and promises to give you 90 % of what you want?
4. In consequence the political direction tends to hop between extremes instead of settling on compromises. One group gets really unhappy with the current situation, shows up for elections, votes their party into power, moves the situation into the direction of a different extreme, until others get unhappy enough to start the process all over again.
5. Even in political systems where [sometimes] a coalition of parties exercises the power and they are forced to compromise, the outcome is all but ideal. Things move slowly because finding compromises is hard if you do not really want to compromise. Voters look down on the party they voted for because they are not delivering what they promised but only compromises.
I guess the moral of the story is that the voters have to realize that their view is not the only valid one and that voting for compromises would probably yield better outcomes than voting for extremes and either going in that direction for some time until turning around or maybe arriving at a forced compromise that no one voted for.
[1] Exercise for the reader, find something politically relevant that does not depend on perspective and personal preferences.
Although your plane crash example falls afoul of the broken window fallacy: https://en.wikipedia.org/wiki/Parable_of_the_broken_window
Plus the simple fact that, by the same logic, you can try to both-sides plain murder on the ground that the murderer wants to murder (and, again, that the undertakers and coroners and such must be kept in business). Clearly, whilst every policy will have supporters and detractors, some detractions are more worthy of consideration than others.
[...] some detractions are more worthy of consideration than others.
This is probably quite hard to justify in general.
I want zero cents of my money to be wasted on climate change, I live now and I want to enjoy my life as much as possible. I do not care if the planet gets burned to a crisp in a hundred years when I am dead.
One can certainly have a different perspective and vehemently disagree, especially if you have children that will have to live through that future, but otherwise? How would you argue that this position is somehow less valid than any other?
Over the long term, democracies have mostly avoided self-implosion, so this political see-saw does seem to work.
The question is: is there anything we can do to make it feel better to live in?
1. Societies are heterogenous, and some groups are always benefitting from the status quo, which will then rebel against any potential changes.
For instance, in Germany (and multiple other European countries), forming a GmbH (LLC) requires in-person sessions with expensive notaries, who will read the entire paperwork out loud.
This made sense when it was invented in 1892 because company formation was extremely rare, always required large facilities and upfront purchases, etc.
Today, this requirement only benefits notaries, as I reckon extremely few people would voluntarily hire a notary to form their company.
But of course notaries would be against making their services legally required! And of course, if anyone suggests loosening requirements, they will spell out all of the terrible fraud that would happen if company formation didn't require a thorough legal review.
That's an example for something relatively obvious. It gets much harder with genuinely complex topics (pensions? tax reform? healthcare?).
2. Many initiatives to improve society fail because they're viewed through a prism of administration/policy, not the actual people impacted.
Take modern urbanism, which cleanly separates where people live (residential areas), where they work (commercial districts), where they shop (malls), and where they spend their free time (recreational centers, etc.)
This beautifully serves people who plan and administer because it makes their work massively easier vs. a tangle of apartment buildings with a coffee shop and clothing boutique on the ground floor and a playground for the kids in front.
It checks the boxes of daily life, but the people living through it feel alienated from their community with these single-purpose urban areas.
3. We have a bias towards actionism.
We tend to think that big, complicated problems need big, complicated solutions, and politicians (who are evaluated on perception, not results) need to be seen designing and implementing those big, complicated solutions.
Massive amounts of infant mortality were prevented by simply making doctors wash their hands, and Semmelweis (who came up with the hypothesis) was so ridiculed for that idea that he died in an insane asylum.
It's sadly often unacceptable to say "we're going to make a little tweak that'll have a ton of downstream effects and solve this massive problem in 5 years".
these rules exist for good reason -- regs are written in blood -- and ware why your air and water are cleaner than ever: you're not right next to the places where they're leaching into the ground.
one of the original zoning rules was after a giant tank of hot molasses exploded and killed 12 people, burying a chunk of Boston in sticky-sweet tar. people were sitting at home minding their own business before being scalded and drowning in poorly stored food product. and that ain't got nothin on industrial toxins...
I'm not arguing for zero zoning laws, but I'm saying that much of modernist urban planning primarily benefits the planner.