Bayesian inference is more of a swiss-army knife than a hammer. It is a very versatile concept.
Yet, even if it were a hammer, getting the nail-like characteristics of an unknown object is very valuable. You just have to repeat the procedure for the other tools on your toolset and you'll either get to completely know it, or characterize a hole on your toolset that can now be filled. A big gain, either way.
I have often thought this, and I'm glad someone wrote the paper. I'd like to see more research in this area. It could end up being applicable to the design of more intelligent processes for group decision making at government scale.
For long I have been thinking that online communities such as YC and reddit benefit from an accelerated process of group evolution. Where other contexts necessitate 10 years to converge on a common idea/ideology, some communities do it orders of magnitude faster. Maybe this paper could explain why it works faster in some groups than others, given the topology and size.
Memes are a great example of this. They form, spread, and evolve so much more quickly on online communities than in the real world. I wonder if this represents a faster evolving community online, or one with weaker bonds because they're always in flux?
[1] Memetics is a theory that the base unit of culture is a "meme" (in the Richard Dawkins sense of shared worldview, not in terms of Pepe. But him too) https://en.wikipedia.org/wiki/Memetics
Cool paper! learned a lot about shared beliefs. But I was asking myself "why is 50k traders using similar trading patterns an example of collective intelligence"? I expected an example of a collective accomplishing a group goal as opposed to optimizing individual ability to accomplish goals.
Noted in first paragraph: "we still lack a coherent formal perspective on what human collective intelligence actually is". But, why not reference the closest definition or propose your own rigorous definition of collective intelligence to ground assumptions in the paper?
For instance, in 2007 Legg and Hutter presented a paper, "A Definition of Machine Intelligence" (http://arxiv.org/pdf/0712.3329v1.pdf) which inventoried existing definitions of machine intelligence and suggested their own definition. Seems like these definitional papers would be the starting point for presenting research on any kind of intelligence. Do you see meaningful work in this direction regarding defining collection intelligence?
The collective intelligence task in this case is for the community on the site to identify who are the best traders to be mimicking. It's not an explicit group goal, but we are treating it as an implicit one.
I wouldn't say we are trying to define collective intelligence per se. e.g., when you offer a model of a dynamical physical process (maybe a ball arching through the air), you can hardly say that your model provides a definition of that physical system. But you can say that the physical model describes the movement of the ball.
In the same way, I'd instead say we are simply trying to grabble with how to think about collective intelligence.
We are offering a model and a class of models that we expect to display collectively intelligent behavior, and we thereby hypothesize that these models might also explain how collective intelligence in human groups might arise.
I am curious how one accounts for language and logic in this approach?
After all, a logical statement involves something like X iff Y and Z and Q, so the truth of X might varying discontinuously with average truth values of Y, Z and Q.
You may have one population where 50% think Y, Z and Q are truth and 50% don't and another population where 50% think only Y and Z are truth and 50% think only Z and Q are true.
In the first population, 50% think X is true and in the other population 0% think X. But the average truth values of Y, Z and Q would higher in the second population.
I'm kind of shooting in the dark on this one, but Bayesian inference does function as a probabilistic generalization of propositional logic, so in principle it seems like the framework should be able to accommodate at least some kind of scenarios involving logical statements. There are also lots of Bayesian language models.
At the same time, there are impossibility theorems that show beliefs across a group cannot in general be aggregated in coherent ways. (Some of the counter examples look a lot like the one you give.)
So the type of model we are introducing is necessarily only going to be able to apply to certain groups (and perhaps could help characterize collectively rational vs irrational groups).
One other related comment: the case we have dealt with so far is mainly about how groups integrate new information (collective learning), not about the existing prior knowledge knowledge within the group.
In this framework, treating popularity as a prior over all previous information, what do you make of groups that maintain irrational beliefs for very long periods of time? Specifically things like superstitions where accidental correlations between unrelated events don't seem to be corrected by individual actors adding just a bit of objectivity at each step
This is really neat by the way, thanks for producing this paper, I think it's an important area to start poking around in.
The model predicts that areas in which there is little evidence one way or the other available are the ones where incorrect beliefs could be sustained, particularly when the belief is held by those around you.
I guess one interesting characteristic of certain superstitions is that people will avoid the scenario that they think will lead to a bad outcome, so they actually don't collect personal evidence discrediting that superstition.
This line of argument suggests that positive superstitions should be less persistent than negative superstitions, though I am not sure if that is true.
Ok, so let's keep going from there. I mean, it ought to be intuitive, with everyone having a favorite personal example, that groups of people seem pretty able to sustain very flagrantly wrong, and even downright stupid, beliefs over long periods of time. So a theory of socially distributed cognition should ideally either explain how this happens, or explain it away as an unfortunate side-effect of rational thinking in ambiguous situations (like certain optical illusions and the Bayesian brain theory).
One mechanism I've heard posited (by Friston's neurosci lab) for how humans deal with noisy environments is to weight our sensory inputs by their expected precision. I might look at the ocean's surface and expect to see a froth of sea-foam, particulate matter, and water, so I won't pay enough attention to see a camouflaged fish. I'll still notice if Godzilla walks up out of it, though.
Would a similar precision-weighting mechanism help to model social trust, and to thus give us an idea of how groups can fixate on incorrect group-beliefs? Are people treating outgroup members or "opponents", so to speak, as uselessly noisy, and thus down-weighting any need to update based on what those persons say or do? Could this contribute to people's (supposed) occasional failures to accurately model their outgroups, since they're intuitively modelling those outgroups precisely as mostly outputting random noise rather than precise information?
Lots of good ideas here. Trust in social information, and which social information you trust are super interesting questions that I would love to do more work on.
It's possible that it's something along the lines you say. Another possibility is people just don't have good estimates of popularity among people they don't know, so they overestimate the popularity of opinions represented in their immediate networks.
Zooming way out, when groups appear to hold irrational beliefs even while containing some rational members, any insights on what sort of conditions need to be put in place for groups to become more rational over time?
The predictions of the model I have thought about regarding this question are fairly intuitive. Exposure to evidence is important. Integrating networks is likely important, too, though I haven't done those simulations yet.
I was just reading Michael Nielson's free ebook: Neural Networks and Deep Learning: A Principle-Oriented Approach[1] yesterday. When Dr. Nielsen tries to reason why it may or may not have a simple algorithm that represent the intelligence, this excerpt strike me the most:
In the 1970s and 1980s Marvin Minsky developed his "Society of Mind" theory,
based on the idea that human intelligence is the result of a large
society of individually simple (but very different) computational processes
which Minsky calls agents. In his book describing the theory, Minsky sums up
what he sees as the power of this point of view:
What magical trick makes us intelligent? The trick is that there is no trick.
The power of intelligence stems from our vast diversity, not from any single,
perfect principle.
Asset markets are a poor choice for studying "accurate shared beliefs". They're an excellent choice for studying groupthink, though.
The whole premise that people arrive at accurate shared beliefs is rather extraordinary to begin with. In my experience, widely shared beliefs are much more likely to be inaccurate.
23 comments
[ 3.7 ms ] story [ 32.9 ms ] threadYet, even if it were a hammer, getting the nail-like characteristics of an unknown object is very valuable. You just have to repeat the procedure for the other tools on your toolset and you'll either get to completely know it, or characterize a hole on your toolset that can now be filled. A big gain, either way.
Yeah, it's called direct democracy and democratisation of media.
Aircraft searches are a big thing, and there's a book on how it has been applied to MH370...
[0] - https://www.amazon.co.uk/Theory-That-Would-Not-Die-ebook/dp/...
[1] Memetics is a theory that the base unit of culture is a "meme" (in the Richard Dawkins sense of shared worldview, not in terms of Pepe. But him too) https://en.wikipedia.org/wiki/Memetics
Noted in first paragraph: "we still lack a coherent formal perspective on what human collective intelligence actually is". But, why not reference the closest definition or propose your own rigorous definition of collective intelligence to ground assumptions in the paper?
For instance, in 2007 Legg and Hutter presented a paper, "A Definition of Machine Intelligence" (http://arxiv.org/pdf/0712.3329v1.pdf) which inventoried existing definitions of machine intelligence and suggested their own definition. Seems like these definitional papers would be the starting point for presenting research on any kind of intelligence. Do you see meaningful work in this direction regarding defining collection intelligence?
The collective intelligence task in this case is for the community on the site to identify who are the best traders to be mimicking. It's not an explicit group goal, but we are treating it as an implicit one.
I wouldn't say we are trying to define collective intelligence per se. e.g., when you offer a model of a dynamical physical process (maybe a ball arching through the air), you can hardly say that your model provides a definition of that physical system. But you can say that the physical model describes the movement of the ball.
In the same way, I'd instead say we are simply trying to grabble with how to think about collective intelligence.
We are offering a model and a class of models that we expect to display collectively intelligent behavior, and we thereby hypothesize that these models might also explain how collective intelligence in human groups might arise.
After all, a logical statement involves something like X iff Y and Z and Q, so the truth of X might varying discontinuously with average truth values of Y, Z and Q.
You may have one population where 50% think Y, Z and Q are truth and 50% don't and another population where 50% think only Y and Z are truth and 50% think only Z and Q are true. In the first population, 50% think X is true and in the other population 0% think X. But the average truth values of Y, Z and Q would higher in the second population.
I'm kind of shooting in the dark on this one, but Bayesian inference does function as a probabilistic generalization of propositional logic, so in principle it seems like the framework should be able to accommodate at least some kind of scenarios involving logical statements. There are also lots of Bayesian language models.
At the same time, there are impossibility theorems that show beliefs across a group cannot in general be aggregated in coherent ways. (Some of the counter examples look a lot like the one you give.)
So the type of model we are introducing is necessarily only going to be able to apply to certain groups (and perhaps could help characterize collectively rational vs irrational groups).
One other related comment: the case we have dealt with so far is mainly about how groups integrate new information (collective learning), not about the existing prior knowledge knowledge within the group.
This is really neat by the way, thanks for producing this paper, I think it's an important area to start poking around in.
The model predicts that areas in which there is little evidence one way or the other available are the ones where incorrect beliefs could be sustained, particularly when the belief is held by those around you.
I guess one interesting characteristic of certain superstitions is that people will avoid the scenario that they think will lead to a bad outcome, so they actually don't collect personal evidence discrediting that superstition.
This line of argument suggests that positive superstitions should be less persistent than negative superstitions, though I am not sure if that is true.
One mechanism I've heard posited (by Friston's neurosci lab) for how humans deal with noisy environments is to weight our sensory inputs by their expected precision. I might look at the ocean's surface and expect to see a froth of sea-foam, particulate matter, and water, so I won't pay enough attention to see a camouflaged fish. I'll still notice if Godzilla walks up out of it, though.
Would a similar precision-weighting mechanism help to model social trust, and to thus give us an idea of how groups can fixate on incorrect group-beliefs? Are people treating outgroup members or "opponents", so to speak, as uselessly noisy, and thus down-weighting any need to update based on what those persons say or do? Could this contribute to people's (supposed) occasional failures to accurately model their outgroups, since they're intuitively modelling those outgroups precisely as mostly outputting random noise rather than precise information?
It's possible that it's something along the lines you say. Another possibility is people just don't have good estimates of popularity among people they don't know, so they overestimate the popularity of opinions represented in their immediate networks.
Or is it a parallel ... system?
Thought it's relevant to mention about it.
[1]: https://news.ycombinator.com/item?id=12305455
The whole premise that people arrive at accurate shared beliefs is rather extraordinary to begin with. In my experience, widely shared beliefs are much more likely to be inaccurate.