HN Explorer

Show HN: We built an open-source social-media simulator for research (misinfogame.com)

49 points by sothatsit ↗ HN
Psychology research is currently facing a crisis: A significant amount of published studies cannot be reliably reproduced. Yet these studies will still be used in news headlines, and treated as fact.

Research into social media has not been immune to this either, with the problem exacerbated by study designs that don’t match the real world. We built the Misinformation Game to aid in this problem.

Our tool allows researchers to design social-media studies that look and feel like social media, while also making sure that each participant has a similar experience. This is really important for the ecological validity of studies (i.e., is it similar to reality), and the reproducibility of studies (chance plays less of a role). The tool’s reproducibility is further helped because the configuration of studies can be downloaded and shared alongside published research, minimising the work required for replication.

I have really enjoyed working on this project, and I hope you also find it interesting! I think there are huge opportunities to aid in research in non-tech fields using software, and I hope this project is a step in that direction.

There are example studies for you to play through that are available through the website. I’m excited to hear your thoughts on it!

All the best, ~ Paddy

17 comments

[ 2.7 ms ] story [ 42.3 ms ] thread
ChicagoBoy11
A non trivial part of social media is the social component, where folks build their own networks according to all sorts of graphs and in many instances using real world identities to do that. Do you have a sense as to the impact that the absence of that has on using something like this? Sure, we can fairly accurately replicate the mechanics of a social network, but the decision making in it surely is influenced by the difference in the social graph/tie to real identity/etc.
sothatsit
Thanks for the comment! You are right that social graphs are important. Our tool does not support them explicitly, as it is designed for studies that can take place over one session (e.g., minutes to hours), not studies over longer periods. However, we hypothesise that participants will still form associations with the identities that are included within studies. These identities can all be selected by the researchers, and they could choose to use real identities.

Our tool is better for research such as:

* Do followers influence perceived credibility?

* Does the number of likes influence whether someone will like a post?

* Does the emotion in posts influence participant’s belief in misinformation? (This is a bit more complicated to test, but can be done)

More complex research regarding social networks would likely require a specialised system. This is aimed for a more general audience, who would otherwise use questionnaires for many of these questions. I hope that helps :)

ChicagoBoy11
Thank you for the reply, it does! I do quite a bit of work with young people around social media and behavior, so I'm just noodling ways where I think this may be useful in setting up scenarios for them! Thanks so much for all the work!!!
sothatsit
I didn’t even think about using it for education! If you end up making use of it, I’d love to hear what you did and how it went.
anigbrowl
A significant amount of published studies cannot be reliably reproduced.

A problem likely to accelerate with the supposedly imminent (Thursday) unavailability of the free Twitter API.

pphysch
How does the ad industry do this sort of testing? Do they just get backdoor access to real social networks?
sothatsit
I’m not sure, but I’d assume they’d have a lot of data from everywhere their ads are placed, and purchasing the data would be much more viable for them.
philipov
Please tell me you called your simulator the Terror Nexus.
pfraze
Youre thinking of the Torment Nexus. The Terror Nexus was a sequel by the author’s son that most people don’t consider canon.
sothatsit
I’m sorry to say that I have no idea what you’re referring to… Are there some books I should read?
disadvantage
What's your thoughts on things like Kinzen[0] who fight misinfo? They recently got acquired by Spotify[1]

[0] https://www.kinzen.com/

[1] https://techcrunch.com/2022/10/05/spotify-acquires-content-m...

sothatsit
Fighting misinformation can be really difficult, so it’s always good to see people working on the problem! However, while a moderation tool is great, I’m not sure if it would help with more mundane misinformation (e.g., controversial figures saying outlandish statements). Personally, it seems like misinformation education would be a longer-term solution. However, I’m not an expert on this, I am just hopeful that with a better understanding of the Psychology behind misinformation, it can be fought more effectively. If you’re interested, I work with people from the Ecker Memory & Cognition Lab, which has released handbooks to tackling misinformation: https://www.emc-lab.org/.