0.05 significance has a 1 in 20 chance of being randomly occurring (1 / 0.05 = 20) and a result of statistical noise rather than evidence of a pattern. This is why reproducibility is important in science - future trials and tests will fail to reproduce the exact pattern of events needed to produce the statistical noise and so the null hypothesis will be accepted instead.
I think one problem is in the way the author searched for weird behavior. Literally searching for the phrase "my cat is acting weird" is probably not representative of all times that a cat is acting weird at all. Additionally, you will not be getting a lot of results by just searching for those terms, adding to the noise problem in the data. They need to either find a way to train an AI to recognize tweets of weird animal behavior, or at least create a less naïve information retrieval system in general (I'm guessing they just used the twitter search api).
Moreover, they need to test this on multiple earthquakes to get a better idea of what the data is like accounting for all events. There is cool science that could be done here, but this was not comprehensive enough to really say one way or the other that the hypothesis (animals can predict earthquakes) was wrong.
8 comments
[ 2.9 ms ] story [ 33.1 ms ] threadMoreover, they need to test this on multiple earthquakes to get a better idea of what the data is like accounting for all events. There is cool science that could be done here, but this was not comprehensive enough to really say one way or the other that the hypothesis (animals can predict earthquakes) was wrong.