I really wish the analysis didn't cut off at exactly 10 numbers. Rerunning the query with higher LIMIT, I noticed the following:
- The very next two numbers are 100 and 0. While 100 is consistent with the article's explanations, 0 still placing high despite fixing the query warrants further investigation. A quick glance at returned headlines shows the problem of phantom zeroes is not, in fact, fixed.
- The query doesn't group decimals and integers together. 2.0 is at #17 with 10k hits, while 1.0 is at #26 with 5k hits. So not only is the "version number" explanation for top numbers wrong - the claim that earlier version numbers are more common than later version numbers is wrong too.
I once did some small work with a mathematician who was an online friend of mine. He was really into Benford's Law. A lot of mathematicians seem to love Benford's Law, and I wonder why that is. I'm not sure if I was rude to him, but we lost touch, and I miss him. I learned so much from his code.
The article mentions one reason for small numbers predominating:
> The first is listicles. “3 ways to do X”, “5 things I wish I’d known”, “10 tools I actually use”. The listicle is practically the native unit of the link post, and its count is almost always small and round. Nobody writes “17 things I wish I’d known” unless they genuinely have 17 things, which is rare, whereas “5 things” is a decision made in a title editor.
But doesn't the title mangler specifically try to strip out numbers that look like they're part of listicle titles?
15 comments
[ 4.0 ms ] story [ 43.8 ms ] thread"to", "the", "of". I don't know what I expected.
benfords law strikes again. useful for fraud analysis too
- The very next two numbers are 100 and 0. While 100 is consistent with the article's explanations, 0 still placing high despite fixing the query warrants further investigation. A quick glance at returned headlines shows the problem of phantom zeroes is not, in fact, fixed.
- The query doesn't group decimals and integers together. 2.0 is at #17 with 10k hits, while 1.0 is at #26 with 5k hits. So not only is the "version number" explanation for top numbers wrong - the claim that earlier version numbers are more common than later version numbers is wrong too.
Edit. Perhaps the title should be 'parsing numbers is hard'.
> The first is listicles. “3 ways to do X”, “5 things I wish I’d known”, “10 tools I actually use”. The listicle is practically the native unit of the link post, and its count is almost always small and round. Nobody writes “17 things I wish I’d known” unless they genuinely have 17 things, which is rare, whereas “5 things” is a decision made in a title editor.
But doesn't the title mangler specifically try to strip out numbers that look like they're part of listicle titles?