Show HN: I built a 2-min quiz that shows you how bad you are at estimating (convexly.app)

There’s a bias, I think. When I saw the title that is about how bad I’m at estimating, I’ve leaned towards counterintuitive answers. This got me quite a high score. I think test set should also include intuitive facts (or maybe I was just lucky).

Interesting data from the quiz so far: 160+ quiz takers! The average is 0.239 (barely better than a coin flip at 0.25), but almost everyone indicates they are confident in their answers.

Update: 400+ quiz takers now... insane. Best Brier score so far is 0.007 (nearly perfect calibration). The worst came in at 0.600. Average is 0.230, still just better than a coin flip. Where did you land?

Why do I need to sign up to get the results? Why couldn't it just be on the page?

I think this might be conflating confidence with accuracy. I tried leaving the slider the the middle (nominally the least confident position) and it gave a score of 0.25 and diagnosed it as 'overconfident'.

I thought it was interesting, but don't appreciate having to give you my email to see full results.

I unsubscribe from mails that aren't useful to me day-to-day because they're distracting.

Other than that it seems like a cool idea. I'd recommend slightly bigger fonts. I often have this issue with Gemini.

  Brier Score: 0.216 (lower is better)
  Diagnosis: Overconfident

Did it twice: once had 0.177, 2nd time got 0.280. Note sure what to make of this, I guess I should always leave it on 50/50?

I didn't find the questions very representative about estimation. that is maybe if happen to know many of random root facts about the world under which they were based, then their application might be a revenant question about ability to estimate. I really felt more like I was making uneducated guesses (0.155). I suppose I was expecting more ping pong balls in airplanes

It is very disappointing that you can't see what you got right or wrong without giving out your email. I'm not even sure if one would learn from the email or whatever the calibration result is.

I'm happy for you if it works but I sure feel cheated. I hope others also feel it's against the spirit of a Show HN. But maybe it's just me.

Is it down? The start and skip button both dont work and I see this error in my console.

Manifest fetch from https://www.convexly.app/manifest.json failed, code 403

Maybe I don't know enough about "calibration" in a technical sense, but it seems like this quiz cant really distinguish between factual knowledge and calibration skill?

Is this type of quiz reproducible for individuals and across various cross-sections of the population?

Are there studies on this? Is the quiz based on these studies?

> Question: A fair die rolling a 6 twice in a row is more likely than rolling 1-2-3-4-5-6 in sequence

Two 6s in a row is 1/36 chance (1/6)^2

1-2-3-4-5-6 is a 1/46656 chance (1/6)^6

Website is claiming they are the same probability:

> Same probability: 1/46,656 — Both outcomes have exactly the same probability: (1/6)^6 = 1/46,656. This illustrates the representativeness heuristic — random-looking sequences feel more probable than ordered ones.

Website's "answer" is wrong: was the question supposed to be rolling a 6 six times in a row?

Why is it asking for email?

Wait, so roughly is it rewarding being confident when correct, and penalizing being confident when wrong? Meaning that the highest score is only achievable if you answer fully confident true or false, and get all 10 correct?

If so, isn't that conflating knowledge with over/under confidence?

“You averaged 97% confidence but were right 80% of the time.”

Heck yeah.

This reminds me of:

https://taketest.xyz/confidence-calibration

The same site also has something with a fixed confidence level: https://taketest.xyz/ci-calibration

I’ve taken the quiz but not been compelled to sign up. The site feels manipulative, e.g., the “show me all the questions” link is tiny and hidden between two larger boxes, and even then it only shows 2 questions with a signup CTA. Maybe that’s best practice growth hacking these days, but to me it’s a manipulative turnoff. If you’d given me all the questions and answers simply then I would signed up for more, especially with the discount code. Otherwise, how am I supposed to even know what I’m signup up for? Every interaction I’ve had with the site so far is a sales attempt, so mostly I expect more of those.

I'd consider removing some questions that are bound to be country specific. e.g. The one about time spent in front of a red light.

>0.188

Slightly above avg - yay

Update at 2 hours: 1350+ quiz takers! 50% overconfident, 40% well-calibrated, and 10% underconfident. The average score is around 0.228, with the best score still at 0.007 (nearly perfect). The pattern so far is people are most overconfident in the 70-90% range, but are right closer to ~55% of the time.

Made a few changes based on feedback from this thread: full results now shown immediately with no email gate, changed the UX to include true/false/uncertain buttons + a confidence slider, I cleaned up the quiz result page, and fixed the die probability question. Thanks for all the honest feedback!

The slider disappearing when sliding between extremes is very confusing. I think the silver should be the only thing displayed and remove the buttons entirely.

Apologies if this is off-topic, but having spent more time than I'd like to admit having to create and edit webapps that emerged entirely out of Claude Code, Cursor, Codex, etc. with minimal to no direct code-writing by their human subscribers, this website has strong AI smells:

- Inter font

- all caps section headers

- Lucide icons

- em dashes, of course the em dashes

- bubble status badges (of course with all-caps "IN PROGRESS" and "COMING SOON" that mean the same thing)

- Uncited claims like "Most founders are overconfident in the 70-90% range" and "Most people score between 0.20 and 0.30"

- No less than FOUR blog articles all published April 4

None of these points is by any means a dealbreaker. And after all, I suppose a product should be judged on its merits and the value it delivers to its users, not on the tools used to create it. But together, the frontend bears the unmistakeable generative AI "smell" that telegraphs that the human(s) directing the tools building this app might be optimizing for speed over rigor and quality (further supported by the volunteer QA/QC happening in the comments), and may only be as good and reliable as the uncritically accepted outputs of a $20/month coding assistant.

heh.. nailed it first go:

Your Calibration Results 9/10 correct direction

Brier Score

0.131

Lower is better (0 = perfect)

Diagnosis

Well Calibrated Strong score. You were right more often than your confidence suggested. Trust your gut more.