Confidence Scores for Exam Questions (nomagicpill.substack.com) 14 points by surprisetalk 3mo ago ↗ HN
[–] vmilner 3mo ago ↗ I seem to remember some medical related multiple choice tests in the UK use a mechanism of +1 for correct , 0 for unanswered , -1 for incorrect.
[–] esafak 3mo ago ↗ https://en.wikipedia.org/wiki/Calibration_(statistics)https://en.wikipedia.org/wiki/Scoring_rule
[–] clickety_clack 3mo ago ↗ It would make more sense to just use IRT for grading the responses than trying to add more complexity to the answers themselves.
[–] korkoros 3mo ago ↗ Moodle has had certainty based marking for at least 15 years.https://docs.moodle.org/502/en/Using_certainty-based_marking
[–] krackers 3mo ago ↗ Terence tao had a blogpost about this for the case of true/false questions, with multiple choice left as an exercise for the reader.https://terrytao.wordpress.com/2016/06/01/how-to-assign-part...At least with the formulation Tao had, one unfortunate side-effect is that if you answer with 100% confidence and get it wrong, you basically fail the entire course.
5 comments
[ 3.8 ms ] story [ 22.5 ms ] threadhttps://en.wikipedia.org/wiki/Scoring_rule
https://docs.moodle.org/502/en/Using_certainty-based_marking
https://terrytao.wordpress.com/2016/06/01/how-to-assign-part...
At least with the formulation Tao had, one unfortunate side-effect is that if you answer with 100% confidence and get it wrong, you basically fail the entire course.