Four groups:
Extreme confidence reduced in groups 3 and 4.
Small effect.
Dan Quayle
from The making of a senator, by Richard F. Fenno, Jr.
We are imperfect judges of confidence. We make errors in both directions. But when our judgment is extreme, errors can only go in one direction.
Thus, when we say 100%, any error will be in the direction of lower accuracy than 100%.
This is psychologically uninteresting, but it has great practical implications.
Yes | No | No | Yes | Yes | Linear | Quadratic | |
A | .90 | .10 | .40 | .80 | .30 | ||
B | .80 | .00 | .30 | .90 | .10 | ||
A' | 1.00 | .00 | .00 | 1.00 | .00 | ||
B' | 1.00 | .00 | .00 | 1.00 | .00 | ||
Ea | .10 | .10 | .40 | .20 | .70 | 1.50 | .71 |
Eb | .20 | .00 | .30 | .10 | .90 | 1.50 | .95 |
Ea' | .00 | .00 | .00 | .00 | 1.00 | 1.00 | 1.00 |
Eb' | .00 | .00 | .00 | .00 | 1.00 | 1.00 | 1.00 |
Linear score (SUM) provides no incentive not to exaggerate probabilities.
Square is "strictly proper." Encourages best guess.