In earlier articles, I identified the significance of understanding how positive a mannequin is about its predictions.
For classification issues, it’s not useful to solely know the ultimate class. We’d like extra info to make well-informed choices in downstream processes. A classification mannequin that solely outputs the ultimate class covers essential info. We have no idea how positive the mannequin is and the way a lot we will belief its prediction.
How can we obtain extra belief within the mannequin?
Two approaches may give us extra perception into classification issues.
We might flip our level prediction right into a prediction set. The objective of the prediction set is to ensure that it incorporates the true class with a given likelihood. The dimensions of the prediction set then tells us how positive our mannequin is about its prediction. The less courses the prediction set incorporates, the surer the mannequin is.