.svg)
.svg)
Confusion Matrix
A prerequisite to understanding precision and recall in machine learning is gaining a grasp of the confusion matrix. Commonly, the majority group is known as the negative result while the less populous group is the positive result. The confusion matrix provides a clear depiction of the predictive model’s performance, indicating which classes are correctly forecasted and which are not, and shedding light on the types of errors being made. In a binary classification problem, the simplest matrix entails negative (class 0) and positive (class 1) groups. The terminologies true positives and false negatives encapsulate the metrics of precision and recall within such a confusion matrix.
Precision
The model's precision addresses how accurately a positive outcome is predicted. The formula for precision in machine learning is: True Positives / (True Positives + False Positives). It proves beneficial when false positives carry a heavy cost. Take for example, the detection of a rare disease. Employing a model with low precision could result in a slew of patients receiving a disease diagnosis they don't have - leading to many cases of misdiagnosis. This in turn leads to unnecessary stress and additional check-ups. Overwhelming number of false positives will eventually be ignored by the monitoring individuals. In the following, a deeper exploration of precision and its calculation will be undertaken given this brief introduction and revisit of the confusion matrix.
Binary Classification and Precision
The range of precision spans from 0.0 (no precision) to 1.0 (absolute precision). If a model designated 150 cases as minority class members, 110 correctly and 40 incorrectly, the precision of the model would be calculated:
Precision = 110 / (110+40)
Precision = 110 / 150
Precision = 0.73
The proportion of correctly predicted positive results to overall positive predictions made becomes synonymous with precision. Yet, precision does not encapsulate the entire story as it does not account for true positive class examples incorrectly dubbed as negatives (false negatives).
Multi-class Classification and Precision
For an imbalanced classification problem involving more than two classes, precision is calculated as:
Precision formula in Multi-class Classification = True Positives in all classes / (True Positives + False Positives in all classes)
This will be applicable in a case where one negative majority class exists but two positive minority classes (class 1 and 2) are present. Precision would express the percentage of correct predictions across both positive classes. Assuming the model predicts 100 cases for the first minority class with 90 correct and 10 incorrect, and for the second class it predicts 175 cases with 150 correct and 25 incorrect, then the model's ML precision would be calculated as follows:
Precision = (90 + 150) / ((90 + 150) + (10 + 25))
Precision = 240 / (240 + 35)
Precision = 240 / 275
Precision = 0.87
Accuracy
The model's training progress and overall performance can be initially gauged by its accuracy. However, it does not shed light on the specifics of its application to the problem at hand. When dealing with extensive class division, accuracy as a primary measure of success in ML can be misleading. For skewed datasets, accuracy can present an illusion of performance. For example, in a study with 95 negative outcomes and 5 positive ones, categorizing all values as negative would result in a 95% score. Other metrics can give a more comprehensive understanding of performance, not impaired by such imbalance.
Conclusion
Precision calculates the percentage of positive class predictions that genuinely belong to the positive class. It's often partnered with recall and combined in the F1 Score for overall performance measurement. In the world of information retrieval, precision assumes a different meaning from 'accuracy' and other conventional scientific/technical terms.
