0.3 Model performance metrics
In classification models, the accuracy of predictions is a generally important metric. However, several other metrics may become more important depending on the nature of the application we deal with.
The following set of model metrics are verbatim reproduced from the help text for confusionMatrix() function from caret package.
Suppose a 2X2 table with notation
| Predicted | Event | No Event |
|---|---|---|
| Event | A | B |
| No Event | C | D |
The formulas used here are:
\[ Sensitivity = \frac{A}{A+C} \]
\[ Specificity = \frac{D}{B+D} \]
\[Prevalence = \frac{A+C}{A+B+C+D}\]
\[PPV = \frac{Sensitivity * Prevalence}{Sensitivity*Prevalence+(1-Specificity)*(1-Prevalence)}\]
\[NPV = \frac{Specificity * (1-Prevalence)}{(1-Sensitivity)*Prevalence + Specificity*(1-Prevalence)}\]
\[ Detection\ Rate = \frac{A}{A+B+C+D}\]
\[Detection\ Prevalence = \frac{A+B}{A+B+C+D}\]
\[Balanced\ Accuracy = \frac{Sensitivity+Specificity}{2}\]
\[Precision = \frac{A}{A+B}\]
\[Recall = \frac{A}{A+C}\]
\[F1 = \frac{(1+\beta^2)*Precision*Recall}{(\beta^2 * Precision)+Recall}\]
where \(\beta\) = 1 for this function.