A Confusion matrix is an N x N matrix used for evaluating the performance of a classification model, where N is the number of target classes. The matrix compares the actual target values with those predicted by the machine learning model. This gives us a holistic view of how well our classification model is performing and what kinds of errors it is making.
True Positive (TP)
The predicted value matches the actual value. The actual value was positive and the model predicted a positive value
True Negative (TN)
The predicted value matches the actual value. The actual value was negative and the model predicted a negative value
False Positive (FP) – Type 1 error
The predicted value was falsely predicted. The actual value was negative but the model predicted a positive value Also known as the Type 1 error
False Negative (FN) – Type 2 error
The predicted value was falsely predicted. The actual value was positive but the model predicted a negative value Also known as the Type 2 error
TruePositives / (TruePositives + FalsePositives)
This would determine whether our model is reliable or not.
TruePositives / (TruePositives + FalseNegatives)
F1-score is a harmonic mean of Precision and Recall, and so it gives a combined idea about these two metrics. It is maximum when Precision is equal to Recall.
But there is a catch here. The interpretability of the F1-score is poor. This means that we don’t know what our classifier is maximizing – precision or recall? So, we use it in combination with other evaluation metrics which gives us a complete picture of the result.
F-Measure = (2 * Precision * Recall) / (Precision + Recall)
# confusion matrix in sklearn from sklearn.metrics import confusion_matrix from sklearn.metrics import classification_report # actual values actual = [1,0,0,1,0,0,1,0,0,1] # predicted values predicted = [1,0,0,1,0,0,0,1,0,0] # confusion matrix matrix = confusion_matrix(actual,predicted, labels=[1,0]) print('Confusion matrix : \n',matrix) # outcome values order in sklearn tp, fn, fp, tn = confusion_matrix(actual,predicted,labels=[1,0]).reshape(-1) print('Outcome values : \n', tp, fn, fp, tn) # classification report for precision, recall f1-score and accuracy matrix = classification_report(actual,predicted,labels=[1,0]) print('Classification report : \n',matrix)
Confusion matrix : [[2 2] [1 5]] Outcome values : 2 2 1 5 Classification report : precision recall f1-score support 1 0.67 0.50 0.57 4 0 0.71 0.83 0.77 6 accuracy 0.70 10 macro avg 0.69 0.67 0.67 10 weighted avg 0.70 0.70 0.69 10