Evaluation Metric in Machine Learning

It is very important to check efficiency of the Machine Learning model. To check this, you have to evaluate the model based on some metric. But more important is a choice of metric to evaluate your machine learning model.

Confusion Metric

Confusion Metric is use to check the accuracy or correctness of your model. It is represent as :

Definition of the terms used in confusion metric –

  • True Positive -> Observation = True (1) & Prediction = True (1)
  • False Positive -> Observation = False (0) & Prediction = True (1)
  • False Negative -> Observation = True (1) & Prediction = False (0)
  • True Negative -> Observation = False (0) & Prediction = False (0)

Accuracy

Accuracy is given by the formula:

This image has an empty alt attribute; its file name is Confusion_Matrix2_2.png

Accuracy is good to measure in case of the categorical data means the target classes are approximately balanced. And do not use when the target variable are continuous, it may lead you to wrong direction.

Precision

Precision is given by the formula:

This image has an empty alt attribute; its file name is Confusion_Matrix4_4.png

Precision is defined as the ratio of true positive value to the total number of predicted positive value, means precision is the number of correct results divided by the number of all returned results.

Precision tells, when it predicts correct, how often is it correct.

Recall

Recall is given by the formula:

This image has an empty alt attribute; its file name is Confusion_Matrix3_3.png

Recall is the ratio of total number of correctly classified positive value divide to the total number of positive values. According to Wikipedia definition recall is the number of correct results divided by the number of results that should have been returned.

Recall tells, when it is actually correct, how often does it predict correct.

F Measure

Both Recall and Precision are combined into single measure is called F Measure. It is a measure of test’s accuracy and most commonly use metric for classification problems.

Alone recall and precision is not enough, means we can have good precision and worst recall or worst precision and good recall, also we cannot calculate both every time. So, we use F Measure which takes the Harmonic mean of precision and recall, where an F Measure reaches its best value at 1 and worst score at 0. We did not take Arithmetic mean because it may fail in some cases.

F Measure is given by the formula:

This image has an empty alt attribute; its file name is Confusion_Matrix5_5.png

F measure is also called as F1-Score or Dice similarity coefficient.

Leave a Reply

UP↑