Unbalanced Image Classification Testing Properly

[Spawnfile]

Hello first of all, I have a unbalanced classification dataset. Which method/approach should I use to test this model ? Is confusion matrix enough or there is other metrics that I should use ?

Suggestions will be greatly appreciated.

[SkafteNicki]

In general it depends on how unbalanced your dataset is. A small unbalance is not nessesarily something you need to take care of.
That said, vizualizing the confusion matrix is a great start at getting a feeling how the unbalance effects the performance of your model. In general I would not recommend only looking at the accuracy of your model, as accuracy is prone to giving too high scores when data is unbalanced. Instead consider also calculating the Recall and Precision.

You could always consider doing the following to counter the unbalance in your dataset:

• re-weightning your loss function
• under-sample your over represented class
• over-sample your under represented class

[Spawnfile]

Thanks for your suggestions. I am using weighted loss function already but my essential interested point is that over-sample methods like SMOTE which I will implement to my code, causing some side effects on the minority class like overfitting ?