Unable to even overfit

[olofhq]

Hi,

I've been trying to train a model for pose estimation but I've been unable to do so successfully. My last test has been to extract so that I only have 10 images in the training set with a batch size of 10 images to see if I can overfit to those 10 images. But it is not able to overfit to those even after 2000 epochs and testing multiple different learning rates.

I've also written a visualization script and ensured that the the dataset looks fine. I've used the same visualization on the coco-dataset gotten from the converter and the format looks the same.

I also have my own training script and when using that one I'm able to train a good model, using the same data, that has roughly the same number of parameters, and the same backbone, as the st_movenet_lightning_heatmaps model.

I'm unable to find the culprit for what is going wrong. My dataset is only using 6 keypoints and it is of a non-human object, could this be the issue? I've changed def f3(): return tf.ones(nb_kpts, tf.float32) * 0.08 to still have a standard deviation for my points.

The maximum oks achieved when overfitting to 10 images was 0.8 but when running the prediction mode it only looks good on some of the images whilst it looks mostly random on most of the 10 training images.

[olofhq]

Hi again,

I've looked more into the code and found that the GT for the heatmaps is just ones for the exact pixel that is the correct keypoint and then zero everywhere else. I think that can partly explain the poor performance as I think most heatmap methods use gaussian centered GT, for example Simple Baseline and HRNet use gaussians.

[RSERSTM]

Hi,

I've been trying to train a model for pose estimation but I've been unable to do so successfully. My last test has been to extract so that I only have 10 images in the training set with a batch size of 10 images to see if I can overfit to those 10 images. But it is not able to overfit to those even after 2000 epochs and testing multiple different learning rates.

I've also written a visualization script and ensured that the the dataset looks fine. I've used the same visualization on the coco-dataset gotten from the converter and the format looks the same.

I also have my own training script and when using that one I'm able to train a good model, using the same data, that has roughly the same number of parameters, and the same backbone, as the st_movenet_lightning_heatmaps model.

I'm unable to find the culprit for what is going wrong. My dataset is only using 6 keypoints and it is of a non-human object, could this be the issue? I've changed def f3(): return tf.ones(nb_kpts, tf.float32) * 0.08 to still have a standard deviation for my points.

The maximum oks achieved when overfitting to 10 images was 0.8 but when running the prediction mode it only looks good on some of the images whilst it looks mostly random on most of the 10 training images.

Hi Olofhq,

Have you tried commenting the data-augmentation part of your config file ?

Thanks,

[olofhq]

Hi,

Yes I've tried that. When completely commenting out data_augmentation I get a crash but when keeping only random_brightness with 0.3 as a factor, and commenting out all other parameters, I still get that it cannot overfit to a small dataset.