Why does the model's performance decrease when using a pre-trained model and then training on the Mapillary Vistas dataset?

[1183498834]

Initially, I mixed the Mapillary Vistas dataset with my custom dataset for training, and observed a degradation in model performance. After investigation, I concluded that my custom data was not problematic. Therefore, I proceeded to train solely on the Mapillary Vistas dataset. As you can see from the images, most objects are still being annotated after training, but the Lane Marking - General markings have disappeared. This indicates a decline in model performance, and I'm curious as to why this happened?​