loss does not converge

[DLC-jjj]

Hi, i have problem in training process of Pytorch version.I made no changes to the project and used the original BIPEDv2 dataset for training, and the parameters used the default parameters of the project. After training for 17 epochs, the loss barely changes. In the end, it can predict the image, but the effect is not very good. What could be the reason? Looking forward to your reply.

[xavysp]

Hi can I see the tensorboad graph?

[LvGuangzu]

Have you solved this problem? I also have this problem, looking forward to your reply, if it is not solved we can communicate a bit.

[DLC-jjj]

Have you solved this problem? I also have this problem, looking forward to your reply, if it is not solved we can communicate a bit.

I'm very sorry that I haven't solved it yet. I asked one of my classmates to try this project and encountered the same problem, which has not been solved yet.

[DLC-jjj]

Hi can I see the tensorboad graph?

Very sorry I didn't have tensorboard installed. The loss fluctuates from 1.5 to 3.5 from the first epoch to the end, so there is nothing wrong with the loss curve. I asked one of my classmates to try this project and encountered the same problem, and the following person also encountered the same problem. I'm a little troubled.

[xavysp]

Well if you used dexined without changing the tensorboad part, maybe you have the data, you just need to see the graph. We need to see if there is an improvement by epochs. The labels In edge detection are very sensitive, in some training samples may be detected more edges than in the GT and you'll find diferen loss value but that does not mean dexined is not training. You should check the average loss of each epoch. It happen in DL based edge detectors.

[xavysp]

Hi can I see the tensorboad graph?

Very sorry I didn't have tensorboard installed. The loss fluctuates from 1.5 to 3.5 from the first epoch to the end, so there is nothing wrong with the loss curve. I asked one of my classmates to try this project and encountered the same problem, and the following person also encountered the same problem. I'm a little troubled.

How many training data do you have?

[LvGuangzu]

I used the data set BIPED in the project in the training process, and the training set contains 200 pictures. My loss dropped to 0.9 in the 34th round, but it rose to 2.1 in the 35th round, and the overall loss was around 2.

[LvGuangzu]

Then I trained it again using the 0.9 pth file as my pretrained model and found that he started to oscillate around 1.7.

[LvGuangzu]

I read that your experiment in the paper was performed 150K times. I don't know if it is because I trained too few times. I trained for 100 rounds and found that there was no improvement.

[DLC-jjj]

嗨，我可以看到张量板图吗？

非常抱歉我没有张张。从第一个纪元结束到的，安装损失在 1.5 到 5 之间的波动量板，所以让项目失败没有任何问题。下面的人也遇到了同样的问题。我有点烦恼。

你有多少训练数据？

Thank you for your reply, I am using BIPEDv2 data, 200 training images. The training loss is similar to the loss function curve of the previous layer reply, and there is no tendency to converge.

[xavysp]

I used the data set BIPED in the project in the training process, and the training set contains 200 pictures. My loss dropped to 0.9 in the 34th round, but it rose to 2.1 in the 35th round, and the overall loss was around 2.

Did you change some hyperparameters? May you check with my lightweight model?
LDC: Lightweight Dense CNN for Edge Detection
I cannot find any error, just to make sure that the problem is not the data.

[LvGuangzu]

Yes, I changed some hyperparameters. I didn't change any hyperparameters on the first run, but found that the loss did not converge, I think it was a problem that the learning rate dropped too quickly, and then I modified the hyperparameters. First, is_testing=False; Then I modified the learning rate to drop 10x every 20 rounds.

[LvGuangzu]

Ok, I'll try to reproduce the LDC code over the next two days, and if it's not difficult to reproduce, I'll get back to you soon.

[LvGuangzu]

I used the data set BIPED in the project in the training process, and the training set contains 200 pictures. My loss dropped to 0.9 in the 34th round, but it rose to 2.1 in the 35th round, and the overall loss was around 2.

Did you change some hyperparameters? May you check with my lightweight model? LDC: Lightweight Dense CNN for Edge Detection I cannot find any error, just to make sure that the problem is not the data.

Hello, I have reproduced the LCD model, and I have modified some of the hyperparameters. Below I will list all the hyperparameters I have modified.

1. is_testing=True -> is_testing=False
2. epochs = 25 -> epochs = 50
3. adjust_lr = [6,12,18] -> adjust_lr = [12,24,36]
4. The BIPED dataset is still used for training, and resume=False is set, and the training is started from scratch.
   The final result shows signs of convergence.Below is the loss curve I got after training for 50 epochs.
   

[LvGuangzu]

I used the data set BIPED in the project in the training process, and the training set contains 200 pictures. My loss dropped to 0.9 in the 34th round, but it rose to 2.1 in the 35th round, and the overall loss was around 2.

Did you change some hyperparameters? May you check with my lightweight model? LDC: Lightweight Dense CNN for Edge Detection I cannot find any error, just to make sure that the problem is not the data.

There is one place I don't understand, as shown in the figure below, what is the role of the seed here? Will it affect the results if I delete it? Maybe I was careless and didn't see the introduction of the relevant content, I hope you can help me answer it.