[kitti-evaluation-toolkit-format-labels-dumping] Minor code changes to dump prediction labels in KITTI evaluation toolkit expected format.

[viplix3]

A brief overview of changes done and the files affected mentioned below.

• Retaining box prediction confidence when converting the box from LiDAR coordinate frame to camera coordinate frame. File changed:

  • src/data_process/kitti_bev_utils.py
  • src/data_process/transformation.py
  • src/utils/visualization_utils.py

• Dumping the output labels in the KITTI evaluation toolkit expected format. Files changed:

  • src/test.py
  • src/utils/misc.py

[SofianeB-03]

hello @viplix3, thank you! Have you evaluate with a kitti tool ? Because I evaluate with https://github.com/prclibo/kitti_eval but I have a wrong result.

[viplix3]

@SofianeB-03 yes, I was able to evaluate using the KITTI evaluation toolkit.
I used this GitHub repo.
I used the pre-trained weights of the complex-yolo model shared by the author of this repository as well as one I trained myself.

It would be helpful if you could elaborate in what sense are you getting the wrong results.

[SofianeB-03]

Thank you for your help. My results are: AP ~ 0.02 for pedestrian and cyclist and ~0.20 for car. It seems wrong (with the evaluation from evaluate.py on this repo, I have mAP ~ 0.88).

[SofianeB-03]

@viplix3 I used the same kitti tool to evaluate this model ComplexYolov4 with pre-trained weights on this repo (on Validation dataset kitti) but I have wrong results again : AP ~ 0.0 Ped,Cycl and AP ~ 0.10 for car. Can you share your results of the pre-trained model from this repo please?

[viplix3]

@SofianeB-03, for the model I trained, I am getting these numbers on the KITTI test sub-set I've created from KITTI training data. This subset has not been used in the training/validation set of the model.

BEV (Easy/Moderate/Hard)
Pedestrian: (71.67, 74.56, 74.74)
Cyclist: (59.54, 64.67, 65.10)
Car: (98.89, 96.90, 96.86)

3D (Easy/Moderate/Hard)
Pedestrian: (56.27, 56.37, 56.85)
Cyclist: (1.26, 0.90, 1.96))
Car: (46.17, 40.91, 43.01)

[SofianeB-03]

Thank you very much !

[Petros626]

@viplix3 open for questions related to BEV evaluation with KITTI?

[viplix3]

Hi @Petros626, it's been quite a while since I last worked with KITTI, so my memory might be a bit rusty. That said, I’d be happy to help with any questions you have around BEV evaluation—feel free to ask!

[Petros626]

@viplix3 Thank you very much for the quick response!

I would write you an email and post the final solution to my problem here as a comment.

[Petros626]

@viplix3 did you received my mail?