Skip to content

[FEATURE_REQUEST] Help users understand pycocotools metrics output for undefined values (e.g. AP reported as "-1") in sample notebooks #401

@mawah

Description

@mawah

Description

In e.g. the 01_training_introduction.ipynb notebook in the object detection folder, after each epoch is complete, output from pycocotools such as the following is displayed:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.420
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.823
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.333
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.420
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.526
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.598
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.598
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.598

Users unfamiliar with pycocotools may question what they are seeing, especially why the AP and AR are sometimes reported as "-1." (In fact, it is unclear whether users will be familiar with these metrics at all, and I would consider including a description of the metrics in the notebook.)

Expected behavior with the suggested feature

The markdown cell above cells that will produce strange pycocotools output should describe the undefined metric values that will be displayed, reassuring novice users that no error has occurred. Alternatively, the default pycocotools output can be modified so that the undefined metric values are not displayed at all.

Other Comments

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug-bashIssues that you came across during the bug-bash

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions