Skip to content

Commit 1ffcc74

Browse files
author
György Kovács
committed
docs finalized
1 parent e063bb4 commit 1ffcc74

File tree

2 files changed

+5
-13
lines changed

2 files changed

+5
-13
lines changed

README.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -569,7 +569,7 @@ Experts in various fields are kindly invited to contribute further test bundles
569569
Retinal vessel segmentation
570570
---------------------------
571571

572-
One such field is retinal vessel segmentation [RV]_, where the authors have the option to include or exclude certain parts of the images (the pixels outside the Field-of-View), making the reported scores incomparable. To facilitate the meaningful comparison, evaluation and interpretation of reported scores, we provide two functions to check the internal consistency of scores reported for the DRIVE retinal vessel segmentation dataset.
572+
The segmentation of the vasculature in retinal images [RV]_ gained enormous interest in the recent decades. Typically, the authors have the option to include or exclude certain parts of the images (the pixels outside the Field-of-View), making the reported scores incomparable. (For more details see [RV]_.) To facilitate the meaningful comparison, evaluation and interpretation of reported scores, we provide two functions to check the internal consistency of scores reported for the DRIVE retinal vessel segmentation dataset.
573573

574574
The first function enables the testing of performance scores reported for specific test images. Two tests are executed, one assuming the use of the FoV mask (excluding the pixels outside the FoV) and the other assuming the neglect of the FoV mask (including the pixels outside the FoV). As the following example illustrates, one simply provides the scores and specifies the image (whether it is from the 'test' or 'train' subset and the image identifier) and the consistency results with the two assumptions are returned.
575575

docs/02b_retina.rst

Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
Retinal vessel segmentation
2-
===========================
2+
---------------------------
33

4-
The segmentation of the vasculature in retinal images gained enormous attention in the recent decade. Recently, using the tools implemented in this package, it was shown in [RV]_ that the authors use at least two different evaluation techniques, leading to incomparable performance scores and skewing the field. Retinal images have valueable image content only in the circular Field-of-View (FoV) region:
4+
The segmentation of the vasculature in retinal images [RV]_ gained enormous interest in the recent decades. Typically, the authors have the option to include or exclude certain parts of the images (the pixels outside the Field-of-View), making the reported scores incomparable. (For more details see [RV]_.) To facilitate the meaningful comparison, evaluation and interpretation of reported scores, we provide two functions to check the internal consistency of scores reported for the DRIVE retinal vessel segmentation dataset.
55

6-
.. image:: https://rumc-gcorg-p-public.s3.amazonaws.com/i/2020/01/21/75d803ef.png
7-
8-
When a binary segmentation of the vasculature is evaluated, some authors do calculate the performance scores only in the FoV region, others involve also the pixels outside the FoV region, which increases the number of true negatives (and consequently, accuracy and specificity) enormously. The functionalities implemented in this package are suitable to distinguish the two kinds of evaluations based on the reported performance scores.
9-
10-
The package contains the statistics of the images of the DRIVE dataset, and provides the functionality of checking the consistency of reported scores with the assumptions of using the FoV pixels only or using all pixels for evaluation, for both image level figures and score aggregated for all images of the DRIVE dataset.
11-
12-
Again, we highlight, that the techniques in the package detect inconsistency with certainty. If the use of the FoV pixels only was found to be inconsistent with a certain set of scores, the user can conclude that these scores are not comparable with other scores which are calculated in the FoV region.
13-
14-
The first function enables the testing of performance scores reported for certain test images, the two tests executed assume the use of the FoV mask (excluding the pixels outside the FoV) and the neglection of the FoV mask (including the pixels outside the FoV). As the following example shows, one simply supplies the scores and specifies the images (whether it is from the 'test' or 'train' subset and the identifier of the image) and gets back if inconsistency is identified with any of the two assumptions.
6+
The first function enables the testing of performance scores reported for specific test images. Two tests are executed, one assuming the use of the FoV mask (excluding the pixels outside the FoV) and the other assuming the neglect of the FoV mask (including the pixels outside the FoV). As the following example illustrates, one simply provides the scores and specifies the image (whether it is from the 'test' or 'train' subset and the image identifier) and the consistency results with the two assumptions are returned.
157

168
.. code-block:: Python
179
@@ -23,7 +15,7 @@ The first function enables the testing of performance scores reported for certai
2315
identifier='01')
2416
# {'fov_inconsistency': True, 'no_fov_inconsistency': True}
2517
26-
The interpretation of these results is that the reported scores are inconsistent with any of the reasonable evaluation methodolgoies.
18+
The interpretation of these results is that the reported scores are inconsistent with any of the reasonable evaluation methodologies.
2719

2820
A similar functionality is provided for the aggregated scores calculated on the DRIVE images, in this case the two assumptions of using the pixels outside the FoV is extended with two assumptions on the way of aggregation.
2921

0 commit comments

Comments
 (0)