You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.rst
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -569,7 +569,7 @@ Experts in various fields are kindly invited to contribute further test bundles
569
569
Retinal vessel segmentation
570
570
---------------------------
571
571
572
-
One such field is retinal vessel segmentation [RV]_, where the authors have the option to include or exclude certain parts of the images (the pixels outside the Field-of-View), making the reported scores incomparable. To facilitate the meaningful comparison, evaluation and interpretation of reported scores, we provide two functions to check the internal consistency of scores reported for the DRIVE retinal vessel segmentation dataset.
572
+
The segmentation of the vasculature in retinal images [RV]_ gained enormous interest in the recent decades. Typically, the authors have the option to include or exclude certain parts of the images (the pixels outside the Field-of-View), making the reported scores incomparable. (For more details see [RV]_.) To facilitate the meaningful comparison, evaluation and interpretation of reported scores, we provide two functions to check the internal consistency of scores reported for the DRIVE retinal vessel segmentation dataset.
573
573
574
574
The first function enables the testing of performance scores reported for specific test images. Two tests are executed, one assuming the use of the FoV mask (excluding the pixels outside the FoV) and the other assuming the neglect of the FoV mask (including the pixels outside the FoV). As the following example illustrates, one simply provides the scores and specifies the image (whether it is from the 'test' or 'train' subset and the image identifier) and the consistency results with the two assumptions are returned.
Copy file name to clipboardExpand all lines: docs/02b_retina.rst
+4-12Lines changed: 4 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,9 @@
1
1
Retinal vessel segmentation
2
-
===========================
2
+
---------------------------
3
3
4
-
The segmentation of the vasculature in retinal images gained enormous attention in the recent decade. Recently, using the tools implemented in this package, it was shown in [RV]_ that the authors use at least two different evaluation techniques, leading to incomparable performance scores and skewing the field. Retinal images have valueable image content only in the circular Field-of-View (FoV) region:
4
+
The segmentation of the vasculature in retinal images [RV]_ gained enormous interest in the recent decades. Typically, the authors have the option to include or exclude certain parts of the images (the pixels outside the Field-of-View), making the reported scores incomparable. (For more details see [RV]_.) To facilitate the meaningful comparison, evaluation and interpretation of reported scores, we provide two functions to check the internal consistency of scores reported for the DRIVE retinal vessel segmentation dataset.
When a binary segmentation of the vasculature is evaluated, some authors do calculate the performance scores only in the FoV region, others involve also the pixels outside the FoV region, which increases the number of true negatives (and consequently, accuracy and specificity) enormously. The functionalities implemented in this package are suitable to distinguish the two kinds of evaluations based on the reported performance scores.
9
-
10
-
The package contains the statistics of the images of the DRIVE dataset, and provides the functionality of checking the consistency of reported scores with the assumptions of using the FoV pixels only or using all pixels for evaluation, for both image level figures and score aggregated for all images of the DRIVE dataset.
11
-
12
-
Again, we highlight, that the techniques in the package detect inconsistency with certainty. If the use of the FoV pixels only was found to be inconsistent with a certain set of scores, the user can conclude that these scores are not comparable with other scores which are calculated in the FoV region.
13
-
14
-
The first function enables the testing of performance scores reported for certain test images, the two tests executed assume the use of the FoV mask (excluding the pixels outside the FoV) and the neglection of the FoV mask (including the pixels outside the FoV). As the following example shows, one simply supplies the scores and specifies the images (whether it is from the 'test' or 'train' subset and the identifier of the image) and gets back if inconsistency is identified with any of the two assumptions.
6
+
The first function enables the testing of performance scores reported for specific test images. Two tests are executed, one assuming the use of the FoV mask (excluding the pixels outside the FoV) and the other assuming the neglect of the FoV mask (including the pixels outside the FoV). As the following example illustrates, one simply provides the scores and specifies the image (whether it is from the 'test' or 'train' subset and the image identifier) and the consistency results with the two assumptions are returned.
15
7
16
8
.. code-block:: Python
17
9
@@ -23,7 +15,7 @@ The first function enables the testing of performance scores reported for certai
The interpretation of these results is that the reported scores are inconsistent with any of the reasonable evaluation methodolgoies.
18
+
The interpretation of these results is that the reported scores are inconsistent with any of the reasonable evaluation methodologies.
27
19
28
20
A similar functionality is provided for the aggregated scores calculated on the DRIVE images, in this case the two assumptions of using the pixels outside the FoV is extended with two assumptions on the way of aggregation.
0 commit comments