Furthermore, models trained from data annotated with lower inter-labeler variability outperform those from higher inter-labeler variability. Instead, decreasing the inter-rater variability with the expense of decreasing dataset size increased the model performance. We found that naively increasing the data size at the expense of inter-rater variability does not necessarily lead to better-performing models in cell detection. We propose a method to measure such variability, and by excluding those annotators with low variability, we verify the trade-off between the amount of data and its quality. We present a large-scale study on the variability of cell annotations among 120 board-certified pathologists and how it affects the performance of a deep learning model. Under this setting, the relation between annotators' variability and model performance has received little attention. In particular, annotating different types of cells in histopathology suffer from high inter- and intra-rater variability due to the ambiguity of the task. However, annotating medical images is challenging as it requires expertise and a large budget. Large annotated datasets have been a key component in the success of deep learning.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |