A deep learning framework estimating cell types in a whole slide digital pathology image. The segmented nuclei and their corresponding cell types are provided in the legend.
Researchers at the University of Eastern Finland have developed a cutting-edge deep learning model to predict the presence and to count various cell types in the tumour micro-environment. The multi-modal deep learning approach is expected to improve the accuracy and efficiency of cancer diagnostics and treatment planning.
Identifying the different types of cells present in the tumour micro-environment can provide valuable information about the tumour’s histology and the underlying biology. Accurate and reliable cell type counting is also necessary for research and clinical applications. Researchers can use cell counts to study the distribution of different cell types in the tumour micro-environment and its association with patient outcomes. In clinical practice, cell counts can be used to monitor the response to therapy and track disease progression.
“There is a need for better methods for cell type counting in pathology images using deep learning approaches. Current approaches based on segmentation and regression have limitations such as requiring precise pixel-level annotations, difficulty in handling overlapping nuclei or obscured regions, and lack of information on individual cell type location. Additionally, probabilistic models tend to produce uncertain predictions and can result in over-confidence in predictions,” says Doctoral Researcher Raju Gudhe, who contributed to developing the proposed approach in the Cancer AI research team at the Institute of Clinical Medicine, University of Eastern Finland.
The study, which was presented at SPIE Medical Imaging 2023 in San Diego, proposes a new evidential multi-task deep learning approach, in short CT-EMT, to address the limitations of current methods for cell type counting in whole slide tumour images. The proposed approach formulates cell type density estimation and cell type counting as regression tasks, and nuclei segmentation as a pixel-level classification task. The authors have trained and evaluated the model on two open-source datasets: PanNuke and MoNuSAC. The proposed approach is illustrated below.
The deep learning framework, named CT-EMT, relies on evidential multi-task learning for predicting various cell types in Hematoxylin & Eosin-stained digital pathology images. CT-EMT effectively localizes the presence of nuclei, which further segments and categorizes the different cell types present in the given whole slide image.
The proposed cell type segmentation and counting approach has outperformed the state-of-the-art HoVer-Net and the StarDist models with relative improvements of 21% and 12%, in terms of mean panoptic quality. The developed model can provide compelling interpretations of diverse cell types and can be applied to various downstream tasks in computational pathology, such as tumour grading, prognosis, and treatment planning. This work will contribute to the development of more accurate and robust digital pathology tools that can assist pathologists and clinicians in diagnosing and treating cancer patients.
Some of the results obtained from their study are depicted below:
Performance evaluation of CT-EMT compared to the existing cell-type counting frameworks. The ground-truth annotations are overlayed on the original image patch. The bar-plot represents the estimated cell type counts with the colour codes given in the legend.
“The UEF Cancer AI research team aims to explore the potential of using deep learning technology in cancer and health data analysis,” says Senior Researcher Hamid Behravan of the University of Eastern Finland.
“Our study will involve the development and evaluation of cutting-edge deep learning algorithms for analysing cancer and various types of health-related data, including medical images, genomic data, and electronic health records. We believe that this approach has the potential to significantly improve the accuracy and efficiency of breast cancer diagnosis and treatment planning, as well as to facilitate the discovery of new insights and patterns in cancer data. We hope that our research will contribute to the advancement of precision medicine and the development of more effective and personalised approaches to breast cancer prevention and prognosis.”
Our multi-modal deep learning approach improves the accuracy and efficiency of breast cancer diagnosis and treatment planning.
More information about the study can be obtained from:
MSc., Doctoral Researcher Raju Gudhe, raju.gudhe(a)uef.fi
PhD., Senior Researcher Hamid Behravan, hamid.behravan(a)uef.fi, +358 50 513 9836
Professor Arto Mannermaa, arto.mannermaa(a)uef.fi, +358 40 355 2752
Reference:
Gudhe, Naga Raju, et al. "Predicting cell type counts in whole slide histology images using evidential multi-task learning." Medical Imaging 2023: Digital and Computational Pathology. Vol. 12471. SPIE, 2023.
