Xomnia, the Leiden University Medical Center (LUMC), and the Dutch national research institute for mathematics and computer science (CWI: Centrum Wiskunde & Informatica) are collaborating to advance the precision of medical treatments, specifically radiation therapy for cervical cancer patients.
Central to this is the task of deformable image registration (DIR). DIR is the process of aligning, i.e., matching, different imaging scans of a patient, which is necessary as the anatomy visualized in the scans for the same patient can look quite different when taken at different points in time e.g., following treatment courses.
The team is developing a system that can break through barriers in DIR by enabling insightful tuning, increased robustness, and obtaining accurate and realistic deformations for hard DIR problems. This project also seeks to develop machine learning based segmentation (i.e., delineation) of organs/regions of interest, which will be used in different ways to advance the DIR approach (e.g., as guidance information in the DIR process).
The aim of this collaboration is to ultimately allow radiation treatments to be planned in a better way, and precisely target the cancer while sparing the surrounding organs as much as possible, hence helping in increasing the precision of the treatment.
“Existing DIR approaches are single-objective and notoriously hard to tune, impeding their clinical use. In this project, we develop a novel, multi-objective approach to DIR to remove this problem, introducing new models and making synergies with machine learning and biomechanical modeling to account for large deformations,” Dr. Tanja Alderliesten, Associate Professor at the Department of Radiation Oncology of the Leiden University Medical Center
Case
As a cancer patient undergoes different CT and/or MRI exams over time, the resulting images can look quite different from each other. This can be due to the shape of a tumor changing, different patient positioning when taking the scan, a full vs. empty bladder...etc. For clinical practice, it is important to be able to match the anatomy visualized in these images.
“DIR is especially hard when structures have (dis)appeared (i.e., content mismatch). Moreover, especially when deformations are large, it is not always immediately clear how much deformation is desirable to obtain a better image match because an exact image match may actually not be possible. This reveals the true nature of the optimization problem underlying DIR: it is multi-objective,” says Prof. Dr. Peter Bosman from the CWI.
In 2017, Dr. Tanja Alderliesten initiated this project while working at the Amsterdam UMC (she later moved to work as an associate professor at the department of Radiation Oncology of the LUMC). The project was co-initiated by Prof. Dr. Peter Bosman. In total, four researchers (one postdoctoral researcher and three PhD students), and a radiation therapy technologist are appointed on the project. In addition, Elekta, a company specialized in precision radiation medicine is closely involved. The research further entails close collaboration with radiation oncologists and radiotherapy medical physics experts.
As part of Xomnia’s AI for Good initiative, we were keen on contributing to this research by assigning a few hours every week to our consultants to help the research team in their mission.
Solution
Xomnia’s consultants are helping in making a model to segment organs in the scans using deep neural networks, through which the research team aims to find a solution that enhances the DIR approach, by enabling tackling the hard DIR problems that involve large deformations and/or content mismatches (e.g., anatomy or instrumentation) between scans.
Currently, a basic UNet model is applied on 3D CT scans of the abdomen area to do image segmentation. However, since clinical data is used in a retrospective setting of patients treated for various tumors in the abdomen, contours of organs with labels of interest for cervical cancer patients were not available for many images, leaving us with a relatively small training set.
Starting 2021, Xomnia’s machine learning engineer Geert Klop joined the team and started applying semi-supervised learning, which is relatively new in the field. This type of learning utilizes both images with and without labels. The model is first trained on images with labels and then for the other images, pseudo labels are predicted by the model. Then the model is trained in an iterative fashion on the total set, with the aim to increase its performance.
The researchers will continue to develop the important building blocks for this novel approach to DIR, with a focus on application in the field of radiation oncology, but the technology will also be usable outside that field.
Impact
The current results of this collaboration are segmentation models and associated performance metrics. In the future, these should be incorporated into the software of the multi-objective approach to DIR that creates a set of high quality DIR solutions with different trade offs from which the treatment provider can choose the preferred solution.
The team published their first findings in 2023 in the published research "Clinically Acceptable Segmentation of Organs at Risk in Cervical Cancer Radiation Treatment from Clinically Available Annotations". In this work, they show that by incorporating unlabeled data in a smart way, significant performance gain is obtained on all relevant metrics and the resulting model produces segmentations that are clinically acceptable.