The project uses Artificial Intelligence techniques, including deep machine learning methods, to analyze medical X-ray images and low-dose Computed Tomography from screening imaging for the early detection of pathological changes and prediction of the risk of death.

The aim of the project is to show that with the help of the Artificial Intelligence explanatory techniques used for the analysis of medical images (Trustworthy AI), it is possible to detect pathological changes early and assess the risk of death in screening tests, while providing clinically viable explanation for system output. The main tasks of the project include the development and validation of artificial intelligence models implementing the above tasks, and the development of explainability methods to understand the results of the model, in particular in terms of the impact of low-level and high-level image information.

The basic computing technology for deep machine learning methods, in particular in the field of convolutional neural networks (CNNs), are tensor calculations. One of the most efficient architectures performing tensor operations is the GPU. Due to the number and size of data, model sizes, and the need for iterative multi-epoch computations, supercomputing-class solutions best address the needs, providing computing power of GPUs, large resources of host and GPU memory, as well as quick access to data storage resources and metadata databases.

Artificial Intelligence requires a lot of data sets and metadata (e.g. labels) for training and validation. Often in the order of tens or even hundreds of thousands of images and related records. In case of medical imaging data derived from 3D imaging (eg Computed Tomography), a single data sample is often the entire imaging stack (the so-called series), with a size of 100MB (eg. 200 2D slices 512x512x2B each). The training set of tens of thousands data samples often requires tens of TB, and the process of training the model requires multiple reads of the whole set. Moreover, metadata, and in particular labels, are often assigned at the level of single slices or individual pixels of the study, what requires adequate database resources to efficiently store and search metadata resources in counts several orders of magnitude larger than the counts of the studies.

The task of chest screening analysis undertaken in the Project was based on data resources covering 90,000 Computed Tomography image series (including CDAS NLST study database). Due to dataset size and model complexity, solution to the following problems was required: efficient metadata management at the level of single images (18 million) for experiment planning, efficient access to data sets, efficient calculations, and large GPU memory for the model footprint and the training batch.

A dedicated database solution was prepared, integrating labeling database with the medical image storage system (PACS) for metadata management and image data addressing, based on fast data resources (SSD / NVME) and in-memory databases (IMDB). The „Rysy” cluster based on NVIDIA V100 32GB GPU cards was used for the computations – ensuring high efficiency of calculations and large memory resources for the model. The calculations were performed in Python programming language with the TensorFlow environment. Due to high I/O intensity (high data reading ratio in relation to the calculation time) and the large size of the training and validation set, it was necessary to provide hierarchical data access model for the GPU computing system. The data was stored entirely in Lustre file system („Tetyda”) and asynchronously allocated to fast local resources (SSD / NVME) of the computing cluster.

The use of HPC infrastructure:

• allowed training models with greater complexity at higher;
• batch-size values;
  reduced the metadata database response time several orders of magnitude;
• reduced data access time several orders of magnitude.