Indexing Medicine with Smart Image Databases

Anomaly detection in patients is challenging because reference values are vague

Identifying patients with enlarged livers is important for identifying hepatitis, systemic disease, and even chronic conditions like alcohol abuse. Oversized is, however, a very poorly defined term. What is normal? Currently medicine operates on the basis of painstaking, time-consuming studies made from relatively homogenous populations. Using this ruler means many sick patients are not identified.

Precision Medicine with IQAE

By combining large sets of information about patients together, finding outliers and treating patients can be much better matched to the individual background of that patients. Rather than comparing the patient to an old publication, they can be compared to a group matching their age, gender, and lifestyle.

The graph below shows the same patient with (left) and without (right) precision medicine.

The precision case is clear, the patient’s liver is definitely larger than it should be.
The standard medicine is inconclusive, a bit larger, but nothing definitive.

4Quant

Despite recent boosts in popularity, both personalized and precision medicine are still not part of the standard diagnosis process for many diseases.

Image Query and Analysis Engine

Standard imaging modalities like thorax CT and MRI are commonly used to diagnose a number of different diseases. These images are normally examined once for a single specific diagnosis and then archived. Hospitals are required to archive a majority of patient imaging studies and scans for a minimum of 10 years. These essential data are just sitting unused on PACS, tapes, and other storage volumes.

These archives of images contain exactly the sort of information required to make precision diagnoses, they are simply difficult to access.

IQAE is the key to unlocking the potential of this data and turning it from a burden into an asset. The data can be analyzed in a completely anonymized manner to obtain general, precise statistics as references against which to compare new patients. Additionally careful analysis of all data can reveal mistakes, missed diagnoses, and new potential high risk patients.

Real-time Querying of Images

A hospital database with thousands of patient images can be quickly screening for the right type of images.

SELECT Image as ChestCT FROM PatientImages WHERE Modality="CT" AND Region="Chest"

Perform Segmentation

SELECT CHEST_SEGMENTATION(Image) as ChestSeg FROM ChestCT

To individual organs

Extract the bones

SELECT BoneImage FROM ChestSeq

Extract Meaningful Information

SELECT pt.Age, EstimateBoneMineralDensity(cs.BoneImage) as BMD FROM PatientDatabase AS pt JOIN ChestSeq AS cs ON pt.id == cs.PatientId

4Quant

What else?

Our Image Query and Analysis Engine runs in the cloud or locally and is used as Software as a Service to ensure you are always working with the latest generation of algorithms and tools. Once the image data is inside the possibilities are endless.

SELECT * FROM ChestSeq GROUP BY Tissue.Type

4Quant

From all of the images general trends can be identified by examining all of the phenotypes and trying to identify the important ones for differentiating disease. The following figure shows the relationship between shape (of the super-pixels) and the healthy segments as pink dots and the abnormal as blue dots. The shape and position provide some differentation but are not definitive enough to clearly distinguish the two groups.

Here we show the relative color components for each channel (red, green, and blue) and how they are related to the tissue labeled as healthy and abnormal.

Here rather than showing the relative intensities we show the median absolute deviation which is better at characterizing the variation inside each structure.

The results can also be summarized as statistical outputs for comparing the phenotypes values for the groups of normal and abnormal tissue.

Machine Learning

The quantitatively meaningful data can then be used to train machine learning algorithms (decision trees to SVM) in order to automatically high-light many of the interesting regions and label them as such, dratistically reducing the required time to analyze a single patient.

Here we show a simple decision tree trained to identify lesions using color, position, texture and shape.

4Quant

Furthermore the ability to parallelize and scale means thousands to millions of videos can be analyzed at the same time to learn even more about the structures of the digestive track and identify new possibilities for diagnosis.

How?

The first question is how the data can be processed. The basic work is done by a simple workflow on top of our Spark Image Layer. This abstracts away the complexities of cloud computing and distributed analysis. You focus only on the core task of image processing.

The true value of such a scalable system is not in the single analysis, but in the ability to analyze hundreds, thousands, and even millions of samples at the same time.

4Quant

With cloud-integration and Big Data-based frameworks, even handling an entire city network with 100s of drones and cameras running continuously is an easy task without worrying about networks, topology, or fault-tolerance.

Technical Aspects

Processing the Data

Once the cluster has been comissioned and you have the SparkContext called sc (automatically provided in Databricks Cloud or Zeppelin), the data can be loaded using the Spark Image Layer. Since we are using real-time analysis, we acquire the images from an archive of images and create a database out of the results.

val iqaeDB = sc.createImageDatabase( "s3n://chest-ct/scans/*/*.avi", patientInfo="jdbc://oracle-db/PATIENTS") iqaeDB.registerImageTable("PatientImages")

Although we execute the command on one machine, the analysis will be distributed over the entire set of cluster resources available to sc. To further process the images, we can take advantage of the rich set of functionality built into Spark Image Layer.

The entire pipeline can then be started to run in real-time on all the new images as they stream in. If the tasks become more computationally intensive, then the computing power can be scaled up and down elastically.

Learn More

4Quant is active in a number of different areas from medicine to remote sensing. Our image processing framework (Spark Image Layer) and our query engine (Image Query and Analysis Engine) are widely adaptable to a number of different specific applications.

Check out our other use-cases to see how 4Quant can help you