TOMAAT: volumetric medical image analysis as a cloud service

Fausto Milletari, Johann Frei, Seyed-Ahmad AhmadiarXiv 2018 · arXiv

TL;DR

TOMAAT is an open-source, cloud-based framework designed to make advanced 3D medical image segmentation models easily accessible without requiring high-end local hardware. By decoupling heavy deep learning computation from the user interface via a lightweight client-server architecture, it enables clinicians and researchers to run state-of-the-art volumetric analysis directly within standard tools like 3D Slicer.

What Problem Does This Paper Address?

Deep learning has revolutionized medical image analysis, particularly for tasks like 3D segmentation in MRI, CT, and ultrasound scans. However, transitioning these models from a research paper to practical clinical use remains a major bottleneck.

Currently, when researchers publish their code and trained models, they rarely follow a common standard. Setting up these models locally often requires complex software dependencies and expensive, high-performance GPU hardware that typical clinical workstations lack. While existing solutions like DeepInfer package models into Docker containers to simplify installation, they still execute the code locally, leaving the hardware barrier intact. TOMAAT addresses this gap by moving the heavy computational lifting entirely to the cloud.

Methods

TOMAAT introduces a decentralized, cloud-based architecture split into three core components:

  1. Server (Prediction Endpoint): A remote machine equipped with adequate GPU hardware that wraps a trained deep learning model alongside its specific pre- and post-processing data pipelines. Powered by the Python micro-framework Klein, the server exposes a customizable HTTP 1.1 interface capable of handling concurrent client requests.
  2. Announcement Service (Service Registry): A centralized registry where public servers can securely announce their availability using a unique API key. It maintains a live directory detailing each server's URL, supported imaging modalities, anatomical targets, and specific tasks.
  3. Client: A lightweight frontend interface. The authors implemented a primary example as an extension for 3D Slicer, a widely used medical image visualization platform. The client communicates with the servers via standard HTTP GET and POST requests.
Fig. 1. Schematic representation of the architecture of the TOMAAT framework.

Datasets

Not discussed in the paper.

(Note: The paper focuses on evaluating the framework's operational latency across three pre-trained model use cases—heart ultrasound, prostate MRI, and brain tumor MRI—but does not explicitly detail the original training datasets or introduce new imaging benchmarks).

Key Insights and Ideas

  • Dynamic UI Generation: Instead of hardcoding client interfaces for every new model, the TOMAAT client queries the server's endpoint to discover its required inputs (e.g., number of volumes, threshold sliders). The client then dynamically builds the appropriate user interface widgets inside 3D Slicer on the fly.
  • Decoupling Compute from Visualization: By offloading 3D convolutional neural networks (CNNs) to cloud servers, any standard computer capable of running 3D Slicer can leverage state-of-the-art AI without local hardware upgrades.
  • API-Key Secured Registry: The announcement service acts as a curated directory, ensuring only trusted developers can publish endpoints, reducing the risk of broken or malicious third-party implementations.
Fig. 2. 3D slicer module allowing interaction with TOMAAT announcement service and prediction endpoints.

Key Results

The authors evaluated TOMAAT's network latency across three distinct segmentation tasks under three network conditions: Local Area Network (LAN), DSL, and a 4G mobile connection.

  • 3D V-Net Heart (Ultrasound): Operating at a high resolution of $256^3$ cubic voxels, it achieved a total latency of 3.93 seconds on LAN (with 1.25s of net GPU compute time) and 8.70 seconds over a 4G connection.
  • 3D V-Net Prostate (MRI): Processing a smaller volume of $256 \times 256 \times 64$ voxels, it showed a LAN latency of 3.40 seconds (0.38s GPU compute) and 13.53 seconds over 4G.
  • Brain Tumor Segmentation: This heavy pipeline required transferring four distinct MRI volumes (Flair, T1, T1c, and T2) to be processed slice-by-slice via multiple 2D networks. Due to the large data transfer overhead, latency ranged from 20.12 seconds on LAN up to 82.49 seconds over 4G.
Table 1. TOMAAT latency for different scenarios.

Limitations and Future Work

  • GPU Locking Bottleneck: While the server uses Python's Klein framework to handle multiple clients via multi-threading, these threads ultimately compete for the same physical GPU through a locking mechanism. This could limit scalability under heavy concurrent clinical loads.
  • Data Transfer Overhead: As highlighted by the brain tumor use case, transferring multiple raw 3D medical volumes over slower connections (like 4G or DSL) introduces significant network latency. Future iterations will need to explore optimized data compression techniques or hybrid edge-compute architectures to mitigate bandwidth bottlenecks.

Summary updated

Abstract

Deep learning has been recently applied to a multitude of computer vision and medical image analysis problems. Although recent research efforts have improved the state of the art, most of the methods cannot be easily accessed, compared or used by either researchers or the general public. Researchers often publish their code and trained models on the internet, but this does not always enable these approaches to be easily used or integrated in stand-alone applications and existing workflows. In this paper we propose a framework which allows easy deployment and access of deep learning methods for segmentation through a cloud-based architecture. Our approach comprises three parts: a server, which wraps trained deep learning models and their pre- and post-processing data pipelines and makes them available on the cloud; a client which interfaces with the server to obtain predictions on user data; a service registry that informs clients about available prediction endpoints that are available in the cloud. These three parts constitute the open-source TOMAAT framework.
BibTeX
@misc{milletari2018tomaat,
  author = {Fausto Milletari and Johann Frei and Seyed-Ahmad Ahmadi},
  title = {{TOMAAT: volumetric medical image analysis as a cloud service}},
  year = {2018},
  eprint = {1803.06784},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.06784},
}

Loading related content...

Figure