Deep learning models excel at discovering high-dimensional, abstract visual patterns that elude human observers. However, when applied to medical imaging or specialized signal processing, these "black boxes" often struggle with small datasets, lack interpretability, and can overlook explicit geometric or statistical truths. Conversely, traditional radiomics and expert-engineered workflows rely on deterministic, handcrafted mathematical formulas to quantify explicit features like tumor shape, volume, texture, or signal variability.
Rather than treating these two paradigms as mutually exclusive, a powerful research direction focuses on hybrid feature fusion. By integrating handcrafted clinical features with deep-learning-derived representations, researchers are building diagnostic models that are more accurate, stable, and clinically interpretable than either approach alone.
The Complementary Nature of Features
To understand why hybrid fusion works, we must look at what each feature type brings to the table.
- Handcrafted Radiomic/Expert Features: These are predefined mathematical descriptors of macro-level morphology, intensity distributions, and spatial relationships (e.g., sphericity, Gray-Level Co-occurrence Matrices, or heart rate variability metrics). They provide a stable, human-interpretable baseline rooted in domain knowledge.
- Deep Learning Features: These are latent representations automatically learned by convolutional neural networks (CNNs) or transformers. They capture highly complex, non-linear, and sub-visual textures across multiple levels of abstraction.
When a model relies solely on raw pixels, it must learn fundamental geometric concepts from scratch—a daunting task when sample sizes are small. As Yuan Xie et al. demonstrate in predicting intracranial aneurysm rupture risk, combining a 3D ResNet's pattern matching with traditional geometric radiomics allows the network to respect physical risk factors like sphericity while simultaneously evaluating subtle wall textures. This dual-perspective analysis captures both global anatomical context and local signal variations, a synergy highlighted across diverse domains from stroke lesion aging by Adam Marcus et al. to lung nodule invasiveness by Du Hai et al..
Fusion Architectures: How the Worlds Collide
Fusing these distinct feature sets requires careful architectural design. The literature generally splits these methodologies into three structural paradigms: late fusion, dual-path extraction, and intermediate/deep radiomic integration.
Late Fusion:
[Image] ----> [Deep Network] ------> [Deep Features] -------\
+--> [Joint Classifier] ---> Prediction
[Image] ----> [Radiomic Math] -----> [Handcrafted Feat] ----/
Dual-Path / Intermediate Fusion:
[Image] ----> [Deep Layers] --------> [Latent Maps]
^ |
| (Spatial Injection) v
[Image] ----> [Radiomic Patches] ---> [Radiomic Math on Maps] --> Prediction
1. Late Fusion and Decision-Level Integration
The most straightforward approach extracts deep features and handcrafted features independently, concatenating them before passing them to a final classifier or fully connected layer. For instance, Tahsin Kurç et al. utilize late fusion to combine quantitative MRI shape metrics with histology-derived deep features for glioma research.
Similarly, Kangneng Zhou et al. feed both automated deep signatures and manually engineered characteristics into a fully connected layer to evaluate gastric cancer tumor microenvironments. To maximize stability, Hongjian Zhang et al. paired this joint representation with a robust, low-complexity Logistic Regression classifier, outperforming purely deep architectures in brain tumor genotyping.
2. Dual-Path Feature Extraction
Instead of treating fusion as an afterthought, dual-path networks process images through parallel pipelines optimized for different feature scales. Xiaofeng Zhang et al. design a dual-path framework for thyroid carcinoma ultrasound analysis, ensuring that global structural textures and subtle pixel-level patterns are isolated before being unified. This concept is extended to spatial subregions by Jiaxin Shi et al., who partition tumors into inner and marginal zones, running hybrid deep-radiomic extractions on each subregion to capture localized heterogeneity.
3. Intermediate and Deep Radiomic Integration
A more tightly coupled strategy injects handcrafted domain knowledge directly into the hidden layers of a neural network. In their Radiomics-Informed Deep Learning (RIDL) framework, Weihang Dai et al. calculate local radiomic texture descriptors for patches around each voxel and feed them directly into the early layers of a 3D CNN. This provides spatially aligned domain knowledge, drastically reducing the network's learning burden.
Alternatively, Ahmad Chaddad et al. apply traditional statistical quantifier functions (like Gray-Level Co-occurrence Matrices) directly to the activation maps of a deep network rather than raw pixels, capturing how the model itself perceives structural complexity at multiple levels of abstraction.
Why It Works: The Intuition
The consistent success of hybrid feature fusion across tasks—ranging from cardiac signal analysis by Shenda Hong et al. to species distribution modeling by Titouan Lorieul et al.—stems from three core machine learning mechanics.
Mitigating Small-Sample Overfitting
Medical imaging datasets are notoriously small due to privacy and annotation costs. Purely data-driven deep networks have millions of parameters and easily overfit to spurious correlations in small cohorts. Handcrafted radiomics act as a powerful inductive bias. Because these features are mathematically fixed (e.g., calculating a volume or an intensity gradient), they do not require training data to emerge. Injecting them anchors the model, preventing the deep learning components from drifting during optimization.
Multi-Scale Representation and Biological Anchors
Pathology operates on multiple scales, from macroscopic tumor geometry to microscopic cellular arrangements. Fusing handcrafted metrics with deep learning creates a multi-scale representation. For example, in predicting spatial gene expression from H&E slides, Hang Shi integrates handcrafted nuclear morphology features with deep representations. The explicit nuclear shapes provide a "biological anchor" that ensures the model tracks known structural tissue characteristics, while the deep network picks up on subtle, sub-visual background textures.
Balancing Performance with Clinical Trust
Pure deep learning models face resistance in clinical settings due to their uninterpretable nature. Fusing these models with clinical metadata and explicit radiomics bridges the gap between performance and transparency. As reviewed by Mingyang Li et al., hybrid decision support systems balance high-dimensional representation power with structured, human-verifiable data.
This transparency is vital; Alexandre Bône et al. showed that forcing a model to predict standardized clinical criteria alongside radiomic features boosted performance by up to 18 AUC points over a standard black-box model. By pairing abstract neural outputs with deterministic clinical risk factors—such as patient age in glioma prediction (Ryohei Fukuma et al.) or smoking history in lung cancer screening (Weicheng Huang et al.)—hybrid fusion delivers the predictive power of modern AI without sacrificing the mathematical and clinical baselines that practitioners trust.
Key Foundational Work
Prediction of IDH and TERT promoter mutations in low-grade glioma from magnetic resonance images using a convolutional neural network (2019) — contributed 1 concept elements to this direction.
Differentiation of Recurrence from Radiation Necrosis in Gliomas Based on the Radiomics of Combinational Features and Multimodality MRI Images (2019) — contributed 1 concept elements to this direction.
ENCASE: an ENsemble ClASsifiEr for ECG Classification Using Expert Features and Deep Neural Networks (2017) — contributed 1 concept elements to this direction.