Consider two images: a perfect checkerboard of alternating black and white pixels, and an image where the left half is entirely black and the right half is entirely white.
If you count the pixels, both images contain exactly 50% black and 50% white pixels. Consequently, any global statistical measure—such as the mean intensity, variance, skewness, or kurtosis—will evaluate these two images as identical. These are first-order statistics; they describe the distribution of pixel values across a region but completely discard their spatial arrangement.
To a human observer, or a machine learning model tasked with diagnosing a tumor or classifying a satellite image, these two patterns represent entirely different physical realities. To capture these differences mathematically, we must transition from first-order statistics to second-order and higher-order spatial texture analysis. This methodology extracts sub-visual spatial relationships between pixels to quantify structural complexity, roughness, and tissue heterogeneity.
The Core Mechanism: Mapping Spatial Relationships
The foundational tool for capturing second-order spatial properties is the Gray-Level Co-occurrence Matrix (GLCM). Instead of asking "how many pixels have an intensity of $X$?", a GLCM asks: "how often does a pixel of intensity $i$ occur at a specific distance and orientation from a pixel of intensity $j$?"
As explained by Dieckmeyer et al., first-order metrics describe global heterogeneity without considering pixel placement, whereas second-order spatial pattern analysis evaluates the localized relationships between adjacent pixels. By defining a displacement vector (a distance $d$ and an angle $\theta$), we construct a square matrix where the dimensions equal the number of gray levels in the image. Each cell $(i, j)$ accumulates the frequency of that specific pixel pair configuration across the region of interest, as detailed by Chaddad et al..
Once the GLCM is constructed, it serves as a statistical footprint from which several highly descriptive, interpretable features (often called Haralick features) can be derived:
- Contrast: Measures the local variations in the matrix. In our checkerboard example, because a black pixel ($i=0$) is always adjacent to a white pixel ($j=255$), the off-diagonal entries of the matrix are heavily populated, yielding high contrast.
- Homogeneity: Measures the closeness of the distribution of elements in the GLCM to the GLCM diagonal. The split-screen image would yield high homogeneity because most neighboring pixels share identical values.
- Entropy: Quantifies the randomness or disorder of the spatial distribution. High entropy indicates a complex, non-repetitive texture, which Shan et al. and Huang et al. leverage as a quantitative biomarker for structural complexity.
By calculating these features across multiple orientations (e.g., $0^\circ, 45^\circ, 90^\circ, 135^\circ$), the descriptors remain robust to directional bias, capturing complex spatial relationships while maintaining clinical and physical transparency (Dia et al.).
Higher-Order Matrices and Alternative Descriptors
While the GLCM tracks pixel pairs, higher-order texture matrices track larger structural groupings to capture macro-level "patchiness" or structural regularity:
- Gray Level Run Length Matrices (GLRLM): Quantify the number of consecutive pixels sharing the same intensity value along a specific linear direction. This highlights whether a texture is coarse (long runs of identical intensities) or fine (short, frequent interruptions).
- Gray Level Size Zone Matrices (GLSZM): Count the number of multi-directional, connected groups of pixels that share the same gray level. As demonstrated by Liang et al. and Sollmann et al., combining GLRLM and GLSZM features allows models to quantify internal tumor or tissue degradation before significant changes in average density occur.
- Neighborhood Gray Tone Difference Matrices (NGTDM): Focus on the difference between a pixel's value and the average value of its neighbors, aligning closely with human visual perceptions of texture properties like coarseness, contrast, and busyness (Jia et al.).
Stepping outside traditional intensity matrices, researchers have also adapted concepts from other disciplines to describe texture. For example, Ataky introduced a bio-inspired approach that treats an image like an ecosystem. By mapping pixel intensities to biological "species," the author applies ecological metrics like species richness and taxonomic distinctness to build highly explainable, computationally efficient texture descriptors.
Integrating Texture into Deep Learning
Standard Convolutional Neural Networks (CNNs) excel at learning hierarchical features, but their deep, context-focused layers often discard fine-grained, stochastic texture details in favor of global semantic shapes. To bridge this gap, modern architectures integrate statistical texture constraints directly into deep learning pipelines.
Learnable Co-occurrence and Histogram Layers
Instead of treating texture extraction as a static preprocessing step, Ji et al. developed a framework for learnable spatial co-occurrence modeling. This acts as a differentiable version of a GLCM, allowing a neural network to retain and optimize fine-grained local dependencies during backpropagation. Similarly, Peeples et al. introduced dedicated histogram layers into UNet architectures, forcing the network to explicitly model the statistical distribution of feature maps to better segment textures that standard convolutional filters overlook.
Differentiable Spatial Entropy
In generative modeling and image enhancement, optimizing solely for pixel-wise distance (like L1 or L2 loss) frequently results in blurry, "texture-less" outputs. To combat this, Lian et al. formulated a differentiable spatial entropy loss. By modeling the joint distribution of a pixel and its surrounding neighborhood in a mathematically differentiable way, generative models can be penalized for losing structural complexity, forcing them to preserve natural tactile textures.
Cross-Domain Applications
Because spatial texture analysis provides a mathematical proxy for physical surface properties and internal material complexity, it has become a cornerstone technique across multiple fields.
Radiomics and Medical Imaging
In clinical oncology, tumors that look identical in size and average brightness can have vastly different cellular architectures, vascular patterns, and genetic profiles. Radiomics uses spatial texture matrices to expose this hidden heterogeneity.
Cai et al. and Jiang et al. demonstrate that extracting GLCM features from multi-modal MRIs captures sub-visual spatial patterns that reliably differentiate malignant breast masses from benign tissue. Beyond oncology, Burian et al. apply GLCM algorithms to evaluate the spatial distribution of vertebral bone marrow fat, uncovering structural micro-patterns that signal skeletal degradation long before global bone mineral density drops.
Remote Sensing and Canopy Analysis
In satellite and UAV imaging, raw spectral reflectance (color bands) can saturate or fail when observing dense, complex environments. Texture analysis adds a geometric dimension to the data.
When estimating crop biomass, Wang et al. and Zhai et al. combine spectral indices with GLCM features extracted from individual bands. The texture features capture the physical roughness and structural patterns of the plant canopy, significantly improving prediction stability. Similarly, Mikeli et al. utilize GLCM and Local Binary Patterns to separate marine plastic debris from "spectral twins" like natural organic matter or ships, relying on the distinct surface roughness of man-made materials to achieve accurate classification.
Synthetic Image Verification
Texture analysis also serves as a powerful diagnostic tool for evaluating generative models. Liu demonstrates that analyzing the contrast and correlation of pixel values through spatial co-occurrence matrices exposes fundamental differences between real and synthetic images. While generative models can easily match global first-order color distributions, they struggle to replicate the complex, high-contrast micro-patterns inherent in real-world textures, leaving a statistical signature that makes fake face detection highly robust against compression and distortion.