Patch-Driven Tongue Image Segmentation Using Sparse Representation

Weixia Liu, Changen Zhou, Zuoyong Li, Zhongyi HuIEEE Access 2020 · OpenReview

Abstract

Tongue diagnosis plays a key role in Traditional Chinese Medicine (TCM). Accurate tongue image segmentation is essential for quantitative analysis, yet it remains challenging due to significant personal variations in color, texture, and shape. Furthermore, the similar color profiles of the tongue and surrounding tissues, such as the lips, often result in "weak edges" that baffle standard segmentation algorithms. This paper proposes a patch-driven segmentation method utilizing sparse representation. By representing testing image patches as sparse combinations of training patches from a spatially varying dictionary, the system estimates tongue probability. Final segmentation is achieved via the maximum a posteriori (MAP) rule and morphological polishing. Evaluated on a dataset of 290 subjects, the method demonstrates superior accuracy and stability compared to state-of-the-art techniques.

Introduction

In Traditional Chinese Medicine, the tongue serves as a vital diagnostic window, with its color, texture, and coating providing critical cues for a patient's health. Automating this diagnostic process requires precise tongue image segmentation—the isolation of the tongue body from the lips, teeth, and throat.

This task is notoriously difficult for computer vision. Tongues vary wildly in morphology between individuals. More importantly, the color of the tongue often blends into the surrounding lips, creating "weak edges" where standard gradient-based algorithms fail. Existing methods frequently rely on single color channels or simple geometric priors, leading to "leaky" segmentations or total failure on subjects with thick coatings or strong textures.

Methodology

The proposed method shifts away from global edge detection toward a local, patch-based logic. The pipeline consists of four primary stages designed to handle the complexities of tongue morphology and color similarity.

The high-level pipeline of the proposed patch-driven segmentation method, moving from input images to final segmentation.
  1. Color-Aware Dictionary Construction: For a given pixel in a test image, the algorithm examines a small surrounding patch. It searches training images for similar patches using a similarity measure that extends the Structural Similarity Index (SSIM) into the RGB color space. This ensures the most relevant training data is pre-selected.
  2. Sparse Representation via Elastic-Net: The test patch is represented as a linear combination of pre-selected training patches. The authors employ a non-negative Elastic-Net approach, combining L1 regularization (for sparsity) and L2 regularization (for smoothness/grouping). This allows the model to ignore irrelevant noise and focus on "expert" patches.
  3. Probability Mapping: By analyzing the manual labels of the training patches used in the sparse representation, the system calculates the probability that the center pixel of the test patch belongs to the tongue.
  4. Morphological Polishing: After generating a raw probability map, a Maximum A Posteriori (MAP) rule creates a binary mask. Standard morphological operations (opening and closing) then fill small holes and remove isolated noise pixels.

Detailed Framework

The core of the innovation lies in how the dictionary is constructed and utilized. Unlike global models, this spatially varying dictionary adapts to the specific local characteristics of the testing patch.

Detailed framework of the patch-driven sparse representation, illustrating the selection of training patches to form a localized dictionary (Dx) for a specific testing patch (Sx).

By focusing on local patches, the method is robust against global shape variations and facial features like wrinkles that often distract traditional Active Contour Models (ACM). The use of full RGB color information provides the stability needed to distinguish the subtle nuances between the "red" of a tongue and the "red" of a lip.

Experimental Results

The study utilized a dataset of 290 subjects for primary evaluation using 10-fold cross-validation, plus 10 additional "unseen" subjects to test generalization. The proposed method was compared against GVF-RM (a gradient-based method) and nonlocal-means (a standard patch-based label fusion method).

  • Accuracy: The proposed method achieved a Kappa Index (KI) of 0.972, significantly outperforming GVF-RM (0.806) and nonlocal-means (0.899).
  • Error Rates: It recorded the lowest Misclassification Error (ME) at 0.012, compared to 0.091 for GVF-RM.
  • Robustness: Visual inspections confirmed that while other methods often "leaked" into the lips or missed the tongue tip, the sparse representation method maintained tight, accurate boundaries across diverse tongue types.

Conclusion and Future Work

While highly accurate, the primary drawback of this approach is its computational cost. Processing one image took approximately 5 minutes on a standard server due to the per-pixel sparse representation requirements. However, the authors demonstrated that parallel computing can reduce this runtime to roughly 20 seconds. Future research will focus on optimizing dictionary construction and exploring deep learning architectures that can replicate this patch-based logic at higher speeds.

Summary updated

Abstract

Tongue diagnosis plays a key role in TCM (Traditional Chinese Medicine) diagnosis. Tongue image segmentation lays a solid foundation for quantitative tongue analysis and diagnosis. However, the segmentation of tongue body is challenging due to the factors such as large personal variation of tongue body on color, texture and shape, as well as weak edges caused by similar color between tongue body and neighboring tissues, especially the lip. Existing segmentation methods usually use only single color component and simple prior knowledge, thus leading to inaccuracy and instability. To alleviate these issues, a patch-driven segmentation method with sparse representation is proposed in this paper. Specifically, each patch in the testing image is sparsely represented by patches in the spatially varying dictionary, which is constructed by the local patches of training images. The derived sparse coefficients are then employed to estimate the tongue probability. Finally, the hard segmentation is obtained by applying the maximum a posteriori (MAP) rule on the tongue probability map and further polished with morphological operations. The proposed method has been extensively evaluated on a tongue image dataset including 290 subjects using 10-fold cross-validation, as well as additional 10 unseen testing subjects. The proposed method has achieved more accurate segmentation results, compared with the state-of-the-art methods.
BibTeX
@inproceedings{liu2020patch,
  author = {Weixia Liu and Changen Zhou and Zuoyong Li and Zhongyi Hu},
  title = {{Patch-Driven Tongue Image Segmentation Using Sparse Representation}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=PKPoM79aIS},
}
Figure