Dense Trajectories and DHOG for Classification of Viewpoints from Echocardiogram Videos

Liqin Huang, Xiangyu Zhang, Wei LiComputational 2016 · DOI

TL;DR

This paper introduces a novel method for automatically classifying echocardiography videos based on their viewpoint, a crucial step for computer-aided diagnosis. By combining dense trajectory tracking with a new "difference HOG" (DHOG) feature descriptor, they achieve significantly improved classification accuracy, reaching 100% for primary viewpoints.

What Problem Does This Paper Address?

Echocardiography videos are vital for modern medical diagnosis, providing detailed anatomical and functional information about the heart. However, these videos are often captured from various standard views (eight in total), and accurately identifying these viewpoints is essential for efficient disease diagnosis and research. The process is currently manual, relying on sonographer expertise, which is time-consuming, prone to errors due to image noise, low resolution, and subjective experience. This paper aims to automate this classification process, making it more efficient and reliable.

Methods

The proposed method leverages two key components:

  • Dense Trajectory Tracking: Instead of relying on sparse feature points, this approach samples feature points densely across each frame of the echocardiography video. These points are then tracked using dense optical flow. To enhance robustness against rapid and irregular heart movements, they incorporate a trajectory description algorithm that uses optical flow derivatives to capture motion trajectory information. This method also helps differentiate foreground (heart) from background.
  • Difference Histograms of Oriented Gradients (DHOG): To describe the motion and structural information of the dense trajectories, the authors introduce DHOG. This descriptor builds upon the traditional Histograms of Oriented Gradients (HOG) by considering the differences between gradient directions, making it more robust to illumination changes and better at distinguishing objects with similar shapes but different motion patterns. They also incorporate Histograms of Optical Flow (HOF) and Motion Boundary Histograms (MBH) as complementary features.
  • Feature Encoding and Classification: To manage the high dimensionality of these features and avoid the "dimension disaster," they employ Fisher's vector encoding. This method, combined with a linear Support Vector Machine (SVM) classifier, is used for the final classification of the echocardiography videos.

Datasets

The study utilizes a dataset of 228 echocardiography videos collected from 72 patients (including normal individuals and those with cardiac wall motion abnormalities). These videos were provided by Tsinghua University First Hospital, with a resolution of 434x636 and a length of 1-2 seconds. The dataset comprises eight standard viewpoints. For evaluation, the data was split into a "half and half" train/test set.

Key Insights and Ideas

  • Dense Sampling is Key: Densely sampling feature points across frames, rather than relying on sparse features, ensures comprehensive coverage of the heart's motion and structure, leading to more robust tracking.
  • DHOG for Enhanced Discrimination: The novel DHOG descriptor proves crucial for improving classification accuracy by better capturing subtle differences in motion and structure, especially in the presence of noise and varying illumination common in echocardiograms.
  • Fisher Vectors for Dimensionality Reduction: Effectively reducing the high-dimensional feature space using Fisher vectors is essential for enabling efficient and accurate classification with a linear SVM.
  • Primary Viewpoints are Easier: The study highlights that classifying the three primary viewpoints (AA, PLA, PSA) is significantly easier and can achieve near-perfect accuracy, suggesting a potential pathway for initial screening or simplified diagnostic tools.

Key Results

  • The proposed method achieved an average classification accuracy of 77.12% for all eight viewpoints.
  • For the three primary viewpoints (AA, PLA, PSA), the method achieved an impressive 100% accuracy.
  • When the dataset was reduced to exclude less represented viewpoints (A5C, PSAB, PSAP), the classification accuracy for the remaining five viewpoints significantly improved to 98.36%.
  • The authors compare their method to several other algorithms, showing a notable improvement in accuracy, particularly for the three primary viewpoints.

Limitations and Future Work

  • Data Scarcity: The paper acknowledges that the lower accuracy for certain viewpoints (A5C, PSAB, PSAP) is primarily due to a lack of sufficient training data for these specific views.
  • Computational Complexity and Time: While the method shows improved accuracy, the authors note that balancing computational complexity and time cost remains a challenge for real-time applications.
  • Future Directions: The authors plan to gradually expand the database of echocardiography videos and explore newer, more effective methods to further improve classification accuracy and reduce processing time. They also aim to address the challenges posed by video noise and low resolution.

Summary updated

Abstract

In echo-cardiac clinical computer-aided diagnosis, an important step is to automatically classify echocardiography videos from different angles and different regions. We propose a kind of echocardiography video classification algorithm based on the dense trajectory and difference histograms of oriented gradients (DHOG). First, we use the dense grid method to describe feature characteristics in each frame of echocardiography sequence and then track these feature points by applying the dense optical flow. In order to overcome the influence of the rapid and irregular movement of echocardiography videos and get more robust tracking results, we also design a trajectory description algorithm which uses the derivative of the optical flow to obtain the motion trajectory information and associates the different characteristics (e.g., the trajectory shape, DHOG, HOF, and MBH) with embedded structural information of the spatiotemporal pyramid. To avoid "dimension disaster," we apply Fisher's vector to reduce the dimension of feature description followed by the SVM linear classifier to improve the final classification result. The average accuracy of echocardiography video classification is 77.12% for all eight viewpoints and 100% for three primary viewpoints.
BibTeX
@inproceedings{huang2016dense,
  author = {Liqin Huang and Xiangyu Zhang and Wei Li},
  title = {{Dense Trajectories and DHOG for Classification of Viewpoints from Echocardiogram Videos}},
  booktitle = {Computational},
  year = {2016},
  doi = {10.1155/2016/9610192},
}
Figure