A Data Mining Algorithm for Association Rules with Chronic Disease Constraints

Yan‐rong Liu, Lijun Wang, Rong Miao, HengNi RenComputational 2022 · DOI

TL;DR

This paper introduces an optimized data mining algorithm for discovering association rules in chronic disease data. By combining a clustering matrix for data compression and pruning strategies to eliminate redundant rules, the proposed method significantly reduces computational load and improves efficiency compared to the traditional Apriori algorithm.

What Problem Does This Paper Address?

The paper tackles the challenges of applying association rule mining to chronic disease data, particularly within the context of medical fields. The traditional Apriori algorithm, while widely used, suffers from inefficiencies such as repeatedly scanning large transaction databases, leading to the generation of a massive number of redundant rules. This not only impacts performance but also makes it difficult to extract truly useful insights for diagnosis and prevention. The authors aim to address these issues by proposing an enhanced algorithm that is more efficient and generates fewer, more relevant association rules.

Methods

The core of the proposed solution lies in enhancing the traditional Apriori algorithm with two key innovations:

  • Clustering Matrix Strategy: Instead of directly processing the raw transaction database, the algorithm first compresses it into a clustering matrix. This matrix represents the data in a more compact form, where columns represent items and rows represent transactions. This compression aims to reduce the size of the data that needs to be scanned.
  • Pruning Strategy (Pre- and Post-pruning): The algorithm incorporates both pre- and post-pruning techniques.
  • Pre-pruning: This strategy aims to identify and eliminate infrequent item sets before they are used to generate candidate item sets. This is based on the property that if a subset of an item set is infrequent, the entire item set must also be infrequent.
  • Post-pruning: This strategy is applied after candidate item sets are generated, further filtering out those that do not meet certain criteria.
  • Constraint Conditions: The algorithm also allows for the incorporation of specific constraints (e.g., related to chronic diseases like hypertension or diabetes) to guide the mining process and ensure the generated rules are relevant to the domain.

Datasets

The experiments were conducted using the Mushroom dataset from the UCI standard test dataset. This dataset contains 8,124 records, each with 23 attributes, and each attribute has 12 enumeration values.

Key Insights and Ideas

  • Data Compression is Key: The clustering matrix approach effectively compresses the transaction database, reducing the need for extensive scanning and thereby lowering I/O load.
  • Pruning is Crucial for Efficiency: The combination of pre- and post-pruning strategies is highly effective in drastically reducing the number of candidate item sets generated. This is a significant bottleneck in the traditional Apriori algorithm.
  • Domain Constraints Enhance Relevance: Incorporating specific constraints related to chronic diseases helps in generating association rules that are more meaningful and directly applicable to medical diagnosis and prevention.
  • Efficiency Gains are Substantial: The proposed algorithm demonstrates significant improvements in both the number of candidate item sets generated and the overall execution time, making it more practical for large-scale medical data analysis.

Key Results

The experimental results, visualized in Figures 2, 3, and 4, highlight the advantages of the proposed optimized algorithm:

  • Reduced Candidate Item Sets: Across various support thresholds, the optimized algorithm consistently generated fewer candidate item sets compared to the traditional Apriori algorithm. For instance, at a min_supp of 30, the optimized algorithm produced 256 candidate sets, while the traditional Apriori produced 385.
  • Faster Execution Time: The optimized algorithm showed a significant reduction in execution time. For example, with a min_supp of 10, the execution time was 112 units for the optimized algorithm versus 175 units for the traditional one. Similarly, with 8000 datasets, the optimized algorithm took 55 units of time, compared to 81 units for the traditional Apriori.
  • Improved Efficiency: The paper concludes that the optimized algorithm greatly reduces database scans and the number of candidate item sets, leading to a substantial improvement in overall running efficiency.

Limitations and Future Work

  • Inconvenient Rule Presentation: The authors acknowledge that the association rules are currently presented in a formal formula, which may be inconvenient for users to understand.
  • Future Work: The authors suggest that in future work, they will focus on the visualization of the data mining process. This would make the generated rules and the mining process more accessible and interpretable for users.

Summary updated

Abstract

The Apriori algorithm in association rules is the main algorithm used in the treatment and prevention of chronic diseases in data mining, and the algorithm in the current stage of China's medical field of association between chronic diseases has some problems, such as the need to scan the transaction database of cases several times, producing a large data set and more redundant rules. To address the above problems, a data mining algorithm of association rules combining clustering matrix and pruning strategy is proposed, which improves the algorithm by using the clustering matrix method to compress the stored transaction database and introducing the prepruning and postpruning strategy methods on the basis of adding constraint conditions. The experimental results show that the optimization algorithm has unique advantages in reducing the number of database scans and the number of candidate item sets generated and ultimately greatly reduces the running time and I/O load of the algorithm, and the running efficiency of the algorithm is greatly improved.
BibTeX
@inproceedings{liu2022data,
  author = {Yan‐rong Liu and Lijun Wang and Rong Miao and HengNi Ren},
  title = {{A Data Mining Algorithm for Association Rules with Chronic Disease Constraints}},
  booktitle = {Computational},
  year = {2022},
  doi = {10.1155/2022/8526256},
}
Figure