Abstract
As database volumes expand, traditional query methods suffer from increased latency and reduced accuracy. This paper presents an optimized query method utilizing a hybrid approach of fuzzy correlation clustering and decision tree algorithms. By pre-sorting data into correlated clusters and navigating these clusters via an entropy-optimized decision tree, the proposed method maintains high accuracy even as data volume scales. Experimental results demonstrate a significant reduction in error coefficients and improved query efficiency compared to traditional algorithms.
Introduction
In the era of big data, database systems frequently encounter "data overflow," where the sheer volume and heterogeneity of information overwhelm standard retrieval mechanisms. Traditional query methods often require an excessive number of restrictions to maintain precision, leading to significant time consumption and a drop in accuracy. While standard tools like Support Vector Machines (SVM) and basic decision trees are available, they often lack the structural optimization required to handle the complex correlations inherent in massive datasets. This research proposes a dual-stage optimization strategy to address these bottlenecks.
Methodology
The proposed optimization framework consists of two primary stages: initial data organization through clustering and subsequent retrieval optimization through decision trees.
1. Fuzzy Correlation Clustering
To avoid the inefficiencies of searching through unorganized data, the system first applies fuzzy clustering. By defining a mathematical objective function, $L_p(Y, B)$, the algorithm calculates membership degrees and cluster centers. This iterative process identifies hidden correlations between data points, effectively "pre-sorting" the database into meaningful categories. This fuzzy approach allows data points to maintain relationships with multiple categories, providing the flexibility needed for complex queries.
2. Decision Tree Optimization
Once the data is clustered, the system constructs an optimized decision tree to navigate queries. The construction is guided by the Information Gain Ratio ($I_t(C_h)$). By selecting attributes with the highest information gain as branch nodes, the system ensures that the search space is narrowed down as rapidly as possible, minimizing the number of logical checks required to locate specific data.
Experimental Results and Analysis
The performance of the proposed method was evaluated against a traditional query algorithm using a simulated massive database. The primary metric for evaluation was the Error Coefficient, measured as the data size scaled from 100 to 500 units.
As shown in the experimental data, the traditional algorithm's error coefficient increases dramatically as the database grows, rising from approximately 0.1 to 0.8. In contrast, the proposed method exhibits remarkable stability. Its error coefficient remains consistently low, starting near 0.15 and only reaching approximately 0.25 at the highest data volume tested. This suggests that the hybrid clustering-decision tree approach is significantly more resilient to the challenges of data overflow.
Conclusion
The integration of fuzzy correlation clustering with optimized decision trees provides a robust solution for querying massive databases. By leveraging the mathematical strengths of entropy-based branching and the flexibility of fuzzy logic, the proposed method achieves higher accuracy and better scalability than traditional approaches. Future research may focus on the computational overhead of the initial clustering phase and the adaptation of this model for real-time streaming data environments.
Summary updated