TL;DR
This paper investigates using the Pearson correlation coefficient as a fitness function for symbolic regression, a task where the goal is to find mathematical equations that fit data. They found that correlation, when combined with a simple linear regression alignment step, significantly outperforms the traditional Root Mean Squared Error (RMSE) fitness function, leading to faster discovery of correct equations with fewer data points and improved robustness to noise.
What Problem Does This Paper Address?
Symbolic regression (SR) is a challenging problem in machine learning and genetic programming (GP) where the aim is to discover mathematical expressions that model a given dataset. Traditionally, the Root Mean Squared Error (RMSE) has been the go-to fitness function, measuring the average difference between predicted and actual values. However, RMSE can be sensitive to the scale and offset of the data, potentially hindering the discovery of the correct underlying functional form, especially in the presence of noise or limited data. This paper explores an alternative fitness function, the Pearson correlation coefficient, to see if it can overcome these limitations and improve the efficiency and accuracy of symbolic regression.
Methods
The core of this research involves comparing two fitness functions for symbolic regression:
- RMSE (Root Mean Squared Error): The standard approach, calculated as the square root of the average squared difference between the target output ($y_i$) and the program's predicted output ($\hat{y}_i$).
- Correlation (with alignment): This novel approach uses the Pearson correlation coefficient ($R$) as the primary fitness metric. Crucially, it's combined with a post-processing step: a simple linear regression to align the discovered model's output to the target data. This alignment minimizes the absolute difference between the target and the scaled/shifted model output. The paper emphasizes that correlation focuses on the relative positions of data points, making it less sensitive to absolute scaling and offsets compared to RMSE.
The experiments were conducted using a stack-based genetic programming system called StackGP. They systematically varied parameters like the number of data points, noise levels, and dimensionality of the input to assess the performance of both fitness functions.
Datasets
The paper evaluates the proposed correlation-based fitness function on several benchmark datasets:
- Koza's Benchmarks: Classic problems used in early GP research, including polynomial functions.
- Keijzer Benchmarks: A set of more challenging problems, including Keijzer-5.
- Vladislavleva Benchmarks: Including Vladislavleva-1.
- Nguyen Benchmarks: Including Nguyen-5.
- Feynman Symbolic Regression Benchmark: A collection of physics-inspired equations and their corresponding data, notably used for evaluating performance with limited data and noise.
No new benchmarks were introduced, but the Feynman set was a particular focus due to its challenging nature and the authors' findings regarding its suitability for testing minimal data requirements.
Key Insights and Ideas
- Correlation Captures Shape, Alignment Handles Scale: The key insight is that the correlation coefficient excels at identifying the correct functional form or shape of the relationship, irrespective of its absolute scale or offset. The subsequent linear regression step then efficiently handles the scaling and shifting, making the overall process more robust.
- Fewer Data Points Needed: The correlation-based approach demonstrated a remarkable ability to find correct equations with significantly fewer data points than RMSE. In some cases, as few as 3 data points were sufficient.
- Faster Convergence: Solutions were found in fewer generations when using correlation as the fitness function, indicating a more efficient search process.
- Robustness to Noise: While correlation's performance does degrade with increasing noise, it generally maintains better performance than RMSE, especially when sufficient data is available.
- "Trivial" Problems Become Even Easier: For problems that are already simple for RMSE, correlation often solves them even faster, highlighting its efficiency.
Key Results
- Significant Performance Gains: Across various benchmarks, the correlation-based fitness function consistently outperformed RMSE, especially when dealing with limited data or noisy conditions.
- Feynman Benchmark Success: On the Feynman benchmark, with just 3 data points and no noise, correlation found better models in 38 out of 100 cases and perfectly solved 21 problems, whereas RMSE struggled with these minimal conditions.
- Noise Tolerance: For the Keijzer-5 benchmark with 2,000 points, correlation generally found more accurate models even with up to 20% noise. However, for the Korns-8 benchmark with 200 points, correlation's advantage diminished beyond 12% noise. With 2,000 points on Korns-8, more data helped correlation overcome noise.
- Sensitivity to Constants: While correlation's ability to find the correct functional form was largely unaffected by constants, the alignment step could lead to off-scale or off-position solutions. However, even with multiplicative noise, correlation still significantly outperformed RMSE in these scenarios.
Limitations and Future Work
- Sensitivity to Noise (with limited data): While correlation is generally robust to noise, its performance can degrade significantly when combined with a very small number of data points.
- Alignment Step: The paper acknowledges that the linear regression alignment step, while effective, might not be optimal for all scenarios and could be further explored.
- GP System Specificity: The authors suggest that the observed success might be partly due to the specific GP system used (StackGP) and that further investigation is needed to see if these results generalize to other GP implementations.
- Transferability: The authors speculate that the lessons learned from using correlation in symbolic regression might be transferable to other tasks like classification and could facilitate solving harder problems in different domains.
The paper concludes that the fitness function plays a crucial role in evolutionary algorithms and that replacing RMSE with a correlation-based fitness function, especially with an alignment step, offers a powerful and efficient approach for symbolic regression.
Summary updated