Introduction
Inductive learning—the process by which a system generates general rules from specific examples—is a cornerstone of artificial intelligence. However, these systems frequently encounter a "combinatorial explosion" when attempting to refine their models. When a prediction fails, the system must identify which specific feature or combination of features was responsible for the error. In real-world environments with dozens of variables, the number of possible hypotheses grows exponentially, making exhaustive search impossible.
This paper introduces HANDICAPPER, a system designed to navigate this search space using a dual-layered approach: psychologically-inspired heuristics to generate hypotheses and a "rationalization" process to validate them against a causal model. By applying this architecture to the complex domain of thoroughbred horse racing, the research demonstrates how constrained search can lead to predictive accuracy that significantly surpasses human expertise.
The Problem of Credit Assignment
The primary challenge in inductive learning is the credit assignment problem. If a system predicts a horse will win and it loses, which of the 50+ features of that race caused the failure? Was it the track condition, the jockey's experience, the horse's recent rest period, or a combination of all three? With just 10 features, there are over 1,000 possible combinations to consider. In a race with 50 features, the search space becomes astronomical. To be effective, a learner must have a method for deciding which features are likely to be relevant without checking every permutation.
The HANDICAPPER Architecture
HANDICAPPER addresses the combinatorial explosion through a two-step pipeline that mimics human cognitive biases.
1. Heuristic Search
Instead of an exhaustive search, HANDICAPPER employs nine specific heuristics to prune the hypothesis space. These heuristics are grounded in human psychology and suggest where to "assign blame" for a failed prediction:
- Unusualness: If a failure occurs, the system prioritizes features that are statistically rare (e.g., a horse running with a brand-new jockey).
- Uncertainty: The system assigns blame to features it has encountered the least, assuming its knowledge of familiar features is more likely to be correct.
- Conservatism: This heuristic prefers the smallest possible modification to the existing rule set, maintaining stability in the model.
- Proximity: The system looks for events or features that occurred closest in time or space to the failure.
2. Rationalization
To prevent the system from forming "superstitious" rules—correlations that are purely coincidental—HANDICAPPER uses a rationalization module. Once a heuristic suggests a hypothesis (e.g., "The horse lost because the track was muddy"), the system attempts to "explain" this hypothesis using a pre-existing, hand-fed causal network.
Using forward chaining, the rationalization module checks if the suggested feature logically leads to a "likely to lose" conclusion within its causal model. If the hypothesis cannot be rationalized, it is discarded, and the heuristics are prompted to suggest an alternative. This feedback loop ensures that the system's learning is both efficient (via heuristics) and logical (via causal verification).
Experimental Results
The system was evaluated using real-world data from thoroughbred horse racing, comparing its performance against professional human handicappers whose predictions are published in the Daily Racing Form.
Datasets
- Primary Dataset: 46 races used for initial training and testing.
- Generalization Dataset: 41 races from a different track to test the system's robustness without re-tuning.
Performance Metrics
HANDICAPPER demonstrated a significant advantage over human experts. In the initial set of 45 races, the system correctly predicted 24.4% of winners, while human experts achieved only 11.7%.
As the system processed more data, its performance improved, illustrating a clear learning curve. In the final 30 races of the first dataset, HANDICAPPER’s accuracy rose to 33.3%. When moved to the second dataset (the new track), the system's performance remained substantially better than the experts, proving that the heuristics were capturing general principles of the domain rather than overfitting to a specific location.
Discussion and Limitations
A key insight of this research is the shift toward episodic memory. Rather than tracking individual horses in isolation, HANDICAPPER treats entire races as episodes. This allows the system to judge similarity based on the context of the event, which is vital in high-variance domains.
However, the system has notable limitations:
- Knowledge Dependency: The causal model used for rationalization was manually constructed. A fully autonomous learner would need the capability to derive these causal links from data.
- Local Maxima: The Conservatism heuristic, while providing stability, risks trapping the system in "hill-climbing" behavior. By preferring small changes, the system might miss radical model shifts necessary for higher-level breakthroughs.
Conclusion
HANDICAPPER demonstrates that by encoding human-like biases as search heuristics and filtering them through causal logic, inductive learning systems can overcome the combinatorial explosion of complex domains. The system's ability to outperform human experts in horse racing suggests that this dual-layered approach is a powerful framework for practical machine learning applications.
Summary updated