Abstract
Facebook operates a family of services used by over two billion people daily on a huge variety of mobile devices. Engineers monitor and triage millions of crash reports logged each day to check for bugs and regressions. Debugging groups of crashes is a manually intensive process requiring deep domain expertise. We introduce Continuous Contrast Set Mining (CCSM), a discriminative pattern mining approach that directly analyzes continuous data without discretization. We also define a unified anomaly score and uncertainty measures to communicate confidence to developers. We demonstrate CCSM's value on Facebook production logs, achieving a 40x speedup over baseline discretization approaches.
Introduction
At Facebook's scale, "one-in-a-million" bugs occur thousands of times daily. Triaging these reports is a significant engineering challenge. While stack traces often provide clues, certain errors—such as Out-of-Memory (OOM) crashes on iOS—frequently lack them, leaving metadata as the primary source for debugging. Engineers must identify what distinguishes a specific group of crashes from the general population to find root causes.
Methodology: Continuous Contrast Set Mining (CCSM)
Traditional Contrast Set Mining (CSM) is limited to categorical data. To handle continuous features like process uptime or navigation logs, researchers typically use "discretization" (bucketing), which is computationally expensive and can obscure patterns. CCSM addresses this by applying contrast learning directly to continuous data. The system architecture, shown in Figure 1, integrates both categorical and continuous feature extraction into a unified pipeline.
Key technical innovations include:
- Statistical Shift: Utilizing one-way ANOVA F-tests instead of Chi-Squared tests to identify significant differences in continuous feature averages between crash groups.
- Unified Anomaly Scoring: Implementing Cohen’s d for continuous data and Cohen’s h for categorical data. This allows different feature types to be ranked on a single, comparable scale of "effect size."
- Navigation Encodings: User journeys are transformed into continuous vectors using TF-IDF weights, enabling the algorithm to process sequential navigation paths mathematically.
Key Insights
A critical finding of this research is that raw frequency does not equate to an anomaly. As illustrated in Figure 2, while certain app builds may have high crash counts, they might simply be the most prevalent builds in the general population.
CCSM identifies true anomalies by comparing observed rates against expected distributions. For instance, a build with fewer total crashes might be flagged as a higher priority if its crash rate is significantly higher than expected. Furthermore, CCSM provides confidence intervals, allowing developers to gauge the reliability of a detected pattern before committing to an investigation.
Results and Impact
The researchers evaluated CCSM on a dataset of 60,000 iOS OOM crashes from Facebook production logs.
- Performance: CCSM delivered results in approximately 7.5 minutes, representing a 40x speedup over discretization-based baselines, which often failed due to memory limits.
- Actionability: In a manual audit of 32 high-ranked anomalies, 12 were directly useful for resolution, and 18 provided supporting evidence for root causes.
- Root Cause Discovery: The tool successfully pinpointed a specific navigation path involving a comments module that was causing OOM errors, leading to a confirmed fix by product engineers.
Conclusion
CCSM represents a significant advancement in automated crash debugging. By eliminating the need for discretization, it provides a faster, more accurate, and more scalable way to analyze complex crash metadata. Future work will explore handling feature dependencies through cross-feature entropy and adopting non-parametric tests for highly skewed data distributions.
Summary updated