Abstract
Back-of-the-envelope (BotE) reasoning is a vital human cognitive ability that allows for rapid, "ballpark" estimations in situations where data is incomplete or precise models are unavailable. This paper formalizes the strategic knowledge underlying BotE reasoning by identifying a taxonomy of seven core strategies. We describe the implementation of BotE-Solver, a computational system that utilizes the Cyc knowledge base and these strategies to solve estimation problems. Our evaluation against expert-level physics problems and common-sense queries demonstrates that these seven strategies provide high coverage of the reasoning domain and that domain-independent strategies are particularly effective for everyday estimation tasks.
Introduction
In both professional engineering and daily life, individuals frequently encounter problems where exact data is missing. Whether estimating the feasibility of a new infrastructure project or approximating the annual cost of a national service, humans employ "Back-of-the-Envelope" (BotE) reasoning to generate answers within an order of magnitude of the truth.
While traditional Qualitative Reasoning (QR) in artificial intelligence often requires complete domain theories and high-fidelity models, human common sense is characterized by its flexibility and aggressive use of heuristics. This research addresses the gap between rigid AI modeling and the heuristic-driven nature of human estimation. By formalizing the fundamental strategies used to simplify complex problems, we aim to build AI systems capable of robust, common-sense reasoning.
The Taxonomy of BotE Strategies
We propose that BotE reasoning involves two distinct processes: Strategies, which transform a complex problem into simpler sub-problems, and Estimation, which assigns numeric values to those sub-problems. We represent a typical BotE problem as a triplet $(Q, O, ?V)$, where $Q$ is a quantity (e.g., cardinality), $O$ is an object (e.g., K-8 teachers), and $?V$ is the unknown value to be determined.
Through empirical analysis, we identified seven core strategies categorized by the type of transformation they perform:
1. Object-based Transformations
- Mereology (S1): Decomposing an object into its constituent parts. For instance, the total weight of a system is estimated as the sum of the weights of its parts.
- Similarity (S2): Using analogy to project values from a known similar object to the target object (e.g., estimating the population of one country based on a similar neighbor).
- Ontology (S3): Leveraging class-level knowledge. If a specific individual's attribute is unknown, the system uses the average value for that individual's class (e.g., using the average height of a "point guard" to estimate a specific player's height).
2. Quantity-based Transformations
- Density (S4): Utilizing rates or averages to derive a total (e.g., multiplying the number of students by the average teacher-to-student ratio).
- Domain Laws (S5): Applying fundamental physics equations, such as $F=ma$. This often involves the "spherical cow" approach—intentionally simplifying geometry or physics to make the math tractable.
3. System-based Transformations
- System Laws (S6): Applying conservation laws (e.g., energy or momentum) to bypass the need for modeling internal system complexities.
- Scale-up (S7): Extrapolating data from a small-scale model to estimate the behavior of a full-scale prototype.
System Implementation: BotE-Solver
To test this framework, we developed BotE-Solver. The system is built upon the Cyc knowledge base, which contains over 1.2 million facts, and utilizes the FIRE reasoning engine.
BotE-Solver manages the estimation process using an AND/OR tree. In this structure, "OR" nodes represent the various strategies that could potentially solve a goal, while "AND" nodes represent the specific sub-goals required to fulfill a chosen strategy. This allows the system to explore multiple reasoning paths and backtrack if a specific strategy lacks the necessary data in the knowledge base.
Strategic Distribution and Empirical Evaluation
We evaluated the coverage and effectiveness of these strategies using two distinct datasets: Clifford Swartz’s Back-of-the-Envelope Physics and a set of 13 common-sense estimation problems.
As shown in the table above, the seven strategies accounted for 94% of the reasoning steps used in the expert physics textbook. Interestingly, the distribution of strategies varies significantly by domain. In physics problems, domain-specific laws (S5) are the most frequent (37%). However, in common-sense reasoning, domain-independent strategies like Density (S4) and Mereology (S1) dominate, accounting for approximately 85% of the reasoning steps. This suggests that a small set of general-purpose strategies is sufficient for a wide range of everyday estimation tasks.
Performance and Accuracy
BotE-Solver successfully solved all 13 common-sense problems. Because the goal of BotE reasoning is to be in the "ballpark," we measured success by the system's ability to stay within an order of magnitude of the actual value. For example, the system estimated the annual U.S. healthcare cost at $0.8 trillion, which was approximately half the actual value at the time—a highly successful result for a rough estimation task.
Conclusion and Future Work
Our findings indicate that BotE reasoning can be effectively modeled through a finite set of strategies. While the current system is highly capable, it faces a "knowledge bottleneck" when the Cyc KB lacks specific rates or densities. Future work will focus on improving the system's ability to "guess" typical values when data is missing—a capability we are exploring through a sub-project called CARVE. Additionally, we aim to apply these strategies in reverse to help AI systems explain complex quantitative data found in news articles to human users.
Summary updated