What's in a Deep Model? A Characterization of Knowledge Depth in Intelligent Safety Systems

TL;DR

This paper tackles the ambiguity in defining "deep" vs. "shallow" AI models by proposing an operational definition based on explicit knowledge representation. It then applies this to "intelligent safety systems" (ISS), analyzing how increasing model depth impacts plan determination and evaluation in critical domains like nuclear power and chemical plants.

What Problem Does This Paper Address?

The field of expert systems, particularly in safety-critical applications, has long distinguished between "deep" and "shallow" models. Shallow models rely on direct inference from observed facts and heuristics, making them easy to build and efficient but inflexible and prone to errors when encountering novel situations. Deep models, conversely, aim to reason from first principles, offering greater robustness and better explanations but at the cost of increased complexity and slower execution. However, a clear, operational definition of "knowledge depth" and guidelines for choosing the appropriate depth for a given application have been lacking. This paper aims to bridge this gap by providing a framework for analyzing and comparing models of varying depth within the context of intelligent safety systems.

Methods

The authors propose an operational definition of "knowledge depth": Model M' is deeper than model M if knowledge implicit in M is explicitly represented or computed in M'. They then analyze the reasoning task of "providing a safety function" in intelligent safety systems (ISS). This task is decomposed into "plan determination" and "plan evaluation." For each subtask, they present a hierarchy of progressively deeper models, illustrating how knowledge representation and reasoning change with increasing depth.

For plan determination, they explore:

  • Hard-coded plans: Situation-action pairs (e.g., JESQ system for computer operations).
  • Hard-coded paths: Explicitly representing component paths for safety functions (e.g., REACTOR system for nuclear reactors).
  • Generating plans from system structure: Using a graph representation of system components to find paths.
  • Generating plans from system structure and behavior: Explicitly modeling component behaviors alongside structure for more complex systems.

For plan evaluation, they examine:

  • Hard-coded priorities: Simple numeric priorities for conflict resolution.
  • Utility theory with encoded objective values: Representing attributes and objectives explicitly to compute priorities.
  • Utility theory with computed objective values: Encoding functions to compute objective values, allowing for deeper reasoning about plan desirability.

Datasets

Not discussed in the paper. The paper focuses on abstract analysis and uses hypothetical examples and references to existing systems (JESQ, REACTOR) rather than introducing new datasets.

Key Insights and Ideas

  • Operationalizing "Depth": The paper provides a concrete, albeit informal, definition of knowledge depth that is crucial for systematic analysis.
  • Depth Trade-offs: Deeper models offer robustness, better explanations, and easier maintenance for complex systems, but come with increased computational cost and development effort.
  • Domain Independence: The proposed analysis framework for plan determination and evaluation is designed to be domain-independent, allowing for broader applicability.
  • Subtask-Specific Depth: The authors argue that depth should be considered at the subtask level, as composite systems with varying depths in different subtasks are not strictly ordered by overall "depth."
  • Behavioral Modeling is Key: For complex systems with heterogeneous components, explicitly modeling component behavior (beyond just structure) is essential for deeper reasoning and plan generation.

Key Results

The paper does not present quantitative results but rather a qualitative analysis of different modeling approaches. The key "results" are the characterization of the different levels of depth for plan determination and evaluation, and the discussion of their respective advantages and disadvantages in various application contexts (nuclear power, chemical plants, computer operations). For instance, they illustrate how a deeper model can handle unexpected component failures more gracefully than a shallow one.

Limitations and Future Work

The authors acknowledge that their definition of depth is informal and that future work should involve exploring alternative definitions and analyzing other reasoning tasks. They also suggest that the ultimate goal is to develop guidelines for selecting model depth based on general domain characteristics, which would require more extensive analysis across various domains and reasoning tasks. The paper sketches an analysis, and further empirical validation and refinement of the proposed framework are implied.

Summary updated

BibTeX
@inproceedings{1987what,
  title = {{What's in a Deep Model? A Characterization of Knowledge Depth in Intelligent Safety Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1987},
}
Figure