Introduction
Software engineering often suffers from the "garbage in, garbage out" principle. While significant effort is spent on verifying that code meets its specification, the specification itself is frequently taken on blind faith. However, the cost of fixing a requirement error discovered late in the development cycle is exponentially higher than fixing it at the start. This paper introduces a "suspicious" approach to software design, utilizing a knowledge-based critic to scrutinize formal specifications against high-level domain policies. By bridging the gap between technical "how" and organizational "why," the system identifies missing goals, obstructions, and unnecessary complexity before a single line of production code is written.
Methodology: The Kate Critic
The authors developed a critic component within the Kate project, designed to treat every specification as potentially flawed until proven rational. The system operates on three primary pillars:
1. Augmented Petri-nets
To analyze a specification, both the domain model and the user’s specific design are represented using an augmented Petri-net language. This formalism supports objects, transitions, and non-deterministic control, enabling the system to "run" the specification as a dynamic model.
2. Policy-Based Critiquing
The system maintains a library of "model cases" linked to high-level domain policies, such as "Maintain user privacy" or "Minimize staffing costs." The critic evaluates the specification for three types of discrepancies:
- Non-support: A critical policy has no corresponding implementation in the spec.
- Obstruction: A specified component actively prevents a policy from being achieved.
- Superfluousness: A component exists without supporting any known high-level policy.
3. Dynamic Simulation
To provide persuasive feedback, the system generates scenarios—simulated executions of the Petri-net. For instance, it might simulate a "run on the library" to demonstrate how resources are depleted, providing a "verbal protocol" that explains flaws more intuitively than static error messages.
Conceptual Framework: Policy Hierarchies and Conflict
A central insight of this work is that policies are rarely flat; they are hierarchical and often in conflict. A library system, for example, must balance the goal of providing users with a large selection against the need to ensure fairness through borrowing restrictions.
By refining high-level goals into sub-policies (e.g., "preventing theft" vs. "reminding forgetful users"), the critic can be tuned to different perspectives, such as that of a "selfish user" or an "administrator." This acknowledges that most specifications are compromises between competing interests.
Evaluation: The Library Database System
The system was tested using the standard "Library Database System" problem from the 1987 International Workshop on Software Specification and Design. The critic successfully identified several subtle issues that might escape a standard technical review:
- Privacy Violations: Flagged query actions that revealed who last borrowed a book as obstructive to privacy policies.
- Operational Inefficiency: Noted that requiring manual staff intervention for every checkout obstructed the policy of minimizing circulation staff.
- Incomplete Logic: Identified the absence of mechanisms to manage the lifecycle of "staff" and "borrower" entities.
When compared to a human library analyst, the system's critiques were remarkably similar, though the human expert remained superior at prioritizing which policy violations were acceptable trade-offs.
Limitations and Future Directions
While promising, the current system faces challenges in "rematching"—automatically updating the link between the domain model and the specification after a user makes edits. Additionally, the authors aim to move beyond simple binary policy importance toward a more nuanced utility model. Future work will also focus on generating a progression of scenarios, from best-case to worst-case, to help designers determine if a potential failure is statistically significant enough to warrant a redesign.
Summary updated