Introduction
The development of Lethal Autonomous Weapons (LASs) is frequently defended by the "moral superiority" argument: the idea that machines, unlike humans, are immune to rage, fatigue, or bias. By encoding the Geneva Conventions directly into their software, proponents argue we can create a more ethical form of warfare. However, this paper demonstrates that "perfect" machine ethics is not just a design challenge, but a mathematical impossibility. By reducing the Halting Problem to a moral dilemma, the authors prove that an autonomous system can be presented with a clear "right" choice and full information, yet remain provably unable to algorithmically decide which action to take.
The Computability of Ethics
While philosophers have long debated the "Trolley Problem" as a matter of moral intuition, this paper shifts the focus to computability. The authors utilize Gedankenexperiments (thought experiments) to bridge the gap between ethics and recursion theory. They construct a series of increasingly complex scenarios culminating in the "Apex" dilemma. This scenario describes a deterministic situation where a unique "right" action exists, all information (including source code) is disclosed, and all actions are deterministic.
To prove that a machine still cannot solve this, the authors use a reduction from the Halting Problem. They demonstrate that determining whether a specific piece of "malicious" code—such as an instruction to fire on a civilian—will ever be executed is undecidable.
Proposition 10 and the Dead-Code Problem
The mathematical core of the paper is Proposition 10. It formally proves that the "dead-code-in-linear-time" problem is undecidable. This means that no algorithm can be guaranteed to identify instructions that will never run, even when the software is restricted to linear execution time. Consequently, no AI can be guaranteed to always identify malicious intent or unethical outcomes in software for all possible inputs. The authors conclude that any AI based on a Turing-equivalent computing device will necessarily fail to identify the "moral" choice in certain scenarios.
The Exculpation Trap and Accountability
The implications of this undecidability extend beyond computer science into the realm of international law. The authors warn of the "Exculpation Trap": if ethical decisions are algorithmically impossible in certain cases, governments might use "mathematical necessity" as a legal shield. War crimes committed by an AI could be dismissed as "provably unavoidable exceptions," creating a "legal limbo" where non-attributable aggression becomes a tool for state actors. This makes autonomous systems attractive for those seeking to commit the "perfect crime" without clear accountability.
Technical and Regulatory Recommendations
To mitigate these risks, the paper concludes with nine specific technical and legal recommendations for the regulation of LASs, including:
- Mandatory Formal Methods: Requiring the use of software verification tools (such as Coq or Isabelle) to mathematically prove the properties of autonomous systems.
- Tamper-Proof Logging: Implementing black-box logging using asymmetric encryption (RSA) to ensure that decision-making processes can be audited without the risk of log manipulation.
- Combatant Identification: A requirement for LASs to carry a "fixed distinctive sign" and a unique ID, similar to the requirements for human combatants under the Geneva Convention.
Conclusion and Future Work
The authors acknowledge that their proofs focus on "worst-case" scenarios derived from recursion theory. While these failures are mathematically possible, the paper does not quantify how likely they are to occur in real-world combat. Future research is suggested to investigate "average-case" complexity—essentially asking how often a machine can be expected to be right, even if it cannot be perfect. Nonetheless, the fundamental proof remains: the desire for perfectly ethical machines is currently at odds with the logical limitations of the hardware and software that control them.
Summary updated