Mesarovician Abstract Learning Systems

Tyler Cody

TL;DR

This paper proposes a shift in how we model Artificial General Intelligence (AGI), moving away from viewing it as a collection of specific "problem-task" pairings and toward viewing it as a formal Abstract Learning System (ALS). By applying Mesarovician systems theory, the author provides a mathematical framework to describe learning at varying levels of abstraction, allowing researchers to study the systemic properties of intelligence independently of specific algorithms.

What Problem Does This Paper Address?

Current AI research is dominated by a "problem-solving" paradigm. We define a domain (input distribution) and a task (mapping inputs to outputs) and then build a specific solution method to minimize an error function.

The author argues this approach is insufficient for AGI for three reasons:

  1. Unbounded Complexity: An AGI "in the wild" might not be easily decomposable into neat domain-task pairs.
  2. Emergence: Intelligence might be an emergent property of a "system of systems" that cannot be reduced to a single solution method.
  3. Contextual Dissolution: AGI will likely influence and be influenced by its environment so deeply that the boundary between the "intelligent agent" and its "context" becomes blurred.

To address this, the paper seeks a meta-theory—a way to talk about learning systems without getting bogged down in the specific math of a neural network or a reinforcement learning agent.

Methods

The paper leverages Mesarovician Abstract Systems Theory (AST). In AST, a system is defined simply as a relation on sets. The author extends this into Abstract Learning Systems Theory (ALST).

The core of the method is stratification—organizing the assumptions of a learning system into three hierarchical levels:

  • Elementary Level: Treats learning as a simple input-output box. It focuses on basic properties like stability and correlation.
  • Cascade Level: Decomposes the system into an inductive part (which picks a hypothesis) and a functional part (the hypothesis itself). This allows for the study of "how" the system interprets the world.
  • Goal-Seeking Level: Unpacks the inductive part to show internal feedback loops (consistency relations). This level acknowledges that learning is an active, seeking process that can't always be reduced to simple aggregation.

Datasets

Not discussed in the paper. This is a theoretical and mathematical treatise; it does not perform empirical tests on specific datasets.

Key Insights and Ideas

  • Learning as a System, Not a Procedure: By treating learning as a formal system ($S$), we can apply engineering concepts like "controllability" and "stability" to AGI in a way that traditional machine learning theory (focused on error bounds) often misses.
  • Mapping Theory to Abstraction: The paper provides a brilliant "Rosetta Stone" (Figure 4) that maps abstract system components to classic learning theory concerns. For example:
  • Elementary Level concerns map to Hardness and Distribution.
  • Cascade Level concerns map to Capacity (VC dimension), Falsifiability, and Sample Complexity.
  • Goal-Seeking Level concerns map to Convergence and Algorithmic Complexity.
  • Horizontal vs. Vertical Scoping: The framework allows engineers to "scope" their model. You can scope horizontally to see how an AGI interacts with other systems in its environment, or vertically to dive deeper into the mechanics of its goal-seeking behavior.

Key Results

The primary result is the formalization of Definition 1, which defines a Learning System $S$ as a relation across eight component sets: Algorithms ($A$), Data ($D$), Parameters ($\Theta$), Consistency Relations ($G, E$), Hypotheses ($H$), Inputs ($\mathcal{X}$), and Outputs ($\mathcal{Y}$).

Qualitatively, the paper demonstrates that this Mesarovician approach "calls back" to the original goals of AI from the 1950s—focusing on the thinking participant rather than just the specific math of the problem being solved. It provides a "latticework" for engineers to structure design decisions for complex, large-scale AI systems.

Limitations and Future Work

  • High Abstraction: Because the theory is a "meta-theory," it does not tell you how to build a better transformer or a more efficient optimizer. It provides the language to describe them, but the implementation remains an exercise for the engineer.
  • Heterarchical Relationships: While the paper focuses on hierarchical stratification (top-down), the author notes that heterarchical (peer-to-peer) relationships between learning systems are a vital area for future exploration.
  • Practical Tooling: The paper suggests this could be used for "Model-Based Systems Engineering" (MBSE), but it does not provide software tools or specific workflows for integrating ALST into current AI development pipelines.

Summary updated

Abstract

The solution methods used to realize artificial general intelligence (AGI) may not contain the formalism needed to adequately model and characterize AGI. In particular, current approaches to learning hold notions of problem domain and problem task as fundamental precepts, but it is hardly apparent that an AGI encountered in the wild will be discernable into a set of domain-task pairings. Nor is it apparent that the outcomes of AGI in a system can be well expressed in terms of domain and task, or as consequences thereof. Thus, there is both a practical and theoretical use for meta-theories of learning which do not express themselves explicitly in terms of solution methods. General systems theory offers such a meta-theory. Herein, Mesarovician abstract systems theory is used as a super-structure for learning. Abstract learning systems are formulated. Subsequent elaboration stratifies the assumptions of learning systems into a hierarchy and considers the hierarchy such stratification projects onto learning theory. The presented Mesarovician abstract learning systems theory calls back to the founding motivations of artificial intelligence research by focusing on the thinking participants directly, in this case, learning systems, in contrast to the contemporary focus on the problems thinking participants solve.
BibTeX
@inproceedings{cody2021mesarovician,
  author = {Tyler Cody},
  title = {{Mesarovician Abstract Learning Systems}},
  booktitle = {AGI},
  year = {2021},
  url = {https://openreview.net/forum?id=XLVLAZGml2},
}

Loading related content...

Figure