When Machine Learning Implies Intelligence [Young Professionals]

Patrick van der Smagt

Abstract

While supervised learning has achieved remarkable success in pattern recognition for large-scale technology firms, its reliance on massive, human-annotated datasets makes it fundamentally ill-suited for data-scarce environments such as manufacturing and robotics. To bridge the gap between current artificial intelligence and true biological-like intelligence, the field must pivot toward Bayesian methods, incremental learning, and unsupervised latent-variable models. This paper explores the theoretical frameworks necessary to move beyond the "data-hungry" paradigm and toward systems that can learn from observation and experience.

Introduction

The current success of machine learning (ML) is largely synonymous with supervised learning. By leveraging massive datasets and high-performance computing, large IT companies have effectively "solved" many pattern recognition problems. However, this model creates a significant economic and geographic divide. While the US-based tech sector thrives on big data, much of the global economy—particularly in Europe—is driven by small and medium enterprises (SMEs) in the manufacturing and robotics sectors. These entities operate in environments where data is complex, expensive to obtain, and rarely labeled. To achieve intelligence that mimics biological organisms, ML must evolve to operate effectively on small datasets without the need for a human "oracle."

The Limitations of the Frequentist Paradigm

Standard neural networks are inherently "data-hungry" due to their reliance on frequentist statistics. Most current models utilize Maximum Likelihood Estimation (MLE), which assumes specific probability densities. For instance, in regression tasks, these models rely on the central limit theorem—a principle that only holds true when data is abundant. When data is sparse, these frequentist assumptions break down, leading to models that are overconfident and prone to failure in real-world applications.

Furthermore, supervised learning is fundamentally un-biological. In nature, learning is rarely supervised by an external labeler. Biological systems learn through continuous observation and the accumulation of experience, building upon previous knowledge rather than starting from scratch for every new task.

Theoretical Frameworks for Biological Intelligence

To address these limitations, several theoretical frameworks are proposed to move AI toward a more robust, biological model of intelligence:

Bayesian Neural Networks (BNNs)

Unlike standard networks, BNNs represent uncertainty within the weights of the network itself. By moving away from point estimates and toward probability distributions, BNNs can learn from significantly fewer data points. This allows the model to express "doubt," which is critical for safety-critical applications in robotics and manufacturing.

Latent-Variable Models

To eliminate the need for human-provided labels, research must focus on latent-variable models. These models combine probabilistic approaches with neural networks to identify underlying structures in data, particularly in time-series information. This allows for unsupervised learning, where the system identifies patterns and causal relationships autonomously.

Incremental and Physics-Informed Learning

Biological intelligence is cumulative. The author advocates for incremental learning approaches that allow systems to build upon previously acquired knowledge. Additionally, integrating handcrafted physical models directly into neural network architectures can provide a "head start," allowing the system to respect the laws of physics while learning from new data.

The Scalability Paradox

We are currently in what the author describes as a "1990s moment." Just as the early 90s saw the first working versions of technologies that would later become ubiquitous, we now possess the building blocks for unsupervised and incremental learning (such as autoencoders). However, a significant hurdle remains: scalability. While Bayesian approaches solve the "small data" problem, they are computationally intensive and currently do not scale as efficiently as frequentist models. The challenge for the next generation of AI research is to make these sophisticated probabilistic models as scalable as the supervised counterparts used today.

Conclusion

Supervised learning may be a solved problem for large-scale pattern recognition, but it is not the final destination for artificial intelligence. The resurrection of AI’s original promise—to create truly intelligent, autonomous agents—lies in the development of unsupervised Bayesian learning and latent-variable models. By focusing on uncertainty, incremental knowledge, and the integration of physical models, the field can move toward a form of machine learning that truly implies intelligence.

Summary updated

Abstract

In the early 1990s, we thought neural networks would be commonplace soon and used everywhere. It took 30 years longer, but that point has now been reached. Neural networks can learn any function from complex and high-dimensional data sets, image or pattern recognition is solved, and probability theory serves as a sound mathematical basis for it all. Neural network -based solutions to supervised learning have provided us with many applications related to image recognition, human -machine interaction, and so on, where large numbers of human -annotated data are available. Where these scale, they scale fast. This gives sound business models for IT companies with large user bases.
BibTeX
@article{smagt2020when,
  author = {Patrick van der Smagt},
  title = {{When Machine Learning Implies Intelligence [Young Professionals]}},
  journal = {IEEE Robotics Autom. Mag.},
  year = {2020},
  doi = {10.1109/mra.2020.2986567},
}
Figure