Abstract: What is neuroinformatics? We can define it as a direction of science and information technology, dealing with development and study of the methods for solution of problems by means of neural networks. A field of science cannot be determined only by fixing what it is "dealing with". The main component, actually constituting a scientific direction, is "THE GREAT PROBLEM", around which the efforts are concentrated. One may state even categorically: if there is no a great problem, there is no a field of science, but only more or less skilful imitation. What is "THE GREAT PROBLEM" for neuroinformatics? The problem of effective parallelism, the study of brain (solution of mysteries of thinking), etc are discussed. The neuroinformatics was considered not only as a science, but as a services sector too. The main ideas of generalized technology of extraction of explicit knowledge from data are presented. The mathematical achievements generated by neuroinformatics, the problem of provability of neurocomputations, and benefits of neural network realization of solution of a problem are discussed.
Introduction: Defining a "Great Problem"
In the late 1990s, A.N. Gorban proposed a manifesto for neuroinformatics, defining it not merely as a collection of "brain-like" tools but as a rigorous scientific discipline. For any field to be considered a true science, it must be centered around a "Great Problem." For neuroinformatics, this problem is twofold: achieving effective parallelism and extracting human-readable knowledge from complex, high-dimensional data.
The Parallelism Bottleneck
A primary driver for neuroinformatics is the quest for "effective parallelism." Traditional computing often faces the "Minsky Hypothesis," which suggests that as the number of processors increases, the gain in efficiency follows a logarithmic curve rather than a linear one, leading to diminishing returns.
Neuroinformatics seeks to overcome this bottleneck by using neural networks as a universal architecture for fine-grained parallelism. The goal is to move toward "dreams" of linear efficiency, where doubling the hardware truly doubles the performance.
From Black Boxes to Explicit Knowledge
A common critique of neural networks is their "black box" nature. Gorban addresses this "Interpretability Gap" by outlining methods for "rarefication" (pruning). By removing unnecessary neurons and synapses through sensitivity analysis, a complex network can be reduced to its core logical structure. This pruned network can then be translated into explicit "If-Then" rules or fuzzy logic, converting implicit weights into natural-language insights that humans can verify.
The Mathematical Miracle of Backpropagation
One of the most significant technical insights in the paper is the "Gradient Miracle." It is a common misconception that calculating a gradient for $n$ variables requires $n$ times the effort of a single function evaluation. Gorban highlights that backpropagation allows for the computation of gradients in roughly three times the time of a single evaluation ($T_{gradH} \approx 3T_H$), regardless of the dimensionality. This efficiency is a fundamental mathematical achievement that enables the training of massive networks.
The Scope and Future of Neuroinformatics
As the field matures, it moves from "elite marks"—specialized academic tools—to "profane techniques" used by non-experts. This democratization is a sign of a healthy, maturing technology. However, challenges remain, particularly in the "theory of reliability," as there is still a lack of standard tools for the exact evaluation of a trained network's reliability.
The future of the field lies in specialized "neurocomputers" and the continued "neironization" of diverse problem domains, from financial forecasting to plasma stability in thermonuclear synthesis. By focusing on the dual goals of computational efficiency and knowledge extraction, neuroinformatics establishes itself as a cornerstone of modern information technology.
Summary updated