ReCALM: A Residual-enhanced Crowdsourced Analytic Learning Mechanism for Distributed Continual Service Construction

Kejia Fan, Yajiang Huang, Jianheng Tang, Jinfeng Xu, Huiping Zhuang, Anfeng Liu, Houbing Herbert Song, Yunhuai LiuarXiv 2026 · arXiv

TL;DR

ReCALM introduces a deep, multi-layer residual architecture for analytic learning, moving beyond the single-layer linear constraints of previous gradient-free methods. By utilizing a novel cross-recursive strategy, it enables distributed continual learning that is mathematically invariant to data heterogeneity while slashing computational costs by over 94% compared to traditional gradient-based approaches.

What Problem Does This Paper Address?

In the world of mobile crowdsensing, AI models are built using data from thousands of distributed devices (workers). This presents two massive hurdles:

  1. Spatio-Temporal Data Heterogeneity (ST-DH): Data varies wildly across different users (spatial) and changes over time as new tasks are introduced (temporal).
  2. Catastrophic Forgetting: Traditional gradient-based models (like those trained with backpropagation) tend to "overwrite" old knowledge when learning new tasks, especially in decentralized settings.

Recent "Analytic Learning" (AL) methods offered a solution by using recursive closed-form solutions (least squares) instead of gradients. Because these solutions are exact, they don't "forget." However, AL has historically been stuck in "shallow land"—limited to single-layer linear classifiers that underfit complex data. The authors ask: Can we build deep, nonlinear analytic networks that still support distributed, continual updates without needing raw data?

Methods

The authors propose ReCALM (Residual-enhanced Crowdsourced Analytic Learning Mechanism), which introduces several innovations:

  • Gradient-Free Residual Blocks: Inspired by ResNet, ReCALM stacks layers where each layer learns to fit the residual (the error) of the previous layer. Crucially, these blocks are learned analytically. To keep the math tractable for continual learning, they "anchor" the input of each residual block to the fixed initial features ($\Phi_0$).
  • The Sandwiched Least Squares Problem: The learnable transformation matrices in the residual blocks are solved using a specialized form of the generalized Sylvester matrix equation. This allows for nonlinear representation learning without a single step of backpropagation.
  • Cross-Recursive Strategy: This is the "secret sauce" for distributed training. The server doesn't need raw data; instead, clients compute and upload Gram-type statistics (inner products of features and labels). ReCALM uses a dual-recursion path:
  • Phase-wise recursion: Accumulates statistics over time as new data arrives.
  • Layer-wise recursion: Reconstructs deep features across layers using the accumulated statistics, ensuring the model remains equivalent to one trained on all data centrally.

Datasets

The researchers evaluated ReCALM using three standard benchmarks, partitioned to simulate a "Si-Blurry" continual learning environment (mixing old and new classes):

  • CIFAR-100: 100 classes of small images.
  • Tiny-ImageNet: A subset of ImageNet with 200 classes at 64x64 resolution.
  • ImageNet-R: A challenging dataset containing 200 classes with various artistic renditions (paintings, sketches, etc.).

Key Insights and Ideas

  • Exact Retention: Because the model is solved via closed-form equations rather than iterative steps, it achieves "exact analytic retention." It doesn't just mitigate forgetting; it mathematically preserves the optimal solution for all previously seen data.
  • Heterogeneity Invariance: A major "aha" moment is the proof that ReCALM is invariant to how data is partitioned. Whether 100 clients provide 10 images each or 1 client provides 1,000, the resulting model weights are identical.
  • Decoupling Depth from Backprop: The paper demonstrates that you can achieve the benefits of deep representation learning (better feature separability) using a forward-only, layer-by-layer construction, provided you use residual connections correctly.

Key Results

  • Accuracy Boost: With 20 residual layers, ReCALM improved accuracy by 8.85% to 9.83% over the single-layer analytic baseline (AFCL/CALM) across all datasets.
  • Efficiency: ReCALM reduced computational costs by 94% to 99% compared to gradient-based federated learning methods like FedAvg or FedProx.
  • Scalability: On CIFAR-100, ReCALM reached 67.87% accuracy, significantly outperforming other Federated Continual Learning (FCL) baselines which struggled with the "blurry" task boundaries.
  • Training Speed: While gradient-based methods take thousands of seconds to converge, ReCALM (even with 20 layers) completes updates in a few hundred seconds.

Limitations and Future Work

  • Backbone Dependency: Like many analytic methods, ReCALM relies on a frozen, pre-trained backbone (e.g., ResNet-18 or ViT) for initial feature extraction. While the residual layers refine these features, the model's ultimate ceiling is somewhat tied to the quality of that initial backbone.
  • Memory for Statistics: While ReCALM saves on computation, it requires storing Gram-type statistics. As the number of features ($d$) grows, the storage requirement for these matrices (which scale at $O(d^2)$) could become a bottleneck for extremely high-dimensional models.
  • Future Directions: The authors suggest exploring adaptive feature extraction (letting the model decide which features are most important) and testing the system in more volatile crowdsourcing environments with highly dynamic client participation.

Summary updated

Abstract

Crowdsourcing has emerged as a promising paradigm for constructing AI services by harnessing the collective capabilities of distributed workers. As consumers' demands continually evolve, service models must acquire new knowledge while retaining previously learned information. However, due to the inherited reliance on gradients for backpropagation, existing approaches typically suffer from catastrophic forgetting during continual model updates on heterogeneous data. Recent studies have introduced analytic learning, a gradient-free paradigm that achieves invariance through recursive closed-form solutions. Nevertheless, existing analytic methods are typically constrained by single-layer linear networks, which tend to suffer from underfitting due to their inability to perform deep representation learning. To address this issue, we propose ReCALM, a Residual-enhanced Crowdsourced Analytic Learning Mechanism that achieves deep nonlinear invariance for distributed continual service construction. Inspired by the success of ResNet, we develop gradient-free residual blocks for ReCALM with closed-form solutions. To implement distributed continual training for ReCALM, we further devise a cross-recursive strategy via least squares. Both theoretical and experimental analyses verify ReCALM's dual superiority in heterogeneity invariance and deep representation. With 20 residual layers, ReCALM improves accuracy by 8.85%-9.83% over the single-layer analytic baseline, while reducing computational costs by over 94% compared with gradient-based baselines.
BibTeX
@misc{fan2026recalm,
  author = {Kejia Fan and Yajiang Huang and Jianheng Tang and Jinfeng Xu and Huiping Zhuang and Anfeng Liu and Houbing Herbert Song and Yunhuai Liu},
  title = {{ReCALM: A Residual-enhanced Crowdsourced Analytic Learning Mechanism for Distributed Continual Service Construction}},
  year = {2026},
  eprint = {2611.19276},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19276},
}
Figure