Suppose many small devices, such as phones, sensors, or kiosks running on solar or scavenged power, jointly fine-tune a shared language model to reason in the languages their users speak. The devices holding low-resource languages are often the ones with the least reliable power. The standard fix for uneven participation is to weight each device's update by how rarely it gets to contribute. That fix keeps the average update fair, but it makes the update very noisy for exactly the languages that need help most. This note proposes a different division of labor, based on what the cross-lingual transfer literature has learned. Reasoning skill transfers well from English or other high-resource data, while alignment of a language's words into the model's shared space does not transfer and needs target-language text. So energy-poor, low-resource devices should spend their scarce energy only on a small "language alignment" module. That update is cheap enough to fit inside one battery charge. Energy-rich devices carry the expensive reasoning update.
The physical constraint that makes this matter is finite battery capacity. Information theory for energy-harvesting transmitters shows a sharp threshold. When the battery cannot hold one full energy arrival, the device wastes energy to overflow and its throughput depends on fine details of the arrivals. Once the battery exceeds that size, extra capacity barely helps (Capacity of the Energy Harvesting Channel with a Finite Battery; Near Optimal Energy Control and Approximate Capacity of Energy Harvesting Communication). The analogous threshold for learning is whether a device's battery can hold the energy of one local update. Our claim is that role-splitting the fine-tuning lets low-resource devices cross this threshold. We expect it to beat participation reweighting in the small-battery regime, and we expect the advantage to disappear once batteries are large. That second prediction is what makes the claim falsifiable.
Core Claim
In federated multilingual reasoning fine-tuning, each device trains locally on private data and a server averages the updates. If devices harvest energy into finite, leaky batteries, and if language resource level is correlated with energy availability, then assigning parameter subsets by role gives higher low-resource reasoning accuracy per Joule than unbiased participation reweighting. Under role assignment, a small alignment module is trained only by target-language devices, and the reasoning adapter is trained mainly by high-resource devices. The gain should be concentrated where per-update energy of the reweighted baseline exceeds battery capacity $\bar{B}$ on low-resource devices, or where their participation probability is small.
The mechanism behind the second condition is simple. Energy-Harvesting Distributed Machine Learning makes the aggregate unbiased by scaling device $i$'s gradient $g_i$ by $1/\beta_i$ when it participates, where $\beta_i$ is its energy-arrival probability. The resulting estimate $\hat g = \sum_i \frac{\mathbb{1}_i}{\beta_i} g_i$, with $\mathbb{1}_i$ indicating that device $i$ had energy, carries extra variance
$$\sum_i \frac{1-\beta_i}{\beta_i}\,\lVert g_i\rVert^2 .$$
This blows up as $\beta_i \to 0$. That is the case of an energy-poor Swahili or Telugu device, which is the case we care about. Role-splitting attacks both factors. It makes $g_i$ low-dimensional and confines it to parameters no high-resource device touches, so these gradients never compete with dominant English gradients. It also makes the update cheap, which raises the effective $\beta_i$, because a smaller energy requirement is met more often.
What Prior Work Supplies and Leaves Open
The role split is grounded in three findings about how cross-lingual reasoning transfer works. Together they motivate it. LinguaLIFT first trains a language-alignment layer on code-switched text, freezes it, and then tunes on English-only reasoning data. It reports 58.9% average accuracy on the low-resource languages of the 48-language MMWP benchmark, against 39.4% for multilingual SFT and 19.3% for monolingual SFT. This shows the two stages are separable. It is still a centralized recipe that assumes all data and compute sit in one place. Bridging the Language Gap finds that factual knowledge sits in language-shared neurons and that non-English failures come from misrouting attention heads. Tuning about 40 heads (<2% of parameters) gives a 15% relative gain on XHOPREASON with an MMLU drop under 1 point, while full fine-tuning and DPO sometimes lose over 60 MMLU points. This supplies a second candidate for the cheap alignment module and a warning against full-parameter updates. Multilingual Mathematical Reasoning and TituLLMs both report that high-resource data acts as a scaffold for low-resource reasoning, which is why we let energy-rich devices carry the reasoning update.
Two papers constrain the design. Breaking Language Barriers in Multilingual Mathematical Reasoning shows that multilingual data also helps English: GSM8K rises from 42.4% to 50.8%. So low-resource text is not useless for reasoning, and our ablations must test whether removing low-resource devices from the reasoning adapter costs accuracy. Beyond English-Centric Training finds that GRPO, a reinforcement learning method, beats SFT by 17.5 to 25.2 points on MGSM. Its rollouts are expensive, however, and we keep the study to SFT, where per-update energy is predictable enough to compare against a battery.
The energy side supplies the method. The finite-battery corridor of A General Framework for the Optimization of Energy Harvesting Communication Systems with Battery Imperfections shows that leakage pushes optimal use toward early, bursty spending. Throughput Maximization for an Energy Harvesting Communication System with Processing Cost shows that a fixed wake-up cost makes it optimal to stay idle below a threshold and then act in bursts. We reuse that logic for local scheduling: a device runs an update only when it can complete one, and it batches to amortize accelerator wake-up. The closest prior work, Güler and Yener, studies CIFAR-10 with a small CNN. It does not model finite buffers, leakage, or heterogeneity in what different devices' data can teach the model. All three are the substance of this proposal.
Research Program
We build a simulator for federated fine-tuning of an open 3B base model (Qwen2.5-3B-Base, as used by Huang et al.). It has 40 to 100 simulated devices, each holding data in one of the ten MGSM languages. Each device has a battery of capacity $\bar{B}$ with per-step leakage, and receives Bernoulli energy arrivals with probability $p_i$. As a second condition, arrivals follow a two-state Markov diurnal process, because the constant-gap results break down for bursty, correlated arrivals.
Per-update energy is measured rather than assumed. We profile the forward and backward pass plus upload size for each update type (full adapter, LoRA reasoning adapter, alignment layer, 40-head subset) on a representative edge accelerator. We record Joules, and we report every result against these measured costs. Energy and language are correlated by assigning lower $p_i$ to low-resource-language devices. The correlation strength is a swept parameter, including zero correlation as a control.
The proposed method has three parts. Low-resource devices hold code-switched and native text and train only the alignment module, using either the LinguaLIFT layer or the misrouted heads identified once on a held-out probe set. High- and medium-resource devices train a LoRA reasoning adapter on translated MGSM8KInstruct-style data, keeping the alignment module frozen. Every device runs a threshold-and-burst local schedule.
We map accuracy as a function of $\bar{B}$ relative to per-update cost and of $p_i$. The goal is to locate the regime boundary empirically and to test the predicted saturation.
Baselines
The naive reference is energy-agnostic federated LoRA SFT on all languages: any device with enough energy uploads, and the server averages without correction. This is the analogue of the Güler and Yener energy-agnostic benchmark, which reached 64% versus about 80% on CIFAR-10. We expect it to drift toward energy-rich languages. A second naive point is wait-for-all synchronization, which reached 52% there because it stalls.
The strongest competitor is Güler and Yener's $1/\beta_i$ unbiased scaling applied to federated LoRA multilingual SFT, with the same threshold-and-burst local schedule we give our method. It directly targets participation bias and, given equal schedules, differs from our method only in how parameters are assigned. We run it on identical data, devices, energy traces, and total harvested Joules, and we give both methods the same learning-rate and rank search budget. No reported result was found on Lacuna for this method on multilingual LLM fine-tuning or MGSM. Its only reported number is roughly 80% CIFAR-10 accuracy, which matched infinite-energy SGD.
As an upper reference, we run centralized LinguaLIFT and centralized multilingual SFT with unlimited energy on the same model. LinguaLIFT reports 58.9% MMWP low-resource accuracy on its own models, and we re-run it rather than borrow that number. We also include MathOctopus-7B at 41.9% average MGSM accuracy as an external anchor.
Beating the strongest baseline means three things. Low-resource MGSM and MMWP accuracy must be higher at equal harvested energy, with confidence intervals over at least three energy-trace seeds. High-resource accuracy must fall by no more than about one point. MMLU forgetting must not be worse.
Three ablations isolate the key ingredient. The first is role-split without energy correlation, which tests whether the gain is just LinguaLIFT's two-stage benefit. The second is role-split with roles swapped. The third is our method with low-resource devices also contributing to the reasoning adapter, which tests the Chen et al. concern about losing low-resource reasoning signal.
Why not just…?
Why not just give every device a small LoRA adapter and keep $1/\beta$ reweighting? This is the real threat, and it is the strongest baseline above. If small adapters already fit every battery and participation is not too rare, reweighting may match us. The derived variance term predicts it will not when $\beta_i$ is small, because low-resource gradients still compete for the same adapter as English gradients. The battery sweep settles it. If reweighted LoRA matches role-splitting across the small-battery regime, the claim is wrong and we will say so.
Why not just collect low-resource text centrally and run LinguaLIFT on a server? Where data can be moved, this is best, and it serves as our upper reference. The proposal targets the setting where text stays on device. LinguaLIFT's own design shows that the irreplaceable target-language ingredient is the alignment stage, and that is exactly what we push to the devices holding that text.
Why not just make batteries bigger? The capacity results predict saturation once $\bar{B}$ exceeds one arrival. Analogously, we predict our advantage shrinks once batteries hold one full reweighted update. We report that crossover explicitly. If it occurs at battery sizes that real harvesting devices already have, the practical case narrows to devices with low arrival rates.
Decisive Checks
Evaluation uses MGSM per-language accuracy (separating Swahili, Telugu, Bengali, and Thai from high-resource languages), MMWP low-resource accuracy, MSVAMP for out-of-domain math, and XHOPREASON when the head-subset alignment module is used. MMLU and Belebele track forgetting. The cost metric is harvested Joules per accuracy point, measured from the profiled update costs.
Three outcomes decide the proposal. First, at small $\bar{B}$ and strong energy–language correlation, role-splitting should beat reweighted LoRA on low-resource MGSM and MMWP. Second, the gap should close as $\bar{B}$ and $p_i$ grow, and should be absent when energy and language are uncorrelated. Third, swapping roles should hurt. If the first outcome fails, the idea is refuted. If the gain appears even without energy correlation, it is a centralized-recipe effect rather than an energy effect, and we will report it that way.
References
- Capacity of the energy harvesting channel with a finite battery
- Near Optimal Energy Control and Approximate Capacity of Energy Harvesting Communication
- Energy-Harvesting Distributed Machine Learning
- LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning
- Bridging the Language Gap: Uncovering and Aligning Shared Circuits for Multi-Hop Reasoning in Multilingual LLMs
- Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English
- TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking
- Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
- Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
- A general framework for the optimization of energy harvesting communication systems with battery imperfections
- Throughput maximization for an energy harvesting communication system with processing cost