Camera equalisation of an 11.5x drop in environmental illuminance: four imitation-learning policies on an SO-101 arm

Simon CowlingarXiv 2026 · arXiv

TL;DR

This paper reveals that environmental light levels (lux) are a poor proxy for what a robot actually "sees." By testing four popular imitation-learning policies under an 11.5x drop in room illuminance, the authors found that automatic camera settings (exposure and white balance) almost entirely compensated for the darkness, leading to virtually no change in robot success rates.

What Problem Does This Paper Address?

In the world of real-robot benchmarks, researchers often record room illuminance in lux to document lighting conditions. The assumption is that if the lux level is the same, the visual input to the robot's policy is consistent. Conversely, a massive drop in lux is expected to significantly degrade performance, as seen in many simulated benchmarks.

However, this paper challenges that assumption. The authors argue that on typical low-cost robot rigs, the camera's internal processing pipeline—specifically automatic exposure and white balance—acts as a powerful "equalizer." This creates a gap between the physical environment and the digital image, meaning that lux alone does not describe the distribution shift a policy actually faces.

Methods

The researchers used a 6-DoF SO-101 follower arm running the LeRobot framework. They evaluated four widely-used imitation-learning policies:

  • ACT (Action Chunking with Transformers)
  • SmolVLA (a vision-language-action model)
  • Diffusion Policy (DP)
  • $\pi_{0.5}$ (a pretrained VLA model)

All models were trained on the same 50 teleoperated demonstrations of a pick-and-place task (picking a green ball while ignoring red distractors) recorded at ~500 lux.

The "stress test" involved running 80 rollouts: 10 per policy in "Normal" light (~520 lux) and 10 per policy in "Low" light (~45 lux, an 11.5x reduction). Crucially, the cameras (a Logitech Brio 300 and a generic wrist module) were left on automatic settings, mirroring common practice in many labs.

Datasets

  • Training Data: 50 teleoperated demonstrations recorded at ~500 lux.
  • Evaluation Data: 80 new rollouts (40 normal, 40 low-light).
  • Public Release: All rollout datasets, scored logs, and analysis scripts are available on Hugging Face under fysical/rollout_s2smoke_*.

Key Insights and Ideas

  • The Camera Pipeline is a Filter: The most striking finding was that an 11.5x drop in room light only resulted in a 3% drop in mean image brightness for the front camera. The camera's automatic exposure simply "cranked up the gain," delivering images to the policy that looked nearly identical to the training data.
  • Lux is Deceptive: Reporting lux in a paper without reporting camera settings (exposure time, gain, white balance) is scientifically incomplete. The robot's "eye" is not a passive sensor; it's an active processor.
  • Sim vs. Real Discrepancy: In simulation, lighting changes usually modify pixel values directly. In reality, the hardware compensates before the policy ever sees a pixel. This suggests that some "robustness" results in literature might actually be measuring the camera's hardware robustness rather than the policy's algorithmic robustness.
  • Single-Block Noise: An accidental repeat of one 10-rollout block yielded the same success count (7/10) but failed on different ball placements. This highlights that small-sample benchmarks (like 10 rollouts) are extremely noisy and should be interpreted with caution.

Key Results

The quantitative impact of the 11.5x light drop was negligible:

  • Success Rate: 20/40 in normal light vs. 19/40 in low light.
  • Effect Size: A statistically insignificant change of -2.5 percentage points.
  • Image Statistics: While room light fell by over 1000%, the wrist camera images were only 19% darker, and the front camera was practically unchanged.
Figure 1: Success by policy and light level; 80 rollouts, 10 per cell, placements N00–N09.

As shown in the visual comparison below, the images provided to the models in "low light" are barely distinguishable from the "normal" light training data.

Figure 2: First frames of a training episode and of placement N00 in the normal-light and low-light ACT blocks; front camera left, wrist camera right. The low-light frames are barely darker than the others. In normal light the wrist view shows a bright reflection on the floor that does not appear in training or in low light.

Limitations and Future Work

  • Sample Size: 10 rollouts per cell is a small sample, capable of detecting only very large performance drops.
  • Confounding Factors: The low-light tests were conducted on a different day and in a different order, which could introduce minor environmental drifts.
  • Lack of Manual Control: The authors did not explicitly test "locked" exposure settings to see how the policies would fail if the images actually became 11x darker.
  • Scope: The study was limited to one specific task and one hardware rig. Future work should investigate if these findings hold for more complex visual scenes or different sensor types.

The authors conclude with a practical recommendation for the field: Report camera settings, not just lux, and lock exposure if you truly want to test a policy's robustness to lighting.

Summary updated

Abstract

Real-robot benchmarks of learned visuomotor policies often record room illuminance as the lighting condition. We show that, on a typical low-cost rig, room light and image light can be almost unrelated. Four imitation-learning policies (ACT, SmolVLA, Diffusion Policy and pi0.5) were trained on the same 50 demonstrations of a pick-and-place task on an SO-101 arm. Each was run for ten rollouts at about 520 lux and again at about 45 lux, an 11.5x reduction. Success was 20/40 in normal light and 19/40 in low light: a change of -2.5 percentage points (95% CI -23 to +19), with no evidence that the policies differed in sensitivity (interaction p = 0.86). The recorded video shows why: mean image brightness was only 3% lower in low light on the front camera and 19% lower on the wrist camera. Both cameras were set to automatic exposure and white balance, which most likely absorbed the difference; exposure was not locked in this experiment. An accidental repeat of one 10-rollout block reproduced its success count (7/10) but matched the first block at only 6 of 10 placements, consistent with chance. We conclude that lux alone does not describe what a policy sees. Camera settings, and image statistics compared with the training recordings, are the quantities to report and control. Data and code are public.
BibTeX
@misc{cowling2026camera,
  author = {Simon Cowling},
  title = {{Camera equalisation of an 11.5x drop in environmental illuminance: four imitation-learning policies on an SO-101 arm}},
  year = {2026},
  eprint = {2611.13810},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13810},
}
Figure