SPICE: Stochastic Policy for Interaction in Cluttered Environments

Nidhi Parayil, Benjamin Greenland, Thierry Peynot, Chris LehnertarXiv 2026 · arXiv

TL;DR

SPICE is the first tactile-conditioned diffusion policy designed to help robots reach targets deep within dense, visually occluded plant canopies. By combining spatial tactile encoding with generative modeling, the system learns to autonomously decide whether to push through foliage, maneuver around stems, or physically displace obstructing branches, achieving an 87.5% zero-shot success rate on real tomato and lime plants.

What Problem Does This Paper Address?

Automating agricultural tasks like harvesting or pruning is notoriously difficult because crops often grow behind a "wall" of leaves and branches. This environment presents two major hurdles for traditional robotics:

  1. Visual Occlusion: Cameras cannot see through dense foliage, making vision-based navigation unreliable.
  2. Physical Clutter: Contact with the environment is unavoidable. Unlike industrial settings where collisions are failures, agricultural robots must interact with "compliant" (flexible) structures.

Current solutions typically rely on rigid, predefined reactive behaviors or complex mathematical models of plant physics, which are difficult to generalize. There is a significant gap in learning flexible, whole-arm behaviors that can adapt to the intermittent and unpredictable tactile feedback found in a garden or orchard.

Methods

The authors introduce SPICE (Stochastic Policy for Interaction in Cluttered Environments). The core of the system is a Diffusion Policy, a type of generative model that excels at learning complex, multi-modal behaviors from human demonstrations.

Key technical components include:

  • Spatial Tactile Encoding: Instead of just measuring total force, SPICE uses a CNN to process data from a 4x4 array of three-axis tactile sensors (taxels). This preserves the spatial structure of the contact, allowing the robot to distinguish between a branch hitting the tip of its finger versus the side.
  • Explicit Contact Conditioning: Because contact only occurs in about 30% of the training data, the model includes an explicit "contact signal" to help it distinguish between moving through free space and interacting with an object.
  • Vision-Free Input: The policy relies solely on the robot's joint configurations and tactile feedback, making it immune to the visual "blindness" caused by leaves.
  • Closed-Loop Replanning: The model predicts a sequence of 16 future joint positions but only executes the first 8 before recalculating based on new tactile feedback. This allows for fluid adaptation as a branch bends or slips.
The SPICE architecture uses a CNN to encode tactile tensors, which conditions a Diffusion Policy to generate joint-space actions.

Datasets

The researchers collected a custom dataset of 240 human-teleoperated demonstrations using a GELLO setup.

  • Training Environment: A controlled mock plant setup using balsa wood branches and artificial leaves to ensure repeatable conditions.
  • Behaviors Captured: Reaching through foliage, maneuvering around vertical branches, and displacing horizontal branches.
  • Evaluation: The model was tested across 180 controlled trials in the mock setup and 40 zero-shot trials on real, unseen tomato and lime plants.

Key Insights and Ideas

  • Tactile "Vision": The spatial arrangement of tactile sensors is as important as the magnitude of the force. Knowing where a branch is touching the gripper allows the policy to infer the branch's orientation and choose the correct response (e.g., "move left" vs. "push down").
  • Generative vs. Reactive: Traditional reactive controllers often get stuck in local minima (like pushing endlessly against a stiff branch). Because SPICE is a generative diffusion policy, it can learn multi-stage "strategies"—like reorienting the gripper before pushing—that are difficult to hand-code.
  • Simplicity Transfers: Despite being trained on a simplified mock plant, the policy's reliance on local tactile gradients allowed it to transfer to real plants with entirely different leaf shapes and branch stiffness without any additional fine-tuning.
Fig. 2. Contact-dependent behaviours for reaching an occluded target (red): direct motion towards target through compliant foliage, manoeuvring around an obstructing branch, and displacing a branch away from the target path. Blue arrows indicate the end-effector trajectories.

Key Results

  • Maneuvering Success: SPICE achieved a 95% success rate in maneuvering around branches, dwarfing the 45% success rate of the strongest tactile-conditioned baseline.
  • Robustness to Occlusion: While image-conditioned policies failed completely under heavy occlusion, SPICE remained effective, proving that tactile sensing is a viable primary modality for cluttered environments.
  • Real-World Transfer: In zero-shot tests on real plants, SPICE achieved an 87.5% success rate. It successfully navigated the thinner, stiffer branches of lime plants and the denser, more compliant foliage of tomato plants.
A sequence showing the robot sensing a branch (t1), pushing it down (t2), moving over it (t3), and reaching the target (t4).

Limitations and Future Work

  • Sensor Coverage: Currently, tactile sensors are only on the end-effector. The robot is "blind" to contacts on its elbow or upper arm, which can lead to stuck configurations in very deep clutter.
  • Demonstration Bottleneck: The system relies on high-quality human demonstrations. Future work could explore using reinforcement learning to "self-correct" or expand behaviors.
  • Visuotactile Integration: While this paper proves tactile-only reaching is possible, the authors suggest that combining global visual guidance (to get near the target) with local tactile adaptation (to reach through the final layer of clutter) would be the ultimate solution for agricultural robotics.

Summary updated

Abstract

Agricultural tasks such as harvesting and pruning require robots to reach through dense foliage, where contact is often unavoidable and vision is obstructed. Intermittent contact creates an imbalance between free-space and contact-rich observations, complicating the learning of contact-responsive behaviours. We present SPICE, a tactileconditioned diffusion policy for reaching targets in dense plant canopies using tactile sensing as the sole environmental feedback. SPICE exhibits three behaviours: moving through compliant foliage, manoeuvring around partially obstructing stems, and manipulating branches that block the target path. To the best of our knowledge, SPICE is the first tactile-conditioned diffusion policy for target reaching through physically cluttered plant canopies. Across over 180 real-robot trials, SPICE was evaluated against tactile-conditioned, image-conditioned, and reactive-control baselines. Spatial tactile information and explicit contact conditioning proved important, with SPICE achieving 95% branch-manoeuvring success versus 45% for the strongest tactile baseline. SPICE also achieved branch manipulation beyond the reactive controller and remained effective under visual occlusion, where the image-conditioned DP failed. Zero-shot transfer across 40 trials on unseen tomato and lime plants achieved 87.5% success rate. These results demonstrate the potential of tactile-conditioned generative policies for interaction in visually occluded plant environments.
BibTeX
@misc{parayil2026spice,
  author = {Nidhi Parayil and Benjamin Greenland and Thierry Peynot and Chris Lehnert},
  title = {{SPICE: Stochastic Policy for Interaction in Cluttered Environments}},
  year = {2026},
  eprint = {2611.23585},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23585},
}
Figure