Studies of how well soil amendments lock up toxic metals depend on a slow wet-chemistry routine called sequential extraction. Soil is shaken with a series of reagents, centrifuged, and the clear liquid above the settled soil is drawn off, once for each chemical "fraction" of the metal. Robots could take over this work, but they have to see inside glass and plastic tubes, and depth cameras are close to blind to transparent material. The liquids in this workflow make it harder still. They are turbid, brown, stratified, and the tube walls get stained with soil. We propose that the extraction protocol can supply its own training signal. Every time a pipette with liquid-level sensing enters a tube, and every time a tube is weighed, the robot receives an exact, free measurement of where the liquid surface sits. We would use these measurements to keep adapting a transparent-vessel depth model while it works. Adaptation would then track each new soil, reagent colour, and lighting condition without anyone collecting labels.
This matters for a reason the remediation literature keeps stating. Laboratory results on metal immobilization often fail to carry over to the field. Closing that gap takes many samples across seasons, sites, and years, as in the five-year field trial of oyster shell powder in red soils and the site-scale challenges surveyed in the mining and metallurgical sites preface. Sample throughput is a practical bottleneck. The perception problem is itself a small lab-to-field gap: models trained on clean glassware meet dirty, coloured, sunlit vessels.
Core Claim
Transparent-object depth models trained on clean household glassware will degrade sharply on soil-extraction vessels. The failures that matter are the meniscus (the curved liquid surface) and the boundary between the settled soil pellet and the liquid above it. We claim that adaptation driven by incidental measurements from the protocol recovers most of that loss. It does better than offline domain randomization, and it does better than deliberate probing when the number of labels is the same. The payoff is chemical, not only geometric. Better estimates of the interface should reduce how much pellet material the robot draws off with the supernatant. They should also bring the robot's fraction-by-fraction metal recoveries into agreement with careful manual extraction. The closest prior work, TransTouch, fine-tunes a stereo network with 100 deliberate touches chosen by an uncertainty score, on clean empty glassware in the lab. We differ in three ways. Our supervision costs no extra robot actions, it arrives continuously during real work rather than in a calibration phase, and it targets liquid and sediment interfaces rather than vessel surfaces.
Why the Obvious Pieces Fall Short
Each existing line of depth completion for transparent objects fails in one of two ways on this problem: it breaks under uncontrolled optics, or it is too heavy to run on the robot. ClearGrasp recovers shape from predicted surface directions and edges. It reports that bright directional light and caustics (bright light patterns focused by curved glass) are mistaken for object boundaries. FDCT is a 0.39-million-parameter network running at 70 frames per second, which makes it practical, but its authors say it degrades under extreme lighting and still suffers a gap between simulated and real data. The multi-view method of Zhou et al. is the most accurate on cluttered scenes. Its authors note that the memory and latency of its 3D matching volumes block deployment on edge hardware. All three were trained and tested on empty or clean-liquid household objects. None of them models a pellet under turbid liquid.
RoboChemist motivates the approach from the other direction. It avoids the depth problem altogether: a vision-language model marks 2D grasp points on RGB images, and a learned policy executes motions. That works for pouring and stirring on benchtop chemistry. It does not provide the metric height in millimetres that pipetting above a pellet requires, and it was never tested on soil chemistry or outside controlled lab lighting.
The hazard-aware benchmarking of WildDash supplies our evaluation method. That work organizes failures into named hazard clusters and found overexposure to be the most damaging one. A segmentation model scoring about 64 mean intersection-over-union (mIoU) on Cityscapes fell to 16.5 on WildDash. We adopt its approach of building a taxonomy of hazards and testing each separately.
Research Program
The system has four parts. The first is a lightweight depth-completion backbone, initialized from FDCT, trained first on the TransCG and ClearGrasp data. The second is a small head that predicts two heights per tube, measured along the tube axis: the meniscus height and the pellet-surface height. Tube pose comes from racks with known geometry, so the vessel shape is known and depth errors convert directly into height errors. The third is a stream of free labels from ordinary protocol actions. These come from three sources. Capacitive liquid-level detection on the pipette tip gives the meniscus height at entry. Pressure changes during aspiration give the pellet height when the tip approaches it. Gravimetric weighings, which the protocol already requires, convert through the known vessel geometry into liquid volume and hence meniscus height. The fourth is an online update of the height head and the last backbone stages, regularized to avoid forgetting, following TransTouch's confidence-based regularizer.
The update minimizes a simple objective:
$$ \mathcal{L} = \sum_{i \in \mathcal{S}} w_i \,\big(\hat h_i - h_i\big)^2 + \lambda \,\big\lVert \hat D - D_0 \big\rVert_{1,\,\bar{\mathcal{V}}} $$
The terms are defined as follows. $\mathcal{S}$ is the set of protocol measurements collected so far. $\hat h_i$ and $h_i$ are the predicted and measured interface heights. $w_i$ is a reliability weight per sensor type, with gravimetric labels trusted most and pressure-contact labels least. The second term keeps the adapted depth map $\hat D$ close to the pre-adaptation map $D_0$ on pixels outside vessels ($\bar{\mathcal{V}}$), and $\lambda$ sets how strongly.
The robot uses the predicted pellet height, with a safety margin that scales with the model's uncertainty, to decide how deep the tip may descend. The perception error therefore feeds straight into the chemistry result.
We would build a hazard benchmark on the WildDash pattern with clusters specific to this workflow:
- turbid supernatant;
- coloured extracts, such as the brown liquids from the organic-matter-bound step;
- soil film and staining on tube walls;
- condensation;
- densely packed racks with overlapping tubes;
- direct sunlight and caustics, from runs of a mobile extraction bench at an outdoor contaminated site.
Ground-truth heights come from calibrated side-view imaging against a backlit scale. For a held-out subset, liquid volumes are also found by weighing.
Baselines
The naive reference point is raw depth from a commercial structured-light camera with no completion, plus a blind descent that relies only on the pipette's liquid-level sensor and a fixed pellet offset taken from the nominal soil mass. TransTouch reports that raw RealSense depth is worse than every learned method. Blind descent is what a standard liquid handler does, so it is the practical status quo.
The strongest existing depth methods are FDCT and Zhou et al. On TransCG, Zhou et al. report a mean absolute error (MAE) of 0.006 m against 0.010 m for FDCT (Transparent Object Depth Completion). We will run both as released and also fine-tuned on the same labelled budget of our vessel images. Zhou et al. is allowed multiple views at whatever latency it needs, so we are not handicapping it. We report its latency separately.
The probing baseline is TransTouch-style active adaptation. It uses the same number of labels as our method, but those labels come from deliberate touches selected by uncertainty, and each touch costs robot time. TransTouch reports a transparent-object depth error of 17.5 mm falling to 15.5 mm. The comparison asks whether our free, protocol-driven labels match deliberate ones.
The 2D grounding baseline follows RoboChemist: a vision-language model marks the meniscus and pellet in the image, and a fixed per-rack calibration maps those marks to height. No reported result exists on Lacuna for any of these methods on liquid or sediment interfaces in soil extracts.
Two ablations isolate our key ingredient. The first uses the same number of labels but draws them at random times, rather than from the current soil and lighting, which separates "fresh labels" from "more labels." The second is offline domain randomization with rendered turbid and coloured liquids and no online updates.
Beating the baselines means two things across the hazard clusters. Interface-height MAE must be lower than that of the best fine-tuned static model, with the gain concentrated in the clusters where static models collapse. The downstream chemical metrics below must also be at least as good as those from a skilled human operator.
Why not just…?
Why not just use fixed racks and the pipette's own liquid sensor, with no vision at all? This is the most serious threat. For the meniscus alone it may be enough. The liquid-level sensor does not see the pellet before contact, though, and descending until the pressure changes risks disturbing the pellet, which is exactly what carries residue into later fractions. Blind descent is our naive baseline. If it matches our method on carryover, the honest conclusion is that vision is unnecessary indoors, and the contribution narrows to field conditions and the benchmark. We would report that outcome.
Why not just bypass depth with a vision-language model, as RoboChemist does? RoboChemist's 2D marks worked for grasp points under lab lighting. A millimetre-scale height behind refracting, stained glass is a different quantity. Glare, flagged by WildDash as the most damaging hazard and by ClearGrasp and FDCT as a failure mode, will hit a model working only from RGB images as well. The 2D grounding baseline tests this directly.
Why not just render turbid liquids and fine-tune once, offline? That is plausible, and it is an ablation. Soils differ in colour and fine-particle content from site to site, and seasons change the lighting. The canna-biochar pot study and the hydrochar column study both show chemistry shifting across matrices and conditions in ways that lab designs did not anticipate. We expect the visual appearance of extracts to shift in a similar way that a fixed renderer cannot cover. If offline randomization matches online adaptation across the hazard clusters, we drop the online component.
Proposed Experiments
First, a no-regression check. The adapted model must stay close to its starting MAE on TransCG, ClearGrasp, and ClearPose. This confirms that adaptation does not damage general transparent-object depth.
Second, the hazard benchmark. We report interface-height MAE per cluster for every baseline. We also plot error against the number of protocol labels collected, to show how quickly adaptation recovers after a change of soil or of site lighting.
Third, and decisively, the chemistry test. We use contaminated soils homogenized and split into replicates, some untreated and some amended with alkaline or biochar materials of the kinds studied above. Robot and manual operators each run the modified BCR sequential extraction (a standard four-step procedure from the Community Bureau of Reference). We measure four outcomes:
- the mass of residue carried over into each supernatant, found by drying and weighing;
- per-fraction metal recovery relative to manual extraction;
- mass balance, meaning the summed fractions against a total digest;
- the spread between replicates.
The acid-extractable fraction is the one immobilization studies care about most, as in the geopolymer work where lead in that fraction fell below 5% (coal bottom ash geopolymers). It is also the most sensitive to carryover, so it is our headline metric.
The main claim is supported only if three conditions hold. Protocol-supervised adaptation must reduce carryover and bring acid-extractable recoveries within the variation seen between manual replicates. Blind descent and static models must not. The advantage must hold in the outdoor sunlight cluster.
References
- Long-term oyster shell powder applications increase crop yields and control soil acidity and cadmium in red soil drylands
- Preface: Heavy metal(loid)s at mining & metallurgical sites: Fate, risk and remediation
- TransTouch: Learning Transparent Objects Depth Sensing Through Sparse Touches
- Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation
- FDCT: Fast Depth Completion for Transparent Objects
- Transparent Object Depth Completion
- RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation
- WildDash - Creating Hazard-Aware Benchmarks
- Synergistic application of canna-derived biochar and arbuscular mycorrhizal fungi enhances cadmium immobilization and soil ecological function in contaminated soils
- Hydrochar did not reduce rice paddy NH3 volatilization compared to pyrochar in a soil column experiment
- Barrier effect of coal bottom ash-based geopolymers on soil contaminated by heavy metals