Fast, small object detectors save compute by splitting each convolution into two cheaper steps. The first filters every feature channel spatially on its own. The second mixes channels at each pixel. This split, called a depthwise separable convolution, cuts the cost of a 3×3 layer by roughly 9×. It is the standard way to fit a detector onto embedded hardware next to the operating table. Laparoscopic surgery is close to a worst case for that trade. The instruments are thin metal shafts with small tips. They are often half-hidden by cautery smoke, glare from wet tissue, or blood, and in those frames any single channel may carry almost no contrast. We propose to measure where the accuracy lost to factorization actually concentrates in a single-stage tool detector: which frames, which resolution levels, and which layers. We then test one targeted repair. It is a high-resolution "detail path" at one quarter of the input resolution, built so that channels are mixed before spatial filtering and so that raw detail is admitted only where deeper features ask for it.
Our expectation is specific. Most of the accuracy a lightweight detector gives up on surgical video sits in degraded frames and on thin, small instruments, not in clean frames. A carefully built stride-4 detail path should recover most of that loss at a small fraction of the cost of running dense convolutions at that resolution. We judge the result by detection quality and also by whether it reduces the missed detections that break tool tracking over a full operation.
Core Claim
At matched latency on embedded hardware, a factorized single-stage detector with a gated, mix-first stride-4 detail path should beat two alternatives on degraded laparoscopic frames. The first is a fully factorized detector without that path. The second is a dense detector shrunk to the same budget. The advantage should carry through to tracking metrics on CholecTrack20. The new content is not "a lightweight YOLO for surgery." It is a degradation-stratified account of where factorization hurts, plus a repair whose individual ingredients we can test one at a time: the ordering of channel mixing and spatial filtering, and the gate on injected detail. If a plain dense high-resolution head at matched latency does just as well, we will report that the high-resolution budget is better spent on dense convolutions. That outcome is itself an actionable finding for edge deployment.
What Exists, and What It Leaves Open
The motivating evidence comes from the surgical tracking literature. CholecTrack20 annotates 20 full cholecystectomy videos drawn from Cholec80 and CholecT50 with tool boxes across seven classes. Its YOLOv7 detector reaches 80.6% AP. Bot-SORT tracking on top of that detector reaches 44.7% HOTA when tools are only followed while visible, and falls to 17.4% HOTA over the whole operation. SurgiTrack raises tracking to 67.3% HOTA by adding a learned estimate of tool motion direction and a graph-matching step. Its own analysis still attributes broken tracks to frames where the detector misses a tool. Both works use a heavy detector and never ask what happens when the detector must be small. The two detection numbers above are the reference points for detection quality and tracking.
The method ingredients come from three papers outside surgical video. GB-YOLO swaps YOLOv5's convolutions for Ghost modules, which produce many feature maps through cheap depthwise operations. It reports 128 FPS against 107 FPS for YOLOv5, but only on clean face images. DSD-YOLOv8 shows that a small YOLOv8n collapses in dust and fog: confidence on a foggy example drops to 0.29. Deformable convolutions, a parameter-free attention module and a dynamic head lift that confidence to 0.74. This is the closest analogue to smoke in surgery, and it suggests that small detectors need explicit help in degraded scenes. Mobile U-ViT pairs large-kernel depthwise separable convolutions with downsampled skip connections for medical segmentation. It uses 1.39M parameters and runs at 51.22 FPS on a Jetson Nano. It also supplies two constraints we adopt. Jumping abruptly from small to large kernels hurts, so any kernel growth in our detail path is gradual. Learned strided downsampling can underperform max pooling on noisy medical images. None of these three works does box detection on surgical video.
Two further works show the cost of preserving detail. ToolTipNet keeps parallel high-resolution streams with HRNet-w32 to localize robotic tool tips. It achieves 3.73-pixel error on simulated data and 9.04 pixels on real SurgPose data. However, it needs SAM 2 masks as input and uses dense convolutions at every resolution, so it shows why detail matters without being deployable. DPSSD fuses high- and low-resolution maps in a single-stage detector and still reaches only 20.6% AP on small COCO objects. That result warns that multi-scale fusion alone does not fix small-object loss. Ran et al.'s improved YOLOv7x obtains 91.5% AP on instrument trays using large kernels and dynamic convolutions, but runs at only 6 to 7 images per second. Heavy fixes do not meet the latency target.
Research Program
The base detector is a YOLOv8n-style single-stage network. It predicts boxes and classes densely from a feature pyramid at strides 8, 16 and 32. We build four versions under one training recipe. The dense version uses standard 3×3 convolutions. The factorized version replaces every 3×3 convolution in the backbone and neck with a depthwise separable one. The factorized + P2 version adds a stride-4 pyramid level and prediction head, also made of depthwise separable blocks in the usual order: spatial filtering first, then channel mixing. The proposed version changes that stride-4 path in two ways.
The first change is ordering. Each block applies a 1×1 channel-mixing convolution before the depthwise spatial filter. The spatial filter therefore operates on channels that already combine color and texture evidence. When smoke washes out one input channel, the boundary signal can survive in a mixed channel. The second change is a gate. Deep, low-resolution features predict a per-pixel weight in [0,1], and only the gated part of the stride-4 features is added to the upsampled pyramid. This addresses the noise-leakage problem raised for skip connections, where glare and sensor noise from early layers contaminate the decoder.
The reason the stride-4 level must be cheap is simple arithmetic. For a layer on an $H\times W$ map with $M$ input channels, $N$ output channels and kernel size $K$, dense cost is $HWK^2MN$ and separable cost is $HWM(K^2+N)$. A stride-4 map has four times the positions of a stride-8 map. A dense stride-4 level at the same width therefore costs about as much as the whole stride-8 level four times over. Factorization brings it back to roughly $1/N + 1/K^2$ of that.
The analysis has two parts, and both run on frames grouped by visual condition. The first is a placement study. Starting from the factorized model, we restore dense convolutions in one stage at a time and record how much accuracy each restored stage buys per millisecond of embedded latency. This shows whether factorization damage is concentrated at high resolution, as the per-channel contrast argument predicts. The second part tests the claimed mechanism directly. We apply synthetic smoke and glare at controlled strength to clean test frames. For each model we track how quickly recall of thin instruments (hook, scissors, clipper) falls as degradation increases.
Cholec80 serves as a large secondary test of generalization. It has tool-presence labels but no boxes. We turn per-frame detections into presence scores and evaluate on Cholec80 videos that do not appear in CholecTrack20. This tests generalization across the 13 surgeons in Cholec80 at much larger scale than the 20 box-annotated videos allow.
Baselines
The naive reference point is the off-the-shelf small detector, YOLOv8n with its standard dense convolutions. It is fine-tuned on CholecTrack20 with the same input resolution, augmentation, number of epochs and three seeds as every other model. It is what a practitioner would deploy without this work, and every change is measured against it.
The strongest existing detector on this data is YOLOv7 from CholecTrack20, reported at 80.6% AP on tool boxes. We retrain it under the CholecTrack20 protocol to reproduce that number before making comparisons. Its role is an upper reference for accuracy, not a latency competitor. The goal is for the proposed model to stay within a few AP points of it at a fraction of its embedded latency.
For tracking, the reference is the reported Bot-SORT result on the YOLOv7 detector: 44.7% HOTA when following visible tools, and 17.4% HOTA over the whole operation. We run Bot-SORT unchanged on each detector's output, so differences in tracking come only from detection. SurgiTrack, at 67.3% HOTA, shows what a stronger tracker adds. We report it for context and do not compete with it, since our contribution is the detector.
The most dangerous competitor is a dense detector of matched cost. This is YOLOv8n with a stride-4 head made of standard convolutions, with network width reduced until its latency on the embedded device equals that of the proposed model. A lightweight GB-YOLO-style variant from GB-YOLO, with Ghost modules and BiFPN fusion, is the second competitor of matched cost.
All models are exported with TensorRT at FP16 and at INT8, and latency is measured on the same Jetson-class device at batch size 1. The proposal counts as beating these baselines only if two conditions hold at equal latency. It must improve recall on degraded frames by more than the spread across seeds. It must not lose AP on clean frames.
The ablations that isolate the key ingredients are the proposed model with depthwise-first ordering in the stride-4 path, and the proposed model with the gate removed (plain addition).
Why not just…?
Why not just fine-tune an off-the-shelf small YOLO? This is the default, and it may be adequate on clean frames. The question is degraded frames. In fog and dust, DSD-YOLOv8 found that YOLOv8n's confidence collapsed to 0.29 and that recall improved by 4.1 points only after architectural changes. If stock YOLOv8n matches the proposed model's recall on smoke- and glare-tagged CholecTrack20 frames, the main claim fails, and the naive baseline is there to show it.
Why not just add a stride-4 head, or raise the input resolution, with ordinary convolutions? This is the real threat. A dense stride-4 head is the textbook fix for small objects, and it may simply work. The obstacle is cost. Dense convolutions at a quarter of the input resolution are expensive, and the heavy surgical detector of Ran et al. runs at 6 to 7 images per second. We settle this with the matched-latency dense baseline and with a matched-latency higher-input-resolution baseline. If either equals the proposed model, we report that the extra detail is worth having but factorizing it is not.
Why not just quantize or prune YOLOv7? Quantization shrinks memory and speeds up integer arithmetic, but leaves the architecture unchanged, and it can be applied to any model. For that reason every model, including ours, is also compared at INT8. That way quantization is applied equally rather than offered as an alternative. A YOLOv7 pruned to the target latency is added as an extra baseline. If it outperforms the factorized designs at equal latency, that is a negative result about factorization worth publishing.
Why not just smooth detections over time? The tracker already exploits temporal continuity, yet CholecTrack20 reports a collapse to 17.4% HOTA over full operations. EndoNet shows a drop from 92.0% offline to 81.7% online phase accuracy once future frames are unavailable. Causal smoothing has limited room to work. We still apply the same simple causal smoother of detection scores to every model, so any remaining gain is attributable to better per-frame detection.
Proposed Experiments and Decisive Checks
All box experiments use CholecTrack20 with its official video splits. Frames are grouped by visual condition into clean, smoke, specular glare, blood or fluid, and motion blur. We use the dataset's own condition annotations if the release provides them. Otherwise we label the test videos at 1 fps with two annotators and report their agreement. Metrics are the dataset's AP plus AP50, recall at a fixed precision for each condition group and for thin versus bulky tool classes, and HOTA across the three tracking perspectives with Bot-SORT held fixed. Embedded latency and energy per frame are measured at FP16 and INT8. On held-out Cholec80 videos we report presence mAP derived from detections, with EndoNet's 81.0% tool-presence mAP from EndoNet as a reference. That is a classification model trained on presence labels directly, so the comparison is indicative rather than strict. No reported result was found on Lacuna for any lightweight or depthwise separable detector on CholecTrack20 or Cholec80, nor for any detector's embedded latency on these datasets. Those numbers are produced here.
Three outcomes would decide the question. First, the gap between the factorized and dense models should be several times larger on degraded frames than on clean frames, and the placement study should show the high-resolution stages recovering most of it per millisecond. If the gap is uniform across conditions, the per-channel contrast story is wrong. Second, at matched latency, the proposed model should beat the dense stride-4 baseline on degraded-frame recall for thin instruments. Removing mix-first ordering or the gate should each remove a measurable part of that advantage. If neither ablation matters, the repair reduces to "add a cheap P2 head." Third, the gain in degraded-frame recall should appear as higher HOTA for tracking over the whole operation under the unchanged tracker. If detection improves but tracking does not, then missed detections are not the bottleneck that the tracking papers claim, and that would redirect effort toward association rather than detection.
References
- CholecTrack20: A Multi-Perspective Tracking Dataset for Surgical Tools
- SurgiTrack: Fine-grained multi-class multi-tool tracking in surgical videos
- Application of YOLOv5 for mask detection on IoT
- Object detection algorithm based on improved YOLOv8 for drill pipe on coal mines
- Mobile U-ViT: Revisiting large kernel and U-shaped ViT for efficient medical image segmentation
- ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detection
- DPSSD: Dual-Path Single-Shot Detector
- Surgical Instrument Detection Algorithm Based on Improved YOLOv7x
- EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos