Credit Assignment and Fine-Tuning Enhanced Reinforcement Learning for Collaborative Spatial Crowdsourcing

Chao Huang, Weichao Cai, Qiuping Jiang, Zhihua Wang, Jiaqi Wu, Mingliang Xu2024 · DOI

Abstract

Medical image segmentation is critical for AI-assisted diagnosis but suffers from high annotation costs and suboptimal multimodal fusion. We present a novel framework that leverages medical text annotations to enhance image segmentation. By modeling features as distributions to reduce information redundancy and incorporating frequency-domain "prompts" to sharpen boundaries, our method achieves state-of-the-art performance on multiple benchmarks while maintaining computational efficiency.

Introduction

Accurate segmentation of organs and lesions is a cornerstone of medical imaging. However, expert pixel-level labeling is expensive and time-consuming. While multimodal approaches incorporating text descriptions offer a solution, they often struggle with information redundancy or ignore fine-grained modality-specific details. Traditional fusion methods like feature addition or contrastive learning often fail to capture the optimal joint representation, leading to noise or loss of unique modality-specific information.

Conceptual comparison of feature fusion methods: (a) Feature Addition, (b) Contrastive Learning, and (c) the proposed Distribution Learning method which reduces redundancy.

Methodology

The proposed framework consists of three primary components designed to optimize multimodal representation and boundary precision.

Multimodal Feature Distribution Learning

Unlike simple element-wise addition, our approach treats text and vision features as probability distributions ($p$ and $q$). We introduce "learnable features" that act as a bridge in the joint feature space. By minimizing the Kullback-Leibler (KL) divergence between these distributions, the model effectively reduces redundancy while preserving unique, task-relevant information from both modalities. This process "squeezes out" noise, ensuring that the fused representation is both concise and informative.

Frequency Prompt Encoder

To address the "low-frequency bias" common in deep learning models—which often leads to blurry boundaries—we utilize the frequency domain. Using a Fast Fourier Transform (FFT), we extract the phase spectrum of the image. This phase information, which captures structural "blueprints" and edges, is processed through a Prompt Encoder and fed into the decoder as a spatial prompt to help the model identify sharp boundaries more accurately.

The overall architecture of the proposed framework, highlighting the Multi-Modal Feature Fusion and Frequency Representation modules.

System Architecture

The architecture utilizes a frozen CLIP text encoder and a vision encoder. The fused distribution features and frequency-domain prompts are integrated within a cross-modality processor and a U-shaped decoder to produce the final segmentation mask. This design allows the model to leverage pre-trained semantic knowledge while refining spatial details.

Experimental Results

The framework was evaluated on three diverse datasets: MoNuSeg (nuclear segmentation), GlaS (gland segmentation), and MosMedData+ (COVID-19 lung infections).

  • Performance: On MoNuSeg, the model achieved an 80.96% Dice score, outperforming the previous SOTA (LViT) by 0.81%. On GlaS, it reached 91.08%, significantly surpassing other multimodal methods like LAVT and TGANet.
  • Efficiency: The model is remarkably lightweight, using only 29.2M parameters on MoNuSeg, compared to 105M for TransUNet and 91M for SAM, while delivering superior accuracy.

Conclusion

By shifting from simple feature fusion to distribution learning and leveraging frequency-domain structural information, this framework effectively addresses the challenges of redundancy and boundary precision in medical segmentation. The results demonstrate that even simple text descriptions can significantly bolster model performance, especially in data-scarce scenarios, providing a robust path forward for multimodal medical AI.

Summary updated

Abstract

Collaborative spatial crowdsourcing leverages distributed workers' collective intelligence to accomplish spatial tasks. A central challenge is to efficiently assign suitable workers to collaborate on these tasks. Although mainstream reinforcement learning (RL) methods have proven effective in task allocation, they face two key obstacles: delayed reward feedback and non-stationary data distributions, both hindering optimal allocation and collaborative efficiency. To address these limitations, we propose CAFE (credit assignment and fine-tuning enhanced), a novel multi-agent RL framework for spatial crowdsourcing. CAFE introduces a credit assignment mechanism that distributes rewards based on workers' contributions and spatiotemporal constraints, coupled with bi-level meta-optimization to jointly optimize credit assignment and RL policy. To handle non-stationary spatial task distributions, CAFE employs an adaptive fine-tuning procedure that efficiently adjusts credit assignment parameters while preserving collaborative knowledge. Experiments on two real-world datasets validate the effectiveness of our framework, demonstrating superior performance in terms of task completion and equitable reward redistribution.
BibTeX
@misc{huang2024credit,
  author = {Chao Huang and Weichao Cai and Qiuping Jiang and Zhihua Wang and Jiaqi Wu and Mingliang Xu},
  title = {{Credit Assignment and Fine-Tuning Enhanced Reinforcement Learning for Collaborative Spatial Crowdsourcing}},
  year = {2024},
  doi = {10.24963/ijcai.2024/459},
}
Figure