CPGFed: Cross-granular and Personalized Prompt-based Green Federated Tuning for Giant Models

Fei Gao, Yunfeng Zhao, Chao Qiu, Xiaofei Wang, Haipeng Yao, Qinghua HuIWQoS 2024 · OpenReview

Abstract

Giant models have transformed vision-language tasks by mastering consistent representations across text and images, highlighting the critical role of deploying such models in the expanding domain of edge scenarios, such as monitoring and segmentation. However, the deployment is challenged by device heterogeneity, limited computational resources, and privacy concerns. Federated learning (FL) presents itself as a viable solution, facilitating decentralized training on devices and preserving data confidentiality. Despite its potential, FL faces obstacles with fine-tuning efficiency, including complex granularity data, static personalized prompt generation, and high energy consumption. This paper introduces a cross-granular and personalized prompt-based green federated tuning (CP2GFed) approach, aiming to address these issues by enabling giant model deployment on devices. CP2GFed introduces a cross-granularity knowledge transfer mechanism to leverage semantic relationships across varying data granularities. Meanwhile, it pioneers in generating dynamic personalized prompts based on inter-device affinities to improve model performance. In addition, CP2GFed meticulously optimizes energy consumption, including model local learning and interaction, by setting local computing steps and selecting communication devices. Empirical results indicate that CP2GFed elevates accuracy by up to 6.64% on diverse datasets and reduces energy consumption by nearly 60% per unit of accuracy, achieving a superior tradeoff between model performance and energy consumption compared to baselines.
BibTeX
@inproceedings{gao2024cpgfed,
  author = {Fei Gao and Yunfeng Zhao and Chao Qiu and Xiaofei Wang and Haipeng Yao and Qinghua Hu},
  title = {{CPGFed: Cross-granular and Personalized Prompt-based Green Federated Tuning for Giant Models}},
  booktitle = {IWQoS},
  year = {2024},
  url = {https://openreview.net/forum?id=o0ZmV6b3Y9},
}

Loading related content...

Figure