Towards Sampling Data Structures for Tensor Products

Zhao Song, Samson ZhouOpenReview 2025 · Submission to ICLR 2025 · Rejected · OpenReview

Abstract

This paper studies the computational challenges of attention-based models in artificial intelligence by introducing innovative sampling methods to accelerate attention computation in large language models (LLM). Inspired by the recent progress of LLM in real-life applications, we introduces a streaming sampler question for attention setting. Our approach significantly reduces the computational burden of traditional attention mechanisms while maintaining or enhancing model performance. We demonstrate these methods' effectiveness from theoretical perspective, including space, update time. Additionally, our framework exhibits scalability and broad applicability across various model architectures and domains.
BibTeX
@misc{song2025towards,
  author = {Zhao Song and Samson Zhou},
  title = {{Towards Sampling Data Structures for Tensor Products}},
  year = {2025},
  url = {https://openreview.net/forum?id=CNPLXcMcSP},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2025},
}

Loading related content...

Figure