No article text was recorded for this version.
intra-document attention masking
Version 1 of 149 papers60 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Analysing The Impact of Sequence Composition on Language Model Pre-Training
- Blog: Positional Overload: Positional Debiasing and Context Window Extension for Large Language Models Using Set Encoding
- Blog: Attend to the Active: Structure-Aware Dynamic Attention in LLMs for Compositional Instruction Following
- Blog: Enhancing Multi-Image Understanding through Delimiter Token Scaling
- Blog: Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems
- Blog: Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation
- Blog: Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA
- Blog: UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
- Blog: Set-LLM: A Permutation-Invariant LLM
- Blog: A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization
- Blog: Enhancing Vision Transformers for Object Detection via Context-Aware Token Selection and Packing
- Blog: ChuXin: 1.6B Technical Report
- Blog: Towards Understanding Massive Activations in Attention Sink Mechanism
- Blog: SkyLadder: Better and Faster Pretraining via Context Window Scheduling
- Blog: Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment
- Blog: Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation
- Blog: Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment
- Blog: How to Train Long-Context Language Models (Effectively)
- Blog: Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models
- Blog: Hierarchical Transformer for Task Oriented Dialog Systems
- Blog: Enhancing Partially Spoofed Audio Localization with Boundary-aware Attention Mechanism
- Blog: Generalized Parallel Scaling with Interdependent Generations
- Blog: Clinical Trial Retrieval via Multi-grained Similarity Learning
- Blog: Revisiting Data Challenges of Computational Pathology: A Pack-based Multiple Instance Learning Framework
- Blog: UP-DETR: Unsupervised Pre-Training for Object Detection With Transformers
- Blog: A Query-Parallel Machine Reading Comprehension Framework for Low-resource NER
- Blog: FlashMask: Efficient and Rich Mask Extension of FlashAttention
- Blog: Mini Diffuser: Fast Multi-task Diffusion Policy Training Using Two-level Mini-batches
- Blog: CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning
- Blog: Fewer Truncations Improve Language Modeling
- Blog: Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective
- Blog: À-la-carte Prompt Tuning (APT): Combining Distinct Data Via Composable Prompting
- Blog: Native-Resolution Image Synthesis
- Blog: MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extraction
- Blog: When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
- Blog: Parallel Context Windows for Large Language Models
- Blog: Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration
- Blog: Parallel-R1: Towards Parallel Thinking via Reinforcement Learning
- Blog: Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs
- Blog: DuSA: Fast and Accurate Dual-Stage Sparse Attention Mechanism Accelerating Both Training and Inference
- Blog: Visual Para-Thinker: Divide-and-Conquer Reasoning for Visual Comprehension
- Blog: Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
- Blog: End-to-End Segmentation-based News Summarization
- Blog: Dimension-Reduction Attack! Video Generative Models are Experts on Controllable Image Synthesis
- Blog: ElasticAST: An Audio Spectrogram Transformer for All Length and Resolutions
- Blog: Knowledge Amalgamation for Object Detection with Transformers
- Blog: LAIT: Efficient Multi-Segment Encoding in Transformers with Layer-Adjustable Interaction
- Blog: Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization
- Blog: ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs
History & citation
Current · Permanent link
- Document Boundary Isolation in Packed Training Sequences merged into this direction (56% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_intra_document_attention_masking_52d23e30,
title = {intra-document attention masking},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/intra-document-attention-masking/rdr_52d23e30a28140d49c09209cf4704d9b},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_intra_document_attention_masking_52d23e30_v1,
title = {intra-document attention masking},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/intra-document-attention-masking/rdr_52d23e30a28140d49c09209cf4704d9b/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}