No article text was recorded for this version.
entropy-based segmentation
Version 1 of 149 papers66 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Unsupervised Segmentation of Chinese Text by Use of Branching Entropy
- Blog: Automatic Syllabification for Manipuri language
- Blog: Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
- Blog: Neural Networks Incorporating Unlabeled and Partially-labeled Data for Cross-domain Chinese Word Segmentation
- Blog: Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Models
- Blog: Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments
- Blog: Phrase Table Training for Precision and Recall: What Makes a Good Phrase and a Good Phrase Pair?
- Blog: Retrieving Collocations by Co-occurrences and Word Order Constraints
- Blog: BabyLM’s First Words: Word Segmentation as a Phonological Probing Task
- Blog: UNDERTOW: Multi-Level Segmentation of Real-Valued Time Series
- Blog: Session Boundary Detection for Association Rule Learning Using n-Gram Language Models
- Blog: Entropy-Guided Dynamic Tokens for Graph-LLM Alignment in Molecular Understanding
- Blog: Entropy-Driven Pre-Tokenization for Byte-Pair Encoding
- Blog: Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception
- Blog: Hierarchical Voting Experts: An Unsupervised Algorithm for Segmenting Hierarchically Structured Sequences
- Blog: Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings
- Blog: Segment-Aligned Policy Optimization for Multi-Modal Reasoning
- Blog: Can MDL Improve Unsupervised Chinese Word Segmentation
- Blog: Uncertainty-Aware Decision Transformer for Stochastic Driving Environments
- Blog: The Role of Prosody and Speech Register in Word Segmentation: A Computational Modelling Perspective
- Blog: An Efficient Algorithm for Unsupervised Word Segmentation with Branching Entropy and MDL
- Blog: AS-ES Learning: Towards efficient CoT learning in small models
- Blog: Relating Unsupervised Word Segmentation to Reported Vocabulary Acquisition
- Blog: Byte Latent Transformer: Patches Scale Better Than Tokens
- Blog: Unsupervized Word Segmentation: the Case for Mandarin Chinese
- Blog: Segmentation et induction de lexique non-supervisées du mandarin (Unsupervised segmentation and induction of mandarin lexicon)
- Blog: Information-Theoretic Segmentation of Natural Language
- Blog: Sensei: Self-Supervised Sensor Name Segmentation
- Blog: AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise
- Blog: Efficient Transformers with Dynamic Token Pooling
- Blog: Statistical Acquisition of Terminology Dictionary
- Blog: Efficient Transformers with Dynamic Token Pooling
- Blog: ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer
- Blog: Can MDL Improve Unsupervised Chinese Word Segmentation?
- Blog: Building A Chinese Text Summarizer with Phrasal Chunks and Domain Knowledge
- Blog: Language Identification in Unknown Signals
- Blog: SJD-SV: Speculative Jacobi Decoding with Semantics Verification for Autoregressive Image Generation
- Blog: Parsing a Natural Language Using Mutual Information Statistics
- Blog: ByteSpan: Information-Driven Subword Tokenisation
- Blog: Rethinking Code Complexity Through the Lens of Large Language Models
- Blog: The Automatic Extraction of Open Compounds from Text Corpora
- Blog: AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence
- Blog: On the Word Boundaries of Emergent Languages Based on Harris's Articulation Scheme
- Blog: Context-level Language Modeling by Learning Predictive Context Embeddings
- Blog: Greek Word Segmentation Using Minimal Information
- Blog: Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
- Blog: A Comparison of Statistical and Rule-Based Models of Melodic Segmentation
- Blog: Fully Unsupervised Word Segmentation with BVE and MDL
- Blog: Unsupervised Tokenization Learning
History & citation
Current · Permanent link
- Sequence Segmentation via Local Maxima of Predictive Entropy merged into this direction (75% overlap)
- Segmentation through Predictive Uncertainty merged into this direction (66% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_entropy_based_segmentation_db9850be,
title = {entropy-based segmentation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/entropy-based-segmentation/rdr_db9850beb8774aadb387652f2a7a8707},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_entropy_based_segmentation_db9850be_v1,
title = {entropy-based segmentation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/entropy-based-segmentation/rdr_db9850beb8774aadb387652f2a7a8707/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}