Boundary Continuity in Concatenative Audio Synthesis

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2223 papers300 conceptsfrozen 2026-10-04

When we stitch two pieces of audio together, the human ear is incredibly sensitive to the "seam." Even if both segments are high-quality, a tiny mismatch in phase, pitch, or spectral energy at the junction creates an audible click or a jarring "robotic" transition. This problem, known as boundary discontinuity, is the central hurdle in concatenative speech synthesis, text-based audio editing, and musical loop generation.

The Geometry of the Seam

In traditional concatenative synthesis, systems build sentences by selecting recorded units—syllables, phones, or diphones—from a database. As Wael Hamza and Raimo Bakis noted in On building a concatenative speech synthesis system from the blizzard challenge speech databases, small datasets exacerbate this because the system is forced to use short, isolated units. Without long contiguous segments, the natural prosodic flow is broken.

The resulting artifacts aren't just volume jumps. They manifest as:

  • Phase Discontinuities: Abrupt shifts in the waveform's cycle that cause clicks.
  • Spectral Mismatches: Sudden changes in timbre (MFCCs) that make the voice sound like a patchwork.
  • Prosodic Glitches: Unnatural jumps in pitch ($F_0$) or duration that break the melodic arc of a sentence.

From Cross-fading to Contextual Bridging

Early solutions relied on signal processing to "soften" the blow. In 基於時域上基週同步疊加法之歌聲合成系統 (Singing Voice Synthesis System Based on Time Domain-Pitch Synchronized Overlap and Add), Ming-Kuan Wu used the TD-PSOLA algorithm to manipulate pitch and apply linear tapering at boundaries. While this reduces clicks, it doesn't solve the underlying lack of "legato"—the smooth, connected flow of natural speech.

Modern research has shifted from hiding the seam to predicting the transition. Instead of treating an edit as a cut-and-paste operation, researchers like Lichao Zhang and Zhou Zhao in EditSinger: Zero-Shot Text-Based Singing Voice Editing System with Diverse Prosody Modeling treat the edit boundary as a gap to be "inpainted." By re-predicting acoustic features at the join points based on the surrounding musical context, the model ensures the pitch and rhythm flow naturally across the gap.

Enforcing Acoustic Consistency

To make these transitions perceptually seamless, models now incorporate explicit "consistency" constraints during training. Rui Liu and Haizhou Li introduced the FluentEditor framework, which uses two specific loss functions:

  1. Acoustic Consistency: Minimizing the variance difference at concatenation points to ensure a smooth "spectral handshake."
  2. Prosody Consistency: Ensuring the global style and emotional "vibe" of the inserted segment matches the original recording.

This move toward "entire inference"—regenerating or conditioning on the whole sentence rather than just the edited snippet—is a recurring theme. Cheng Yu and colleagues demonstrated in Cross-utterance Conditioned Coherent Speech Editing that by using a cross-utterance conditioned VAE, the system can maintain prosodic coherence even when the mood of the surrounding text changes.

Beyond Speech: Music and Textures

The challenge of boundary continuity extends to any sequential signal generation. In music, Larry Heck and Mason Bretan adapted unit selection for music generation, finding that maintaining stylistic relevance at the "seams" of musical measures is just as difficult as in speech. Similarly, Axel Roebel showed in A Two Level Montage Approach to Sound Texture Synthesis that stitching noisy textures like rain or fire requires careful phase management to avoid "swooshing" artifacts.

Even in non-audio domains, like the synthesis of ECG signals, Huaiyu Zhu found that piecewise synthesis methods require spline interpolation to prevent amplitude discontinuities at the boundaries.

Why It Works: The Intuition of Overlap

The most effective modern strategy for continuity is the move from discrete blocks to overlapping patches. As explored in DEX-TTS by Hyun Joon Park, using overlapping patches in transformer-based architectures allows the model to "see" across the boundary during the generation process. This ensures that the end of one segment is mathematically aware of the beginning of the next, effectively dissolving the seam before the audio is even rendered.

Papers in this version

Showing 60 of 223 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 209 papers
  2. Version 1 · 2026-10-04 · 223 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_acoustic_boundary_discontinuity_67f7d687,
  title = {acoustic boundary discontinuity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/acoustic-boundary-discontinuity/rdr_67f7d687d1d148ad955762165bb2c345},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_boundary_continuity_in_concatenative_aud_67f7d687_v1,
  title = {Boundary Continuity in Concatenative Audio Synthesis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/boundary-continuity-in-concatenative-audio-synthesis/rdr_67f7d687d1d148ad955762165bb2c345/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}