Superwideband extension of g.718 and g.729.1 speech codecs

Lasse Laaksonen, Mikko Tammi, Vladimir Malenovsky, Tommy Vaillancourt, Mi Suk Lee, Tomofumi Yamanashi, Masahiro Oshikiri, Claude Lamblin, Balazs Kovesi, Lei Miao, Deming Zhang, Jon Gibbs, Holly FrancoisInterspeech 2010

TL;DR

This paper introduces new superwideband (SWB) extensions for the G.718 and G.729.1 speech codecs, pushing the audio bandwidth to 14 kHz. These extensions leverage embedded scalable layers and a novel two-mode coding approach for high frequencies, significantly improving speech quality at higher bitrates.

What Problem Does This Paper Address?

The existing G.718 and G.729.1 codecs provide wideband (WB) audio, typically up to around 7 kHz. However, to achieve truly natural and immersive speech, especially for applications like teleconferencing, voice over IP, and high-fidelity audio, extending the bandwidth further into the superwideband (SWB) range (up to 14 kHz) is desirable. This extension captures more of the subtle nuances and spectral information in human speech, leading to a richer and more intelligible listening experience. The challenge lies in efficiently encoding this expanded frequency range without a prohibitive increase in bit rate or computational complexity, while maintaining backward compatibility with existing WB codecs.

Methods

The core innovation is the development of SWB extensions for G.718 and G.729.1, standardized as G.718 Annex B and G.729.1 Annex E, respectively. These extensions are built upon the existing WB codecs by adding new scalable layers. A key technology is a two-mode SWB coding method for the high frequencies (7 kHz to 14 kHz). This method employs a tonality estimation algorithm to dynamically switch between a "Generic mode" and a "Sinusoidal mode".

  • Tonality Estimation: This process analyzes the spectral characteristics of the input signal to determine if it's predominantly tonal (like a musical note) or more complex. It involves spectral floor calculation, smoothing, and peak-by-peak correlation analysis between consecutive log-energy spectra.
  • Generic Mode: Used for non-tonal signals, this mode divides the high-frequency band into subbands. It then searches for the most similar spectral match from the already coded low-frequency content and scales it to synthesize the high frequencies. This is achieved through a two-step scaling process to optimize perceptual match.
  • Sinusoidal Mode: Employed for tonal signals, this mode transmits a set of sinusoidal components to reconstruct the high-frequency content. The positions, signs, and amplitudes of these sinusoids are encoded.
  • Additional Sinusoids: Both extensions add further sinusoidal components in subsequent layers to refine the high-frequency representation.
  • G.729.1 Specific Enhancements: The G.729.1 extension also includes improvements to the lower frequency (LF) content (below 7 kHz) in its higher layers, offering enhanced perceptual quality beyond just extending the bandwidth. It also features an adaptive bit allocation mechanism in one of its layers to balance SWB coding and WB improvements.

The SWB extensions operate on signals sampled at 32 kHz, with an additional algorithmic delay of 6.75 ms.

Datasets

The paper discusses performance evaluation using results from the ITU-T Characterization phase. This involved listening tests conducted by 5 laboratories using the Ref-A-B methodology. The tests were performed on three types of audio: clean speech, noisy reverberant speech, and music. The results presented are extracted from this phase. No new datasets were introduced.

Key Insights and Ideas

  • Scalability is Key: The approach of embedding SWB functionality as scalable layers on top of existing WB codecs is crucial for backward compatibility and flexible bit rate management.
  • Tonality-Adaptive Coding: The dynamic switching between Generic and Sinusoidal modes based on signal tonality is a clever way to optimize coding efficiency and quality for different types of speech signals.
  • Leveraging LF for HF: The Generic mode's reliance on matching and scaling existing LF content for HF synthesis is an efficient way to represent complex spectral information without transmitting it entirely anew.
  • G.729.1's Dual Role: The G.729.1 extension's ability to simultaneously enhance both high frequencies and improve the perceptual quality of the existing wideband range makes it a more comprehensive upgrade.

Key Results

The paper presents listening test results from the ITU-T Characterization phase, indicating strong performance for the SWB extensions.

  • Clean Speech: All clean speech requirements and objectives were passed by both extensions.
  • Noisy Reverberant Speech: The extensions passed most requirements, with only minor deviations reported in one laboratory.
  • Music: The extensions performed well on music, passing most requirements with a few exceptions in one laboratory.
  • Complexity: The worst-case complexity for the G.718 SWB extension is 79 WMOPS (or 70 WMOPS in AMR-WB interoperable mode), and for G.729.1 SWB, it's 61 WMOPS. These figures include both core coding and SWB extensions.

Figures 2 and 3 show comparative performance scores for clean and noisy speech, respectively, with the SWB extensions generally outperforming the reference codec (ITU-T G.722.1C) and achieving high scores.

Limitations and Future Work

  • Music Performance: While generally good, the paper notes some music requirements were not fully met in the listening tests, suggesting potential areas for further refinement in music enhancement.
  • Complexity: Although the complexity figures are provided, the paper doesn't delve deeply into the trade-offs between different coding modes and their specific computational costs.
  • Future Work: The paper concludes by stating that this new SWB extension provides state-of-the-art performance. The standardization itself implies a significant step forward, and future work would likely focus on further optimization, broader deployment, and potentially exploring even higher bandwidths or more advanced features like stereo coding (which was mentioned as a subsequent development phase).

Summary updated

BibTeX
@inproceedings{laaksonen2010superwideband,
  author = {Lasse Laaksonen and Mikko Tammi and Vladimir Malenovsky and Tommy Vaillancourt and Mi Suk Lee and Tomofumi Yamanashi and Masahiro Oshikiri and Claude Lamblin and Balazs Kovesi and Lei Miao and Deming Zhang and Jon Gibbs and Holly Francois},
  title = {{Superwideband extension of g.718 and g.729.1 speech codecs}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure