PodcastFillers
Lists
Only you see this
Abstract
<strong>OVERVIEW:</strong><br> The PodcastFillers dataset consists of 199 full-length podcast episodes in English with manually annotated filler words and automatically generated transcripts. The podcast audio recordings, sourced from SoundCloud (www.soundcloud.com), are CC-licensed, gender-balanced, and total 145 hours of audio from over 350 speakers. The annotations are provided under a non-commercial license and consist of 85,803 manually annotated audio events including approximately 35,000 filler words (“uh” and “um”) and 50,000 non-filler events such as breaths, music, laughter, repeated words, and noise. The annotated events are also provided as pre-processed 1-second audio clips. The dataset also includes automatically generated speech transcripts from a speech-to-text system. A detailed description is provided below. The PodcastFillers dataset homepage: <strong>PodcastFillers.github.io</strong><br> The preprocessing utility functions and code repository for reproducing our experimental results: <strong>PodcastFillersUtils</strong> <strong>LICENSE:</strong> The PodcastFillers dataset has separate licenses for the audio data and for the metadata. The metadata includes all annotations, speech-to-text transcriptions, and model outputs including VAD activations and FillerNet classification predictions. <strong>Note:</strong> <strong>PodcastFillers is provided for research purposes only. </strong>The metadata license prohibits commercial use, which in turn prohibits deploying technology developed using the PodcastFillers metadata (such as the CSV annotations or audio clips extracted based on these annotations) in commercial applications. ## License for PodcastFillers Dataset metadata This license agreement (the “License”) between Adobe Inc., having a place of business at 345 Park Avenue, San Jose, California 95110-2704 (“Adobe”), and you, the individual or entity exercising rights under this License (“you” or “your”), sets forth the terms for your use of certain research materials that are owned by Adobe (the “Licensed Materials”). By exercising rights under this License, you accept and agree to be bound by its terms. If you are exercising rights under this License on behalf of an entity, then “you” means you and such entity, and you (personally) represent and warrant that you (personally) have all necessary authority to bind that entity to the terms of this License. 1. GRANT OF LICENSE.<br> 1.1 Adobe grants you a nonexclusive, worldwide, royalty-free, revocable, fully paid license to (A) reproduce, use, modify, and publicly display the Licensed Materials for noncommercial research purposes only; and (B) redistribute the Licensed Materials, and modifications or derivative works thereof, for noncommercial research purposes only, provided that you give recipients a copy of this License upon redistribution.<br> 1.2 You may add your own copyright statement to your modifications and/or provide additional or different license terms for use, reproduction, modification, public display, and redistribution of your modifications and derivative works, provided that such license terms limit the use, reproduction, modification, public display, and redistribution of such modifications and derivative works to noncommercial research purposes only.<br> 1.3 For purposes of this License, noncommercial research purposes include academic research and teaching only. Noncommercial research purposes do not include commercial licensing or distribution, development of commercial products, or any other activity that results in commercial gain.<br> 2. OWNERSHIP AND ATTRIBUTION. Adobe and its licensors own all right, title, and interest in the Licensed Materials. You must retain all copyright notices and/or disclaimers in the Licensed Materials.<br> 3. DISCLAIMER OF WARRANTIES. THE LICENSED MATERIALS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND. THE ENTIRE RISK AS TO THE USE, RESULTS, AND PERFORMANCE OF THE LICENSED MATERIALS IS ASSUMED BY YOU. ADOBE DISCLAIMS ALL WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, WITH REGARD TO YOUR USE OF THE LICENSED MATERIALS, INCLUDING, BUT NOT LIMITED TO, NONINFRINGEMENT OF THIRD-PARTY RIGHTS.<br> 4. LIMITATION OF LIABILITY. IN NO EVENT WILL ADOBE BE LIABLE FOR ANY ACTUAL, INCIDENTAL, SPECIAL OR CONSEQUENTIAL DAMAGES, INCLUDING WITHOUT LIMITATION, LOSS OF PROFITS OR OTHER COMMERCIAL LOSS, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE LICENSED MATERIALS, EVEN IF ADOBE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.<br> 5. TERM AND TERMINATION. <br> 5.1 The License is effective upon acceptance by you and will remain in effect unless terminated earlier in accordance with Section 5.2.<br> 5.2 Any breach of any material provision of this License will automatically terminate the rights granted herein.<br> 5.3 Sections 2 (Ownership and Attribution), 3 (Disclaimer of Warranties), 4 (Limitation of Liability) will survive termination of this License.<br> ## License for PodcastFillers Dataset audio files All of the podcast episode audio files come from SoundCloud. Please see podcast_episode_license.csv (included in the dataset) for a detailed license info for each episode. They include CC-BY-3.0, CC-BY-SA 3.0 and CC-BY-ND-3.0 licenses.<br> <strong>ACKNOWLEDGEMENT:</strong><br> Please cite the following paper in work that makes use of this dataset: Filler Word Detection and Classification: A Dataset and Benchmark<br> Ge Zhu, Juan-Pablo Caceres and Justin Salamon<br> In 23rd Annual Cong. of the Int. Speech Communication Association (INTERSPEECH), Incheon, Korea, Sep. 2022. Bibtex <pre><code>@inproceedings{Zhu:FillerWords:INTERSPEECH:22, title = {Filler Word Detection and Classification: A Dataset and Benchmark}, booktitle = {23rd Annual Cong.~of the Int.~Speech Communication Association (INTERSPEECH)}, address = {Incheon, Korea}, month = {Sep.}, url = {https://arxiv.org/abs/2203.15135}, author = {Zhu, Ge and Caceres, Juan-Pablo and Salamon, Justin}, year = {2022}, }</code></pre> <strong>ANNOTATIONS:</strong><br> The annotations include 85,803 manually annotated audio events covering common English filler-word and non-filler-word events. We also provide automatically-generated speech transcripts from a speech-to-text system, which do not contain the manually annotated events.<br> Full label vocabulary<br> Each of the 85,803 manually annotated events is labeled as one of 5 filler classes or 8 non-filler classes (label: number of events). Fillers<br> - Uh: 17,907<br> - Um: 17,078<br> - You know: 668<br> - Other: 315<br> - Like: 157 Non-fillers<br> - Words: 12,709<br> - Repetitions: 9,024<br> - Breath: 8,288<br> - Laughter: 6,623<br> - Music : 5,060<br> - Agree (agreement sounds, e.g., “mm-hmm”, “ah-ha”): 3,755<br> - Noise : 2,735<br> - Overlap (overlapping speakers): 1,484 Total: 85,803 <br> Consolidated label vocabulary<br> 76,689 of the audio events are also labeled with a smaller, consolidated vocabulary with 6 classes. The consolidated vocabulary was obtained by removing classes with less than 5,000 annotations (like, you know, other, agreement sounds, overlapping speakers, noise), and grouping “repetitions” and “words” into “words”. - Words: 21,733<br> - Uh: 17,907<br> - Um: 17,078<br> - Breath: 8,288<br> - Laughter: 6,623<br> - Music : 5,060 - Total: 76,689 The consolidated vocabulary was used to train FillerNet For a detailed description of how the dataset was created, please see our paper.<br> Data Split for Machine Learning:<br> To facilitate machine learning experiments, the audio data in this dataset (full-length recordings and preprocessed 1-sec clips) are pre-arranged into “train”, “validation”, and “test” folders. This split ensures that episodes from the same podcast show are always in the same subset (train, validation, or test), to prevent speaker leakage. We also ensured that each subset in this split remains gender balanced, same as the complete dataset. We strongly recommend using this split in your experiments. It will ensure your results are not inflated due to overfitting, and that they are comparable to the results published in the FillerNet paper <strong>AUDIO FILES:</strong> 1. Full-length podcast episodes (MP3)<br> 199 audio files of the full-length podcast episode recordings in mp3 format, stereo channels, 44.1 kHz sample rate and 32 bit depth. Filename format: [show name]_[episode name].mp3. 2. Pre-processed full-length podcast episodes (WAV)<br> 199 audio files of the full-length podcast episode recordings in wav format, mono channel, 16 kHz sample rate and 32 bit depth. The files are split into train, validation and test partitions (folders), see Data Split for Machine Learning above. Filename format: [show name]_[episode name].wav 3. Pre-processed WAV clips<br> Pre-processed 1-second audio clips of the annotated events, where each clip is centered on the center of the event. For annotated events longer than 1 second, we truncate them from the center into 1-second. The clips are in the same format as the pre-processed full-length podcast episodes: wav format, mono channel, 16 kHz sample rate and 32 bit depth. The clips that have consolidated vocabulary labels (76,689) are split into “train”, “validation” and “test” partitions (folders), see Data Split for Machine Learning above. The remainder of the clips (9,114) are placed in an “extra” folder. Filename format: [pfID].wav where: [pfID] = the PodcastFillers ID of the audio clip (see metadata below) <strong>METADATA:</strong> 1. Speech-to-text podcasts transcripts<br> Speech transcript in JSON format for each podcast episode. Generated using the SpeechMatics STT Filename format: [show name]_[episode name].json. Each word in the transcript is annotated as a dictionary:<br> {“confidence”:(float), “duration”:(int), “offset”:(int), “text”:(string)}<br> where “confidence” indicates the STT confidence in the prediction, “duration” (unit:microsecond or 1e-6 second) is the duration of the transcribed word, “offset” (u
BibTeX
@inproceedings{zhu2022podcastfillers,
author = {Ge Zhu and Juan-Pablo Cáceres and Justin Salamon},
title = {{PodcastFillers}},
booktitle = {Zenodo},
year = {2022},
doi = {10.5281/zenodo.7121457},
}Loading related content...