0% Complete
Home
/
14th International Conference on Computer and Knowledge Engineering
Leveraging Self-Supervised Models for Automatic Whispered Speech Recognition
Authors :
Aref Farhadipour
1
Homa Asadi
2
Volker Dellwo
3
1- University of Tehran
2- university of isfahan
3- Zurich University of Applied Sciences
Keywords :
Automatic Speech Recognition،whisper speech to text،self-supervised learning،speech processing،deep learning،transformers،Wavlm model،Dialect variation،Whisper model
Abstract :
In automatic speech recognition, any factor that alters the acoustic properties of speech can pose a challenge to the system's performance. This paper presents a novel approach for automatic whispered speech recognition in the Irish dialect using the self-supervised WavLM model. Conventional automatic speech recognition systems often fail to accurately recognise whispered speech due to its distinct acoustic properties and the scarcity of relevant training data. To address this challenge, we utilized a pre-trained WavLM model, fine-tuned with a combination of whispered and normal speech data from the wTIMIT and CHAINS datasets, which include the English language in Singaporean and Irish dialects, respectively. Our baseline evaluation with the OpenAI Whisper model highlighted its limitations, achieving a Word Error Rate (WER) of 18.8% and a Character Error Rate (CER) of 4.24% on whispered speech. In contrast, the proposed WavLM-based system significantly improved performance, achieving a WER of 9.22% and a CER of 2.59%. These results demonstrate the efficacy of our approach in recognising whispered speech and underscore the importance of tailored acoustic modeling for robust automatic speech recognition systems. This study provides valuable insights into developing effective automatic speech recognition solutions for challenging speech affected by whisper and dialect. The source codes for this paper are freely available.
Papers List
List of archived papers
The process of multi class fake news dataset generation
Sajjad Rezaei - Mohsen Kahani - Behshid Behkamal
FarCQA: A Farsi Community Dataset for Question Classification and Answer Selection
Saba Emami - Maedeh Mosharraf
Distinguishing Abstracts of Human-Written and ChatGPT-Generated Papers in the Field of Computer Science
Mohsen Arzani - Hamed Vahdat-Nejad - Matin Hossein-Pour
Improving ADHD Detection with Cost-Sensitive LightGBM
Behnam Yousefimehr - Mehdi Ghatee - Ali Heydari
Adaptive Multi-Scale Attentional Network for Semantic Segmentation of Remote Sensing Images
Melika Zare - Sattar Hashemi
Real-Time Forecasting Using Mixed Frequency Time-Series Data
Armin Khayati - Mohammad Taheri - Koorush Ziarati
Pruning and Mixed Precision Techniques for Accelerating Neural Network
Mahsa Zahedi - Mohammad Sediq Abazari Bozhgani - Abdorreza Savadi
Pyramid Transformer for Traffic Sign Detection
Omid Nejati manzari - Amin Boudesh - Shahriar B. Shokouhi
AVID: A VARIATIONAL INFERENCE DELIBERATION FOR META-LEARNING
Alireza Javaheri - Arsham Gholamzadeh Khoee - Saeed Reza Kheradpisheh - Hadi Farahani - Mohammad Ganjtabesh
An intelligent linguistic error detection approach to automated diagnosis of Dyslexia disorder in Persian speaking children
Fatemeh Asghari - Mahsa Khorasani - Mohsen Kahani - Seyed Amir Amin Yazdi - Mahdi Arkhodi Ghalenoei
more
Samin Hamayesh - Version 41.7.6