0% Complete
Home
/
14th International Conference on Computer and Knowledge Engineering
Leveraging Self-Supervised Models for Automatic Whispered Speech Recognition
Authors :
Aref Farhadipour
1
Homa Asadi
2
Volker Dellwo
3
1- University of Tehran
2- university of isfahan
3- Zurich University of Applied Sciences
Keywords :
Automatic Speech Recognition،whisper speech to text،self-supervised learning،speech processing،deep learning،transformers،Wavlm model،Dialect variation،Whisper model
Abstract :
In automatic speech recognition, any factor that alters the acoustic properties of speech can pose a challenge to the system's performance. This paper presents a novel approach for automatic whispered speech recognition in the Irish dialect using the self-supervised WavLM model. Conventional automatic speech recognition systems often fail to accurately recognise whispered speech due to its distinct acoustic properties and the scarcity of relevant training data. To address this challenge, we utilized a pre-trained WavLM model, fine-tuned with a combination of whispered and normal speech data from the wTIMIT and CHAINS datasets, which include the English language in Singaporean and Irish dialects, respectively. Our baseline evaluation with the OpenAI Whisper model highlighted its limitations, achieving a Word Error Rate (WER) of 18.8% and a Character Error Rate (CER) of 4.24% on whispered speech. In contrast, the proposed WavLM-based system significantly improved performance, achieving a WER of 9.22% and a CER of 2.59%. These results demonstrate the efficacy of our approach in recognising whispered speech and underscore the importance of tailored acoustic modeling for robust automatic speech recognition systems. This study provides valuable insights into developing effective automatic speech recognition solutions for challenging speech affected by whisper and dialect. The source codes for this paper are freely available.
Papers List
List of archived papers
A Language-Independent Approach to Classification of Textual File Fragments: Case Study of Persian, English, and Chinese Languages
Fatemeh Mansouri Hanis - Hamidreza Khoshvaghti - Mehdi Teimouri - Hadi Veisi
A Systematic Embedded Software Design Flow for Robotic Applications
Navid Mahdian - Seyed-Hosein Attarzadeh-Niaki - Armin Salimi-Badr
Decentralized Federated Learning in IoT Environments: A Hierarchical Approach
Majid Mohammadpour - Seyedakbar Mostafavi
Emotion Recognition In Persian Speech Using Deep Neural Networks
Ali Yazdani - Hossein Simchi - Yasser Shekofteh
Bridging the Synthetic-to-Real Gap (BSRG): Creating Simulated Datasets for Domain Adaptation to Enhance Vehicle Detection
Behnaz Sadeghigol - Mohammad Ali Keyvanrad
T-Rank: Graph Data Analytics for Urban Traffic Modeling
Alireza Safarpour - Iman Gholampour - Amirhossain Aghazadeh Fard - Seyed Mohammad Karbasi
A Hybrid Echo State Network for Hypercomplex Pattern Recognition, Classification, and Big Data Analysis
Mohammad Jamshidi - Fatemeh Daneshfar
An Automated Visual Defect Segmentation for Flat Steel Surface Using Deep Neural Networks
Dorna Nourbakhsh Sabet - Mohammad Reza Zarifi - Javad Khoramdel - Yasamin Borhani - Esmaeil Najafi
Enhanced Atrial Fibrillation (AF) Detection via Data Augmentation with Diffusion Model
Arash Vashagh - Amirhossein Akhoondkazemi - Sayed Jalal Zahabi - Davood Shafie
CSI-Based Human Activity Recognition using Convolutional Neural Networks
Parisa Fard Moshiri - Mohammad Nabati - Reza Shahbazian - Seyed Ali Ghorashi
more
Samin Hamayesh - Version 41.3.1