Please wait ...
0% Complete
Home
/
14th International Conference on Computer and Knowledge Engineering
Leveraging Self-Supervised Models for Automatic Whispered Speech Recognition
Authors :
Aref Farhadipour
1
Homa Asadi
2
Volker Dellwo
3
1- University of Tehran
2- university of isfahan
3- Zurich University of Applied Sciences
Keywords :
Automatic Speech Recognition،whisper speech to text،self-supervised learning،speech processing،deep learning،transformers،Wavlm model،Dialect variation،Whisper model
Abstract :
In automatic speech recognition, any factor that alters the acoustic properties of speech can pose a challenge to the system's performance. This paper presents a novel approach for automatic whispered speech recognition in the Irish dialect using the self-supervised WavLM model. Conventional automatic speech recognition systems often fail to accurately recognise whispered speech due to its distinct acoustic properties and the scarcity of relevant training data. To address this challenge, we utilized a pre-trained WavLM model, fine-tuned with a combination of whispered and normal speech data from the wTIMIT and CHAINS datasets, which include the English language in Singaporean and Irish dialects, respectively. Our baseline evaluation with the OpenAI Whisper model highlighted its limitations, achieving a Word Error Rate (WER) of 18.8% and a Character Error Rate (CER) of 4.24% on whispered speech. In contrast, the proposed WavLM-based system significantly improved performance, achieving a WER of 9.22% and a CER of 2.59%. These results demonstrate the efficacy of our approach in recognising whispered speech and underscore the importance of tailored acoustic modeling for robust automatic speech recognition systems. This study provides valuable insights into developing effective automatic speech recognition solutions for challenging speech affected by whisper and dialect. The source codes for this paper are freely available.
Papers List
List of archived papers
AL-YOLO: Accurate and Lightweight Vehicle and Pedestrian Detector in Foggy Weather
Behdad Sadeghian Pour - Hamidreza Mohammadi Jozani - Shahriar Baradaran Shokouhi
Overview of Electric Vehicles Charging Stations in Smart Grids
Mohammed Wadi - Wisam Elmasry - Mohammed Jouda - Hossein Shahinzadeh - Gevork B. Gharehpetian
Novel Insights in Deep Learning for Predicting Climate Phenomena
Mohammad Naisipour - Saghar Ganji - Iraj Saeedpanah - Behnam Mehrakizadeh - Ahmad Reza Labibzadeh
An Efficient Planning Method for Autonomous Navigation of a Wheeled-Robot based on Deep Reinforcement Learning
Ali Salimi Sadr - Mahdi Shahbazi Khojasteh - Hamed Malek - Armin Salimi-Badr
A Formalism for Specifying Capability-based Task Allocation in MAS
Samaneh HoseinDoost - Bahman Zamani - Afsaneh Fatemi
Towards Low-Overhead Mitigation of Trojan Bit-Flip Attacks on DNNs via Causal Inference
Bahare Gholami - Mohsen Raji
A New Hypercube Variant: Pruned Shuffle Connected Cube
Reza Latifi - Mahmoud Naghibzadeh
Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition
Hamid Ahmadabadi - Omid Nejati Manzari - Ahmad Ayatollahi
A large input-space-margin approach for adversarial training
Reihaneh Nikouei - Mohammad Taheri
Extracting structural clusters from NMF feature matrix using Cosine Similarity-Based Weighted Voting
Mehdi Rahimi - Keyhan Khamforoosh - Vafa Maihami
more
Samin Hamayesh - Version 46.0.2