International Conference on Computer and Knowledge Engineering

Home / 14th International Conference on Computer and Knowledge Engineering

Leveraging Self-Supervised Models for Automatic Whispered Speech Recognition

Authors :

Aref Farhadipour¹ Homa Asadi² Volker Dellwo³

1- University of Tehran 2- university of isfahan 3- Zurich University of Applied Sciences

Keywords :

Automatic Speech Recognition،whisper speech to text،self-supervised learning،speech processing،deep learning،transformers،Wavlm model،Dialect variation،Whisper model

Abstract :

In automatic speech recognition, any factor that alters the acoustic properties of speech can pose a challenge to the system's performance. This paper presents a novel approach for automatic whispered speech recognition in the Irish dialect using the self-supervised WavLM model. Conventional automatic speech recognition systems often fail to accurately recognise whispered speech due to its distinct acoustic properties and the scarcity of relevant training data. To address this challenge, we utilized a pre-trained WavLM model, fine-tuned with a combination of whispered and normal speech data from the wTIMIT and CHAINS datasets, which include the English language in Singaporean and Irish dialects, respectively. Our baseline evaluation with the OpenAI Whisper model highlighted its limitations, achieving a Word Error Rate (WER) of 18.8% and a Character Error Rate (CER) of 4.24% on whispered speech. In contrast, the proposed WavLM-based system significantly improved performance, achieving a WER of 9.22% and a CER of 2.59%. These results demonstrate the efficacy of our approach in recognising whispered speech and underscore the importance of tailored acoustic modeling for robust automatic speech recognition systems. This study provides valuable insights into developing effective automatic speech recognition solutions for challenging speech affected by whisper and dialect. The source codes for this paper are freely available.

Papers List

List of archived papers

Extracting Major Topics of COVID-19 Related Tweets

Faezeh Azizi - Hamed Vahdat-Nejad - Hamideh Hajiabadi - Mohammad Hossein Khosravi

Optimizing the controller placement problem in SDN with uncertain parameters with robust optimization

Mohammad Kazemi - AhmadReza Montazerolghaem

Introducing E4MT and LMBNC: Persian pre-processing utilities

Zakieh Shakeri - Mehran Ziabary - Behrooz Vedadian - Fatemeh Azadi - Saeed Torabzadeh - Arian Atefi

XAI for Transparent Autonomous Vehicles: A New Approach to Understanding Decision-Making in Self-driving Cars

Maryam Sadat Hosseini Azad - Amir Abbas Hamidi Imani - Shahriar Baradaran Shokouhi

Automated software design using Machine Learning With Natural Language Processing

Fahimeh Khedmatkon - Seyed Mohammad Hossein Hasheminejad - Jaleh Shoshtarian Malak

An Efficient Approach for Breast Abnormality Detection through High-Level Features of Thermography Images

Farhad Abedinzadeh Torghabeh - Yeganeh Modaresnia - Seyyed Abed Hosseini

A Hybrid Architecture to Optimize Persian FAQ Retrieval using Semantic Similarity Search

Seyed Amir Mohammad Hosseini - Fatemeh Dehbashi - Setare Kahnemuee - Mohsen Kahani - Morteza Fardin

A New Time Series Approach in Churn Prediction with Discriminatory Intervals

Hedieh Ahmadi - Seyed Mohammad Hossein Hasheminejad

Non-Functional Requirement Extracting Methods for AI-based Systems: A Survey

Reza Damirchi - Amineh Amini

Cardiology Disease Diagnosis by Analyzing Histological Microscopic Images Using Deep Learning

Maria Salehpanah - Jafar Tanha - Zahra Jafari - SeyedEhsan Roshan - Sajad Rezaei