International Conference on Computer and Knowledge Engineering

Home / 15th International Conference on Computer and Knowledge Engineering

Human vs NotebookLM for Educational Podcasts: A Controlled Experiment on Two General Topics

Authors :

Ali Banihashemi¹ Amirali Shahriary² Yadollah Yaghoobzadeh³

1- ECE department, University of Tehran 2- ECE department, University of Tehran 3- ECE department, University of Tehran

Keywords :

Educational Podcasts،Google NotebookLM،Human-AI Comparison،Large Language Models (LLMs)،AI-Generated Content

Abstract :

Abstract—Podcasting has become a prominent medium in education, valued for its flexibility, accessibility, and potential to enhance student learning. However, creating high-quality educational podcasts is labor-intensive, limiting widespread adoption. Recent advancements in generative artificial intelligence offer new opportunities for automating podcast creation using large language models (LLMs). This study investigates the effectiveness of LLM-generated podcasts compared to traditional human-generated podcasts, focusing on comprehension and engagement among non-native English speakers. Our primary goal is to evaluate whether LLM-based tools—specifically Google's NotebookLM—can produce educational podcasts that match or surpass the learning value of human-created content. We conduct a controlled experiment using two general topics and human and LLM-generated podcasts to assess comprehension outcomes and subjective listener experiences. Results reveal that LLM-generated podcasts consistently lead to higher comprehension scores and are rated more topically focused than human-generated podcasts, with positive but not always significant trends in clarity and engagement. These findings suggest that LLM-generated podcasts can serve as a scalable and effective alternative to human-generated podcasts, offering new possibilities for personalized, accessible learning in digital education environments.