0% Complete
Home
/
11th International Conference on Computer and Knowledge Engineering
Semi-automatic Detection of Persian Stopwords using FastText Library
Authors :
Mohammad Dehghani
1
Mohammad Manthouri
2
1- Tarbiat Modares University
2- Shahed University
Keywords :
stopword, natural language processing, fastText, word embedding, text mining
Abstract :
A stopword is a word that does not add much semantic information to the text that despite of its very high frequency. Stopwords include prepositions, conjunctions, and pronouns. One of the steps in natural language processing is to remove stopwords to reduce dataset size and process faster. In this study, a semi-automatic method for collecting the Persian language's stopwords is proposed. The proposed method lists the stopwords of each text depending on its subject. For this purpose, based on a corpus of news texts, the Inverse Document Frequency (IDF) weight in the text is calculated for each word and the stopwords candidates are determined. Then, using the fastText library, the vector of each word is obtained. In the next step, five neighbors are found for each vector. Next, by removing duplicate words, the final list of stopwords (1014 stopwords) is collected. The result of simulations show the accuracy of detecting stopwords by the k-nearest neighbor method is 94.6%.
Papers List
List of archived papers
Optimization Resource Allocation in NOMA-based Fog Computing with a Hybrid Algorithm
Zohreh Torki - S.Mojtaba Matinkhah
Intracranial Hemorrhage Classification using CBAM Attention Module and Convolutional Neural Networks
Parnian Rahimi - Marjan Naderan - Amir Jamshidnezhad - Shahram Rafie
Area-Efficient VLSI Implementation of Bit-Serial Multiplier Using Polynomial Basis over GF(2m)
Saeideh Nabipour - Javad Javidan - Gholamreza Zare Fatin
Classification of COVID-19 and Nodule in CT Images using Deep Convolutional Neural Network
Amirhossein Ghaemi - Seyyed Amir Mousavi mobarakeh - Habibollah Danyali - Kamran Kazemi
Adaptive Active Queue Management for Time Slot Channel Hopping in Industrial Internet of Things
Mehdi Zirak - Yasser Sedaghat - Mohammad Hossein Yaghmaee Moghaddam
Data Clustering using Chimp Optimization Algorithm
SAYED PEDRAM HAERI BOROUJENI - ELNAZ PASHAEI
Improved TrustChain for Lightweight Devices
Seyed Salar Ghazi - Haleh Amintoosi
Vision-Based Obstacle Avoidance in Drone Navigation using Deep Reinforcement Learning
Pooyan Rahmanzadeh Gervi - Ahad Harati - Sayed Kamaledin Ghiasi-Shirazi
Assessing Users' Influence on Respondents in Conversation Quality: A Quantitative Study on Reddit Based on the Cooperative Principle
Afsaneh Habibi - Fattaneh Taghiyareh
TrackMine: Topic Tracking in Model Mining using Genetic Algorithm
Mohammad Sajad Kasaei - Mohammadreza Sharbaf - Afsaneh Fatemi - Bahman Zamani
more
Samin Hamayesh - Version 41.7.6