International Conference on Computer and Knowledge Engineering

Home / 11th International Conference on Computer and Knowledge Engineering

Lightweight Local Transformer for COVID-19 Detection Using Chest CT Scans

Authors :

Hojat Asgarian Dehkordi¹ Hossein Kashiani² Amir Abbas Hamidi Imani³ Shahriar Baradaran Shokouhi⁴

1- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran 2- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran 3- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran 4- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran

Keywords :

COVID-19 Diagnosis, Vision Transformer, Limited dataset, Locality, Long-range Dependencies

Abstract :

As COVID-19 spreads around the globe, many studies have leveraged Convolutional Neural Networks (CNNs) for automated diagnosis of COVID-19 by means of CT images. However, CNNs have mainly failed to explicitly model long-range dependencies, primarily because of their intrinsic locality. To address this issue, Transformers have drawn increasing interest in exploiting long-range dependencies among input data. In this study, we aim to enjoy the merits of both local and global feature extractions in CNN and Transformer architectures. To this end, we go beyond the conventional Transformer frameworks and introduce a highly efficient Transformer architecture for early diagnosis and treatment of COVID-19 patients using CT images. Unlike conventional data-hungry Transformers, our model relaxes the requirement of large-scale training data in vision Transformers and attains on-par or even better performance than the state-of-the-art studies. This flexibility empowers our Transformer architecture to be exploited in data-scarce domains such as medical image analysis. Moreover, we tailor our Transformer architecture in two ways to embody the principle of locality, which once belonged to CNNs. First, we minimally inject convolutional inductive bias into the early blocks of our Transformer architecture and eliminate standard image patching in the vanilla Transformers. Second, unlike typical patch integration in the standard Transformers, we benefit from a deformable convolution in our architecture to adaptively attend to a small set of key features corresponding to nearby patches. Extensive experiential evaluations verify that our Transformer architecture surpasses its counterparts, advances the COVID-19 diagnosis by modeling intrinsic locality of CNNs, alleviates the computational complexity of Transformer architectures, and deals with the lack of large-scale training dataset for COVID-19 diagnosis.

Papers List

List of archived papers

Collaborative LLM Reasoning for Vulnerability Detection in Smart Contracts

Amirreza Samari - Parsa Hedayatnia - Seyyed Javad Bozorgzadeh Razavi - Mohammad Allahbakhsh - Haleh Amintoosi

Using Deep Learning for Classification of Lung Cancer on CT Images in Ardabil Province

Mohammad Ali Javadzadeh Barzaki - Jafar Abdollahi - Mohammad Negaresh - Maryam Salimi - Hadi Zolfeghari - Mohsen Mohammadi - Asma Salmani - Rona Jannati - Firouz Amani

Depression Diagnosis Using Optimization of Nonlinear EEG Features Based on Parametric Learning Tactics

Ali Asadi Zeidabadi - Melika Changizi - Mahdi Zolfagharzadeh Kermani - Sara Bargi Barkouk

A Semi-supervised Fake News Detection using Sentiment Encoding and LSTM with Self-Attention

Pouya Shaeri - Ali Katanforoush

Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition

Hamid Ahmadabadi - Omid Nejati Manzari - Ahmad Ayatollahi

GAP: Fault tolerance Improvement of Convolutional Neural Networks through GAN-aided Pruning

Pouya Hosseinzadeh - Yasser Sedaghat - Ahad Harati

Early detection of Parkinson’s disease using Convolutional Neural Networks on SPECT images

Reyhaneh Dehghan - Marjan Naderan - Seyyed Enayatallah Alavi

African Vultures Optimization Algorithm for Optimal Damping Controllers Design in the Electrical Power Grid System

Aliyu Sabo - Theophilus Ebuka Odoh - Samuel Habu - Hossein Shahinzadeh - Farshad Ebrahimi

Atlas-based segmentation of cardiac chambers in systolic and diastolic phases of echocardiographic images

Elham Fathipour - Mahdi Saadatmand

Traffic Sign Recognition Using Local Vision Transformer

Ali Farzipour - Omid Nejati Manzari - Shahriar B. Shokouhi