0% Complete
Home
/
11th International Conference on Computer and Knowledge Engineering
Lightweight Local Transformer for COVID-19 Detection Using Chest CT Scans
Authors :
Hojat Asgarian Dehkordi
1
Hossein Kashiani
2
Amir Abbas Hamidi Imani
3
Shahriar Baradaran Shokouhi
4
1- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
2- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
3- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
4- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
Keywords :
COVID-19 Diagnosis, Vision Transformer, Limited dataset, Locality, Long-range Dependencies
Abstract :
As COVID-19 spreads around the globe, many studies have leveraged Convolutional Neural Networks (CNNs) for automated diagnosis of COVID-19 by means of CT images. However, CNNs have mainly failed to explicitly model long-range dependencies, primarily because of their intrinsic locality. To address this issue, Transformers have drawn increasing interest in exploiting long-range dependencies among input data. In this study, we aim to enjoy the merits of both local and global feature extractions in CNN and Transformer architectures. To this end, we go beyond the conventional Transformer frameworks and introduce a highly efficient Transformer architecture for early diagnosis and treatment of COVID-19 patients using CT images. Unlike conventional data-hungry Transformers, our model relaxes the requirement of large-scale training data in vision Transformers and attains on-par or even better performance than the state-of-the-art studies. This flexibility empowers our Transformer architecture to be exploited in data-scarce domains such as medical image analysis. Moreover, we tailor our Transformer architecture in two ways to embody the principle of locality, which once belonged to CNNs. First, we minimally inject convolutional inductive bias into the early blocks of our Transformer architecture and eliminate standard image patching in the vanilla Transformers. Second, unlike typical patch integration in the standard Transformers, we benefit from a deformable convolution in our architecture to adaptively attend to a small set of key features corresponding to nearby patches. Extensive experiential evaluations verify that our Transformer architecture surpasses its counterparts, advances the COVID-19 diagnosis by modeling intrinsic locality of CNNs, alleviates the computational complexity of Transformer architectures, and deals with the lack of large-scale training dataset for COVID-19 diagnosis.
Papers List
List of archived papers
MIPS-Core Application Specific Instruction-Set Processor for IDEA Cryptography − Comparison between Single-Cycle and Multi-Cycle Architectures
Ahmad Ahmadi - Reza Faghih Mirzaee
Improved TrustChain for Lightweight Devices
Seyed Salar Ghazi - Haleh Amintoosi
Fatty Liver Level Recognition Using Particle Swarm Optimization (PSO) Image Segmentation and Analysis
Seyed Muhammad Hossein Mousavi - Vyacheslav Lyashenko - Atiye Ilanloo - S. Younes Mirinezhad
Optimizing the controller placement problem in SDN with uncertain parameters with robust optimization
Mohammad Kazemi - AhmadReza Montazerolghaem
A Formalism for Specifying Capability-based Task Allocation in MAS
Samaneh HoseinDoost - Bahman Zamani - Afsaneh Fatemi
A Novel Density-Based KNN in Pattern Recognition
Sajad Haghzad Klidbary - Abazar Arabameri
Practical Implementation of Real-Time Waste Detection and Recycling based on Deep Learning for Delta Parallel Robot
Hasan Jalali - Shaya Garjani - Ahmad Kalhor - Mehdi Tale Masouleh - Parisa Yousefi
ExaAEC: A New Multi-label Emotion Classification Corpus in Arabic Tweets
Saeed Sarbazi-Azad - Ahmad Akbari - Mohsen Khazeni
UAV-based Firefighting by Multi-agent Reinforcement Learning
Reza Shami Tanha - Mohsen Hooshmand - Mohsen Afsharchi
Automatic Generation of XACML Code using Model-Driven Approach
Athareh Fatemian - Bahman Zamani - Marzieh Masoumi - Mehran Kamranpour - Behrouz Tork Ladani - Shekoufeh Kolahdouz Rahimi
more
Samin Hamayesh - Version 41.7.6