International Conference on Computer and Knowledge Engineering

Home / 15th International Conference on Computer and Knowledge Engineering

Adaptive Sliding Window Optimization for Multi-Dimensional Data Streams Using Reinforcement Learning

Authors :

Abolfazl Zarghani¹

1- Ferdowsi university of mashhad

Keywords :

Reinforcement Learning،Sliding Window،Multi-Dimensional Data Streams،Concept Drift،Dueling DQN

Abstract :

Multi-dimensional data streams from IoT, financial markets, and real-time analytics require adaptive processing due to their high velocity and complex dependencies. Fixed-size sliding windows struggle with concept drift and bursty patterns. We propose RL-Window, a reinforcement learning (RL) approach using a Dueling Deep Q-Network (DQN) with prioritized experience replay and noisy networks to dynamically optimize window sizes based on stream characteristics like variance, correlations, and spectral features. Evaluated on UCI HAR, PAMAP2, and Yahoo! Finance datasets, RL-Window achieves superior classification accuracy (up to 92.1\%), drift robustness (-3.2\% drop), and energy efficiency (1.1 mJ) compared to baselines like ADWIN [1] and CNN-Adaptive [2]. Comprehensive dataset characterization and qualitative analyses highlight its adaptability for real-time, resource-constrained applications. We employ dual rolling buffers (200-point temporal, 512-point spectral with 50\% overlap) to build the state; stability is reported as the standard deviation of selected window sizes (samples); and energy/latency are measured on Raspberry Pi 4 via inline power metering and end-to-end timing.