International Conference on Computer and Knowledge Engineering

Home / 15th International Conference on Computer and Knowledge Engineering

Beyond Appearance: Transformer-based Person Identification from Conversational Dynamics

Authors :

Masoumeh Chapariniya¹ Teodora Vukovic² Sarah Ebling³ Volker Dellwo⁴

1- University of Zurich 2- university of zurich 3- university of zurich 4- university of zurich

Keywords :

Person identification،conversational gestures،vision transformers،spatial-temporal modeling،keypoint dynamics

Abstract :

This paper investigates the performance of transformer-based architectures for person identification in natural, face-to-face conversation scenario. We implement and evaluate a two-stream framework that separately models spatial configurations and temporal motion patterns of 133 COCO WholeBody keypoints, extracted from a subset of the CANDOR conversational corpus. Our experiments compare pre-trained and from-scratch training, investigate the use of velocity features, and introduce a multi-scale temporal transformer for hierarchical motion modeling. Results demonstrate that domain-specific training significantly outperforms transfer learning, and that spatial configurations carry more discriminative information than temporal dynamics. The spatial transformer achieves 95.74% accuracy, while the multi-scale temporal transformer achieves 93.90%. Feature-level fusion pushes performance to 98.03%, confirming that postural and dynamic information are complementary. These findings highlight the potential of transformer architectures for person identification in natural interactions and provide insights for future multimodal and cross-cultural studies