Abstract
Manual attendance recording systems in academic settings are both inaccurate and require considerable manual effort, and they are also susceptible to proxy attendance scams. We develop a proof-of-concept fully automatic attendance system using Face Recognition technology with Multi-Prototype Person Re-ID to address many problems encountered when tracking students in a classroom environment (including face occlusion, pose variations, and clothing changes across multiple class sessions). Our proposed architecture uses Insight Face's buffalo model for face detection and recognition, creating 512-D feature vectors; Faster R-CNN with a ResNet50-FPN backbone for person detection; and a pre-trained ResNet50 from ImageNet for extracting 2048-D feature vectors for Re-ID. The novel part of this proposal is our dynamic multi-prototype tracking method based upon EMA (exponential moving averages), which tracks each student's maximum of 5 different "appearance" prototypes. The recently developed multimodal interaction fusion (MIF) framework has been an inspiration for our tracking algorithm, providing new ways to address changes in people's clothing. Through four examples (temporarily occluding face, extreme pose inversion, moderately profiled pose, clothing changed with re-locking), we demonstrate that face recognition can serve as an identity anchor and that Re-ID can provide continuous appearance tracking during periods when the face is occluded. The processing speed of our prototype was approximately 0.7-0.9 frames per second on standard CPU hardware and 9.5-14.3 frames per second with GPU acceleration.