Human activity recognition on edge devices using salient region-guided knowledge-distilled lightweight model

Abstract

Understanding human behavior is crucial in disaster management, surveillance and el-der care. In disaster management, drone surveillance can quickly identify activities like walking, running, lying down, standing, and sitting. This helps prioritize rescue opera-tions and save lives. Accurate activity recognition also boosts security and effectively monitors public spaces. Additionally, in elder care, it assists in keeping an eye on the well-being of elderly individuals by promptly detecting falls or unusual behavior, which allows for timely intervention. Overall, these applications highlight the importance of real-time activity recognition for safety and support. Therefore, it’s important to de-velop a lightweight and accurate model that can run on edge devices like Raspberry Pi used in drones and security cameras. Faster models often sacrifice accuracy compared to larger, complex Deep Neural Networks (DNN) which are highly accurate, require a lot of computational power, making it difficult to implement Human Activity Recog-nition (HAR) systems on devices with limited processing capabilities. In this thesis, we have made two major contributions. First, we introduced a new lightweight DNN model called YOLOv5n-light, which is 4 times smaller and 1.5 times less computa-tionally intensive than YOLOv5n, the smallest model in the YOLOv5 family. Second, we proposed a novel training strategy called Pre-trained Weight Pruned Retrained by Salient Feature Guided Knowledge-Distillation (PWPR-SFGKD) to enhance the accu-racy of lightweight YOLO models. In our approach, we first train the model, followed by pruning its weights, and then perform a retraining process using a Salient Feature Guided Knowledge Distillation (SFGKD) technique. For loss calculation, we compute the difference between the feature maps of the full image and the background region, derived from both the teacher and student models. This allows us to capture more meaningful, context-specific information during the distillation process. When we ap-plied this method to the YOLOv5n model, we observed a significant improvement in performance, with the mAP-50 score increasing from 0.64 to 0.71. This result high-lights the effectiveness of our proposed PWPR-SFGKD approach in enhancing model accuracy, outperforming traditional YOLOv5 training strategies.