Abstract:
Road accidents by unlicensed drivers are a serious concern for road safety. In addition to this, car theft is another major crime that is ever increasing. Authorities and researchers have proposed many solutions to combat these crimes using GPS, RFID and IoT technologies. However, they have used fingerprints and facial photos separately. Moreover, the existing systems do not report performance analysis, especially in terms of processing time and latency.
In order to address this issue, an IoT-based smart system using machine learning (ML) is proposed to prevent unlicensed driving and vehicle theft as ML based authentication systems provide more accurate and scalable solutions. In our work, the IoT-based embedded system consists of a fingerprint sensor, a Wi-Fi enabled ESP32 microcontroller, a camera, and an LCD display. A prototype which is located within the vehicle and a web interface are also developed. The web interface will be the central hub for registration of the drivers for a particular vehicle using fingerprint and facial images. Publicly available Sokoto Coventry Fingerprint (SO-COFing) dataset and Labeled Faces in the Wild (LFW) dataset are used in this thesis.
The first contribution in this thesis is to develop the prototype system for performance analysis (such as accuracy, processing time, latency etc.). Secondly, design and implement TinyML DL models for biometric authentication within microcontrollers. Thirdly, design and implement the web based ML model using Convolutional Neural Network (CNN) in addition to the K-Nearest Neighbour (KNN) of Open Source Computer Vision (OpenCV) library.
The accuracy of the KNN is 90.23 percent for fingerprint, whereas the accuracy of the CNN (InceptionResNetV2) is 94.65 percent for fingerprint, and 72 percent for facial data. The accuracy of the TinyML DL model for microcontroller based on MobileNetV3 Small is 81.67 percent and with Wavelet Transform it is 88.43 percent for fingerprint authentication and for Face Recognition it is 67.39 percent.
A comprehensive benchmarking analysis shows that the system can provide a latency of 59.60 seconds and processing time of 46.60 seconds when tested with a database of 1000 fingerprints of drivers using OpenCV but using CNN the latency is 33.69 seconds and processing time is 14.54 seconds only. The system is also scalable by changing the capacity of the server.