Development of a framework for eye-based human machine interface

Abstract

To improve lifestyle, Human Machine Interface(HMI) plays a significant role. There are several ways to interact with machines and image processing is the most effective one. The human eye is one of the most prominent features of the face that provides useful information besides facial expressions, e.g., eye blink, eye motion, and pupil motions. These can be used to create an eye-based platform to control different components that can be beneficial for motor disabled people. Besides biological activities, eye blinking can also be used for interaction with the machine. Already few algorithms have been developed to detect face and eyes and also researchers developed some work of Human Machine Interface(HMI). Some of those are based on external sensors and a few on image processing. However, most of those existing human-machine interfaces required motion of physical organ to operate, which are not feasible to use by motor disabled people. Using the eye blink command to interact with the machine can give a solu- tion. In this work, we have presented an eye-based HMI framework using eye blinks and computer vision techniques that can interact between humans and machines where we do not need any extra devices or sensors rather a simple laptop. Taking user input video from a laptop camera is the first step of this work. Then, we detect face using Haar cascade classifier and track the face region with the help of CamShift algorithm. After that, we have applied the blink detection technique based on real time template matching algorithm. Our proposed framework allows the user to set blink patterns for a specific task, e.g., 1 blink to turn on a light. We use the rules to map blink patterns to perform different activities. To control external appliances integrate an Arduino mi- cro controller interface that helps to control external light, fan, and so on. To control software application, we write scripts to executes. After successful implementation of hardware setup, we have tested the system with more than 50 participants from different environment (indoor, outdoor, lab, low light, bright light and so on). Experimental eval- uation shows that the proposed system is highly effective in detecting human eye blink in real-time and can be used to interact with applications and machines conveniently.