Building real-time unconstrained eye tracking with deep learning
Krafka, Kyle John
MetadataShow full item record
Eye tracking is an important tool with applications in many domains. The ability to measure where a person is looking can be used for psychological studies, medical diagnoses, and human--computer interaction techniques. Existing high-accuracy solutions require custom and expensive hardware, limiting their reach. A more accurate low-cost solution is proposed to bring real-time unconstrained eye tracking to mobile devices. A deep convolutional neural network is used to determine gaze using only an image acquired from the front-facing camera found on modern phones and tablets. As with most deep learning approaches, the model requires a large volume of training data to perform well. Existing gaze datasets suffer from too few subjects or too little variety. This is largely due to the difficulty in conducting such experiments on a large scale. To overcome this data limitation, a crowdsourcing technique is introduced along with an unprecedentedly-large dataset, both in terms of the number of subjects and in variability. With this dataset, the trained model achieves state-of-the-art accuracy by a significant margin. Furthermore, it is robust to various lighting conditions and different user poses. Through an extensive evaluation, a variety of approaches to further improve the model's accuracy are demonstrated. Finally, to enable real-world mobile application of our model, the computation time and memory usage are optimized while maintaining high accuracy. This end-to-end design of an eye tracking system represents how modern computer vision and machine learning techniques can be used to make significant progress in appearance-based problems. These novel contributions represent a significant leap forward for eye tracking and should better equip the next generation of researchers and innovators.