Enabling a multi-party conversational virtual agent through head and mouth motion tracking

Abstract

Conversational gaze behavior is an important component of an embodied conversational agent (ECA). Without proper conversational gaze, conversational agents may be less persuasive, emotive, and ultimately less believable or usable. While many conversational agent systems have been created for one-on-one type interactions, there is a noticeable lack of multi-party-capable systems, i.e., systems capable of dealing with more than one user simultaneously. We present a conversational agent system capable of sensing and reacting to the conversational state of multiple users using computer vision algorithms for head and mouth motion tracking.