IIITB researchers develop pipeline for a ‘human’ engagement with bots

IIITB researchers develop interpretable pipeline to predict engagement in human-robot interactions. Pipeline uses ML & psychology to model engagement & provide deeper insights into human personality, attitudes & emotions. Applications in online learning, assistive robotics & intelligent conversational agents.

While a teacher can always infer if a student is paying attention in the classroom, would an edutech bot know if the lesson on screen is holding his/her attention? The day may not be too far when the distinguishing line between these two kinds of interactions would blur.

Researchers at the International Institute of Informational Technology Bangalore (IIITB) have developed a ready-to-use interpretable pipeline which uses machine learning and psychology for modelling engagement in human-robot interactions. Along with the prediction of engagement, the pipeline can provide deeper insights into human personality, attitudes, and emotions and thereby improve the way the robot interacts with the human user. 

“The robotic interaction systems we have now do not take into account the human feelings, emotions and attitudes at a particular moment whereas in human interactions, we take these factors into account. We want assistive robots, chatbots, and other automated systems to also have a sense of human emotions,” Shrisha Rao, professor at IIIT Bangalore, and senior researcher on the study, told The Hindu. 

While interactive robots are expected to adapt to each individual’s needs when dealing with diverse engagement levels among various people in a crowd, the accurate measurement of each person’s engagement based on their visual and physiological cues is the initial challenge. This is where the pipelines that can deduce these engagement levels come into picture.  

The study which has been published in the PLOS One Journal of the Public Library Of Science has been authored by Soham Joshi and Arpitha Malavalli, graduate students in IIITB, under the supervision of Mr. Rao.  

The Big Five personality traits (extroversion, agreeableness, openness, conscientiousness, and neuroticism), the Interpersonal Circumplex (IPC), and the Triandis Theory of Interpersonal Behavior have been incorporated into the pipeline for the prediction of engagement. 

“Many previous methods for predicting engaging behaviour rely on blindly training deep neural networks to understand the link between multimodal cues and engagement levels. However, this approach differs significantly from how humans assess engagement in their interaction. Our goal is to create a prediction pipeline in human-robot interactions that mimic this human-like assessment of engagement,” said Mr. Joshi.