Multimodal bridge for conversational agents

Abstract

Smart environments are heavily utilized by users to fulfill their daily activities. The interaction between humans and machines witnessed an increase in modalities and providing an efficient user interaction relies on achieving multimodality. This thesis proposes a design mixing multimodality and smart environments in a contextaware system, intending to enhance the user experience by selecting the most efficient response mode, specifically in applications requiring interaction with conversational agents. The suggested solution is based on a fission engine, driven by gathered context and data, resulting in the generation of multimodal feedback. A preliminary prototype was implemented and tested to create a rule based model in charge of selecting between visual and auditory modalities. A second study was done to compare the rule based model with the hybrid one.