Context-Aware Conversation Support with Augmented Reality and Generative AI

Effective and smooth conversations rely on strong social skills and background knowledges. However, gaps in knowledge among participants often disrupt the flow of conversation. For example, a conversational participant might struggle to engage in an effective conversation about planning a trip to Miami if they have never visited before. Recent advances in Generative AI (GenAI) have shown promising capabilities in generating information, while head-mounted Augmented Reality (AR) offers the potential to render and visualize information anywhere in the field of view. However, it remains unclear how GenAI and AR can synergistically support conversations. The overarching goal of this project is to explore the design of an AR-based conversational supporting tools. Ultimately, while wearing the head-mounted AR, useful information should be rendered with right level of details, at the right time and right place in front of the conversational participants.