New Framework Helps Developers Create Explainable AI for Augmented Reality Applications

Artificial intelligence (AI) and machine learning tools have become increasingly common in today’s technological landscape. However, most of these tools are based on “black box” models, which analyze data and make predictions without sharing their processes with human users. To address this issue, researchers at Meta Reality Labs have created a framework called XAIR that aims to create explainable AI (XAI) systems that can be applied in various augmented reality (AR) settings.

Framework and Guidelines

The XAIR framework addresses three open questions: when, what, and how. Developers must first address five key factors: user state and contextual information (AR-specific), system goal, user goal, and user profile (non-AR-specific). Once these factors are identified, developers can refer to the XAIR framework to adapt and polish their XAI systems for AR applications.

The XAIR framework includes guidelines for researchers and developers working on XAI for AR applications. The researchers summarized the findings of over 100 studies rooted in different fields to identify important dimensions that one should consider when developing XAI for AR uses. They conducted a large-scale survey involving over 500 users and held a workshop with 12 experts in the field. The survey responses and views shared by experts during the workshop offered valuable insight that guided their development of XAIR.

To evaluate the value of their framework, the researchers used it to create a real XAI system and tested it in real-time in a series of AR scenarios. They found that users perceived this system as both transparent and trustworthy, suggesting that their framework had effectively guided its development.

Future Research

In future studies, the researchers plan to explore the automation of the design framework, the creation of personalized XAI experience in AR over time, and the enabling of users to provide feedback to further improve the system. The XAIR framework creates the foundation and guidelines for future XAI interaction in AR systems. The researchers are also interested in exploring how the recent explosion of generative AI can influence future XAI design in AR.

Overall, the XAIR framework offers a promising solution to the issue of explainability in AI systems for AR applications. As AI continues to become more powerful, XAI will play an increasingly important role in promoting transparency and trust between human users and intelligent systems.