Merging Internet of Things (IoT) with Augmented/Mixed Reality: A Framework Open to the Public

Abstract

In recent years, there have been significant advancements in Augmented Reality (AR) and Mixed Reality (MR) technologies, offering users immersive experiences by seamlessly integrating virtual elements into the physical world. However, to fully unlock the potential of AR/MR devices, it is crucial to enhance their capabilities for collaborating with tangible objects in the Internet of Things (IoT). Unfortunately, the communication between AR/MR and IoT devices is often complicated due to their use of diverse technologies. Moreover, developing the necessary communication mechanisms typically requires specialized developers with expertise in these technologies.

To tackle these challenges, this article introduces a framework aimed at simplifying the integration of AR/MR and IoT devices, enabling dynamic and real-time communication between them. The proposed AR/MR-IoT framework utilizes established and open-source protocols and tools like MQTT, HTTPS, and Node-RED. Following a detailed exploration of the framework's inner workings, its potential is demonstrated through a practical scenario: monitoring and controlling a smart power socket via Microsoft HoloLens AR/MR glasses. The article also assesses the performance of this practical use case, revealing that, under standard operational conditions, the suggested framework can respond to interaction and data update requests in less than 100 milliseconds