Expect to see more mixed reality apps in the future, at least for the Oculus Quest 2. WinFuturenotes that Oculus has unveiled a toolkit, Passthrough API Experimental, that will make it relatively easy to “seamlessly” merge VR with the real world view from the Quest 2’s cameras.
You can project images on flat surfaces, create composite layers that float in space, and even apply visual styles (akin to social media filters) to real scenes. You could give yourself a virtual monitor to use with your real-world keyboard, for instance, or turn your home into a psychedelic dreamscape by flicking a virtual switch.
Privacy shouldn’t be an issue, Oculus claimed. The API only processes raw camera footage on-device, and apps can’t access, store or view imagery of the world around you. A rogue app shouldn’t transmit video of your home, to put it another way.
Oculus expects to deliver the framework to Unity engine developers with its next software development kit release. It will take a while for finished apps to surface, but don’t be surprised if mixed reality games and productivity tools become relatively commonplace as a result of Oculus’ new tools.