QR Code & Keyboard Tracking With Meta Mixed Reality Utility Kit
I spent some time last weekend testing the QR Code and keyboard tracking capabilities introduced in v78 and later. Keyboard tracking was already available in MRUK as of v72, but QR Code tracking, on the other hand, only became available starting with v78. Today, we’ll dive into the setup and implementation.
First, let’s look at the requirements for both tracking features:
General requirements: Quest 3 / 3s or later, and Unity 2022.3.15f1 or later.
QR Code tracking requirements: Horizon OS v78+ and Meta MR Utility Kit v78+.
Keyboard tracking requirements: Horizon OS v72+ and Meta MR Utility Kit v72+.
Unity Project Setup with MRUK
Fig 1.0 - MRUK v78 (Mixed Reality Utility Kit Package)
To get a project working with QR Code and Keyboard tracking, simply download and install the Meta MR Utility Kit v78+. Go to Window > Package Manager > My Assets, search for “Meta MR Utility Kit”, and install it. If you can’t find it, head over to the Unity Asset Store, add it to your cart (it’s free), and then it should appear under My Assets in the Package Manager.
Next, add two new building blocks by clicking on the Meta XR Tools dropdown > Building Blocks.
Search for “Passthrough” and click the + symbol to add it.
Then search for “Effect Mesh” and add it as well.
The Effect Mesh building block has a dependency on MRUK, which works perfectly since we’ll need that component for detecting QR Codes and keyboards. (Fig. 1.1 shows how your hierarchy should look at this point.)
Fig 1.1 - Unity Hierarchy w/ Building Blocks
At this point, you should have all the components needed to use the Camera with Passthrough features. The [BuildingBlock] MR Utility Kit component is the one we’ll focus on, as it enables QR Code and keyboard detection.
MRUK Scene Settings
To enable these features:
Select the MR Utility Component in the hierarchy.
Locate the MRUK script.
Expand Scene Settings > Tracker Configuration.
Enable QR Code Tracking or Keyboard Tracking.
Camera Rig Component Settings
Select the [BuildingBlock] Camera Rig.
Scroll down to Quest Features.
Ensure Anchor Support is set to Enabled.
Ensure Scene Support is set to Required.
(These settings are usually correct by default.)
For QR Code tracking, enable experimental support by clicking the Experimental button and toggling the checkbox.
Adding QR Code & Keyboard Tracking Capabilities
Most of the tracking magic happens within MRUK. Thankfully, Meta has already handled the complexities of implementing computer vision algorithms in Unity. All we need to do is create a simple script that binds to two methods (callbacks). The two key methods to remember are:
OnTrackableAdded(MRUKTrackable trackable)
OnTrackableRemoved(MRUKTrackable trackable)
These methods are self-explanatory based on their names, but we need to bind to them in order to receive a callback whenever new trackables (QR Codes or Keyboards) are detected. Let me show you a basic example of how to do this, and then I’ll explain how it works.
using Meta.XR.MRUtilityKit;
using UnityEngine;
public class TrackablesManager : MonoBehaviour
{
[SerializeField] private GameObject trackedObjectPrefab;
public void OnTrackableAdded(MRUKTrackable trackable)
{
Debug.Log($"Trackable of type {trackable.TrackableType} added");
Instantiate(trackedObjectPrefab, trackable.transform);
}
public void OnTrackableRemoved(MRUKTrackable trackable)
{
Debug.Log($"Trackable removed: {trackable.name}");
Destroy(trackable.gameObject);
}
}Fig 1.2 - Trackables Manager
The script above implements two methods, OnTrackableAdded and OnTrackableRemoved. When executed, an instance of the trackedObjectPrefab will be created as a child of the trackable object passed into the method parameter.
This trackable object could be a QR Code or a Keyboard. Meta uses this common object type for anything MRUK can track. For now, that’s just QR Codes and Keyboards, but this will likely expand in the future to include other object types.
Now, create a new script with the code shown above, name it TrackablesManager, and assign it to an object in the hierarchy. For simplicity, add it to [BuildingBlock] MR Utility Kit.
Next:
Create a new Cube in the hierarchy.
Scale it down to 0.1 on all axes.
Drag it into a Prefabs folder to turn it into a prefab.
Assign this prefab to your TrackablesManager component in the Inspector window.
(See Fig. 1.2 for details.)
Fig 1.3 - Binding Trackable Methods
To bind to the MRUK OnTrackable… callbacks, we need to connect the TrackablesManager public methods to MRUK. In the Inspector, go to MRUK > Scene Settings > Tracker Configuration. For each Trackable… exposed method, click the + symbol, then drag and drop the TrackablesManager component into the slot.
(See Fig. 1.3 for details if this seems unclear.)
Deploying and testing QR Code & Keyboard Detection
To test QR Code and keyboard tracking, make sure to address all the suggestions in the Project Setup Tool.
Go to Meta XR Tools > Project Setup Tool and apply all fixes for both platforms (Android/Meta Quest and Standalone).
Open XR Plug-in Management and install it.
Enable OpenXR and the Meta XR feature group for all platforms.
In OpenXR Settings (under the XR Plug-in Manager node), add Oculus Touch Controller Profile to the Enabled Interaction Profiles. Do this for all platforms.
Connect your Meta Quest 3 or 3s via USB-C, then build and deploy your application.
QR Code & Keyboard Tracking Latency & Issues
If you’ve tested these features on your headset with v78, you’ve probably noticed that QR Code tracking is slower than Keyboard tracking. Keyboard tracking behaves more like a continuous tracking mechanism, whereas QR Code tracking does not update as smoothly when the QR Code is moved to a new position. This is expected behavior for now, and I’m confident we’ll see improvements in future versions. In addition, I noticed that QR Codes with logos are currently not supported and OnTrackableRemoved callback doesn’t execute when QR Code is no longer within the cameras FoV.
QR Code & Keyboard Bounding Areas
Fig 1.4 - Bounding Areas
I’ll wrap up this post here and invite you to check out my YouTube video, where I demonstrate how to create a bounding area visualizer for both trackables and how to retrieve the QR Code payload.
Let me know in the comments below if you have any additional questions.
Thank you!
Additional Resources:
Full GitHub repo based on my latest QR Code & Keyboard MRUK YouTube video available here
