Skip to main content

UI Interaction

QWR SDK provides a comprehensive system for interacting with UI elements in XR applications. This page explains how to create and customize XR-friendly user interfaces that can be interacted with using hands and controllers.

UI Interaction Overview​

The UI interaction system in QWR SDK includes:

  • Direct touch interaction with UI elements
  • Ray-based pointing and interaction
  • Custom UI components optimized for XR
  • Haptic feedback for UI interactions
  • Integration with Unity's UI system (uGUI)

Core Components​

QWRUIManager​

The QWRUIManager is the central component that coordinates all UI interactions.

// Access the UI manager
QWRUIManager uiManager = QWRUIManager.Instance;

// Configure global UI settings
uiManager.rayInteractionEnabled = true;
uiManager.directTouchInteractionEnabled = true;
uiManager.hapticFeedbackEnabled = true;

QWRUIInteractor​

The QWRUIInteractor component allows hands or controllers to interact with UI elements.

// Get UI interactors
QWRUIInteractor leftHandInteractor = QWRHandController.LeftHand.GetComponent<QWRUIInteractor>();
QWRUIInteractor rightHandInteractor = QWRHandController.RightHand.GetComponent<QWRUIInteractor>();

// Configure interactor settings
leftHandInteractor.interactionType = UIInteractionType.Ray;
rightHandInteractor.interactionType = UIInteractionType.DirectTouch;

// Subscribe to events
leftHandInteractor.OnUIElementClicked += HandleUIElementClicked;

QWRUICanvas​

The QWRUICanvas component makes a Unity Canvas interactable in XR.

// Get reference to UI canvas
QWRUICanvas uiCanvas = GetComponent<QWRUICanvas>();

// Configure canvas settings
uiCanvas.interactionMode = UIInteractionMode.Both; // Both ray and direct touch
uiCanvas.followGaze = true; // Canvas follows user's gaze
uiCanvas.followDistance = 1.0f; // Distance when following

Setting Up XR UI​

Creating an XR Canvas​

To create an XR-compatible UI canvas:

  1. Create a new Canvas in your scene
  2. Add the QWRUICanvas component
  3. Configure the interaction settings
// Example: Creating an XR canvas programmatically
public GameObject CreateXRCanvas()
{
    // Create a new canvas
    GameObject canvasObject = new GameObject("XR Canvas");
    Canvas canvas = canvasObject.AddComponent<Canvas>();
    canvas.renderMode = RenderMode.WorldSpace;
    
    // Add required components
    canvasObject.AddComponent<CanvasScaler>();
    canvasObject.AddComponent<GraphicRaycaster>();
    
    // Add QWR UI Canvas component
    QWRUICanvas qwrCanvas = canvasObject.AddComponent<QWRUICanvas>();
    qwrCanvas.interactionMode = UIInteractionMode.Both;
    
    // Position the canvas
    canvasObject.transform.position = new Vector3(0, 1.6f, 1.0f);
    canvasObject.transform.rotation = Quaternion.Euler(0, 180, 0);
    canvasObject.transform.localScale = new Vector3(0.001f, 0.001f, 0.001f);
    
    return canvasObject;
}

UI Interaction Modes​

The QWRUICanvas supports different interaction modes:

  • UIInteractionMode.Ray: Only ray-based interaction
  • UIInteractionMode.DirectTouch: Only direct touch interaction
  • UIInteractionMode.Both: Both ray and direct touch interaction
  • UIInteractionMode.None: No interaction (display only)
// Configure interaction mode
QWRUICanvas canvas = GetComponent<QWRUICanvas>();
canvas.interactionMode = UIInteractionMode.Both;

Ray Interaction​

Ray interaction allows users to point at and interact with UI elements from a distance.

Ray Visualization​

You can customize the appearance of the interaction ray:

// Configure ray visualization
QWRUIInteractor rayInteractor = GetComponent<QWRUIInteractor>();
rayInteractor.rayColor = Color.blue;
rayInteractor.rayThickness = 0.002f;
rayInteractor.rayLength = 5.0f;
rayInteractor.showRayOnHoverOnly = true;

Ray Interaction Settings​

// Configure ray interaction settings
rayInteractor.cursorPrefab = myCursorPrefab; // Custom cursor at hit point
rayInteractor.curveRay = true; // Use curved ray
rayInteractor.rayBendRatio = 0.5f; // How much the ray bends

Direct Touch Interaction​

Direct touch interaction allows users to physically touch UI elements with their fingers or controllers.

Touch Settings​

// Configure direct touch settings
QWRUIInteractor touchInteractor = GetComponent<QWRUIInteractor>();
touchInteractor.interactionType = UIInteractionType.DirectTouch;
touchInteractor.touchOffset = 0.01f; // Distance from finger tip for touch detection
touchInteractor.touchRadius = 0.01f; // Radius of touch detection sphere

Touch Visualization​

// Configure touch visualization
touchInteractor.showTouchPoint = true;
touchInteractor.touchPointPrefab = myTouchPointPrefab;
touchInteractor.touchPointColor = new Color(1, 1, 1, 0.5f);

XR-Optimized UI Components​

QWR SDK includes several UI components optimized for XR interaction.

QWRButton​

An enhanced button with haptic feedback and visual effects.

// Configure XR button
QWRButton button = GetComponent<QWRButton>();
button.hapticFeedbackStrength = 0.5f;
button.pressDepth = 0.01f; // How deep the button presses
button.clickSound = myClickSound; // Audio clip for click sound

// Subscribe to events
button.OnButtonPressed += HandleButtonPressed;
button.OnButtonReleased += HandleButtonReleased;

QWRSlider​

A slider optimized for XR interaction.

// Configure XR slider
QWRSlider slider = GetComponent<QWRSlider>();
slider.hapticFeedbackEnabled = true;
slider.hapticFeedbackInterval = 0.1f;
slider.grabberSize = 0.05f; // Size of the grab handle

// Subscribe to events
slider.OnValueChanged += HandleSliderValueChanged;

QWRToggle​

A toggle switch optimized for XR interaction.

// Configure XR toggle
QWRToggle toggle = GetComponent<QWRToggle>();
toggle.toggleDepth = 0.01f;
toggle.animationDuration = 0.2f;

// Subscribe to events
toggle.OnValueChanged += HandleToggleValueChanged;

QWRKeyboard​

A virtual keyboard for text input in XR.

// Configure XR keyboard
QWRKeyboard keyboard = GetComponent<QWRKeyboard>();
keyboard.targetInputField = myInputField;
keyboard.keySize = new Vector2(0.04f, 0.04f);
keyboard.keySpacing = 0.01f;
keyboard.hapticFeedbackEnabled = true;

// Show/hide keyboard
keyboard.Show();
keyboard.Hide();

UI Placement and Positioning​

World-Locked UI​

UI that stays fixed in the world space:

// Configure world-locked UI
QWRUICanvas canvas = GetComponent<QWRUICanvas>();
canvas.followType = UIFollowType.None;

Body-Locked UI​

UI that follows the user but stays at a fixed distance:

// Configure body-locked UI
QWRUICanvas canvas = GetComponent<QWRUICanvas>();
canvas.followType = UIFollowType.BodyLocked;
canvas.followDistance = 1.0f;
canvas.followOffset = new Vector3(0, 0, 0);
canvas.followSmoothTime = 0.2f;

Gaze-Locked UI​

UI that follows the user's gaze direction:

// Configure gaze-locked UI
QWRUICanvas canvas = GetComponent<QWRUICanvas>();
canvas.followType = UIFollowType.GazeLocked;
canvas.followDistance = 1.0f;
canvas.gazeLockThreshold = 30f; // Degrees from center of view

Hand-Attached UI​

UI that attaches to the user's hand:

// Configure hand-attached UI
QWRUICanvas canvas = GetComponent<QWRUICanvas>();
canvas.followType = UIFollowType.HandAttached;
canvas.attachedHand = HandType.Left;
canvas.handAttachOffset = new Vector3(0, 0.1f, 0.05f);

UI Events and Interaction​

UI Event System​

QWR SDK integrates with Unity's Event System for UI interactions:

// Configure event system for XR
QWREventSystem eventSystem = FindObjectOfType<QWREventSystem>();
eventSystem.raycastMode = UIRaycastMode.Physics; // Use physics raycasting

Custom UI Events​

You can create custom UI events:

// Create a custom UI event handler
public class CustomUIHandler : MonoBehaviour, IQWRUIHandler
{
    // Implement UI event interfaces
    public void OnQWRPointerEnter(QWRPointerEventData eventData)
    {
        Debug.Log("Pointer entered");
    }
    
    public void OnQWRPointerExit(QWRPointerEventData eventData)
    {
        Debug.Log("Pointer exited");
    }
    
    public void OnQWRPointerDown(QWRPointerEventData eventData)
    {
        Debug.Log("Pointer down");
    }
    
    public void OnQWRPointerUp(QWRPointerEventData eventData)
    {
        Debug.Log("Pointer up");
    }
    
    public void OnQWRPointerClick(QWRPointerEventData eventData)
    {
        Debug.Log("Pointer clicked");
    }
}

Haptic Feedback for UI​

QWR SDK provides haptic feedback for UI interactions:

// Configure haptic feedback for UI
QWRUIHaptics uiHaptics = GetComponent<QWRUIHaptics>();
uiHaptics.buttonClickStrength = 0.3f;
uiHaptics.buttonClickDuration = 0.05f;
uiHaptics.sliderTickStrength = 0.1f;
uiHaptics.toggleSwitchStrength = 0.2f;

UI Sound Effects​

You can add sound effects to UI interactions:

// Configure UI sound effects
QWRUISounds uiSounds = GetComponent<QWRUISounds>();
uiSounds.buttonClickSound = myClickSound;
uiSounds.sliderTickSound = myTickSound;
uiSounds.toggleSwitchSound = mySwitchSound;
uiSounds.volume = 0.8f;

Best Practices for XR UI​

UI Scale and Sizing​

  • Keep UI elements large enough to be easily interacted with (recommended minimum size: 2-3 cm)
  • Use appropriate spacing between interactive elements (recommended minimum spacing: 1 cm)
  • Consider the distance at which users will interact with the UI

UI Placement​

  • Place UI elements within comfortable reach and viewing angles
  • Avoid placing UI elements too close to the user's face
  • Consider the user's field of view when positioning UI

UI Feedback​

  • Provide clear visual feedback for hover and selection states
  • Use haptic feedback to confirm interactions
  • Add sound effects for important UI events

Performance Considerations​

  • Limit the number of UI elements visible at once
  • Use texture atlasing for UI elements
  • Consider using the QWRUIOptimizer component for performance optimization
// Add UI optimizer
QWRUIOptimizer optimizer = canvas.gameObject.AddComponent<QWRUIOptimizer>();
optimizer.batchingEnabled = true;
optimizer.dynamicPixelsPerUnit = 1;

Troubleshooting​

UI Not Interactable​

If UI elements can't be interacted with:

  • Ensure the Canvas has a QWRUICanvas component
  • Check that the QWRUIInteractor is properly configured
  • Verify that the Canvas's layer is included in the UI layer mask
  • Check for issues with the GraphicRaycaster component

Ray Interaction Issues​

If ray interaction isn't working:

  • Ensure ray interaction is enabled in the QWRUIManager
  • Check that the ray visualization settings are correct
  • Verify that the ray length is sufficient to reach the UI
  • Check for objects blocking the ray

Direct Touch Issues​

If direct touch interaction isn't working:

  • Ensure direct touch interaction is enabled in the QWRUIManager
  • Check that the touch detection settings are appropriate
  • Verify that the UI is within reach of the user's hands
  • Check for issues with finger tracking

Next Steps​

Now that you understand UI interaction, you can explore:

  • Multiplayer: Learn about multiplayer networking
  • Hand Tracking: Discover advanced hand tracking features
  • Tutorials: Follow step-by-step tutorials for common use cases