Package org.lwjgl.openxr

Class MSFTHandTrackingMesh

java.lang.Object

org.lwjgl.openxr.MSFTHandTrackingMesh

public class MSFTHandTrackingMesh
extends Object

The XR_MSFT_hand_tracking_mesh extension.

This extension enables hand tracking inputs represented as a dynamic hand mesh. It enables applications to render hands in XR experiences and interact with virtual objects using hand meshes.

The application must also enable the XR_EXT_hand_tracking extension in order to use this extension.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	XR_HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT	XrHandPoseTypeMSFT - Describe type of input hand pose
static final int	XR_HAND_POSE_TYPE_TRACKED_MSFT	XrHandPoseTypeMSFT - Describe type of input hand pose
static final String	XR_MSFT_HAND_TRACKING_MESH_EXTENSION_NAME	The extension name.
static final int	XR_MSFT_hand_tracking_mesh_SPEC_VERSION	The extension specification version.
static final int	XR_TYPE_HAND_MESH_MSFT	Extends XrStructureType.
static final int	XR_TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT	Extends XrStructureType.
static final int	XR_TYPE_HAND_MESH_UPDATE_INFO_MSFT	Extends XrStructureType.
static final int	XR_TYPE_HAND_POSE_TYPE_INFO_MSFT	Extends XrStructureType.
static final int	XR_TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT	Extends XrStructureType.

Method Summary

Modifier and Type	Method	Description
static int	nxrCreateHandMeshSpaceMSFT(XrHandTrackerEXT handTracker, long createInfo, long space)	Unsafe version of: CreateHandMeshSpaceMSFT
static int	nxrUpdateHandMeshMSFT(XrHandTrackerEXT handTracker, long updateInfo, long handMesh)	Unsafe version of: UpdateHandMeshMSFT
static int	xrCreateHandMeshSpaceMSFT(XrHandTrackerEXT handTracker, XrHandMeshSpaceCreateInfoMSFT createInfo, org.lwjgl.PointerBuffer space)	Create a space for hand mesh tracking.
static int	xrUpdateHandMeshMSFT(XrHandTrackerEXT handTracker, XrHandMeshUpdateInfoMSFT updateInfo, XrHandMeshMSFT handMesh)	Update hand mesh buffers.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

XR_MSFT_hand_tracking_mesh_SPEC_VERSION

public static final int XR_MSFT_hand_tracking_mesh_SPEC_VERSION

The extension specification version.

See Also:

• Constant Field Values

XR_MSFT_HAND_TRACKING_MESH_EXTENSION_NAME

public static final String XR_MSFT_HAND_TRACKING_MESH_EXTENSION_NAME

The extension name.

See Also:

• Constant Field Values

XR_TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT

public static final int XR_TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT

Extends XrStructureType.

Enum values:

• TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT
• TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT
• TYPE_HAND_MESH_UPDATE_INFO_MSFT
• TYPE_HAND_MESH_MSFT
• TYPE_HAND_POSE_TYPE_INFO_MSFT

See Also:

• Constant Field Values

XR_TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT

public static final int XR_TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT

Extends XrStructureType.

Enum values:

• TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT
• TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT
• TYPE_HAND_MESH_UPDATE_INFO_MSFT
• TYPE_HAND_MESH_MSFT
• TYPE_HAND_POSE_TYPE_INFO_MSFT

See Also:

• Constant Field Values

XR_TYPE_HAND_MESH_UPDATE_INFO_MSFT

public static final int XR_TYPE_HAND_MESH_UPDATE_INFO_MSFT

Extends XrStructureType.

Enum values:

• TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT
• TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT
• TYPE_HAND_MESH_UPDATE_INFO_MSFT
• TYPE_HAND_MESH_MSFT
• TYPE_HAND_POSE_TYPE_INFO_MSFT

See Also:

• Constant Field Values

XR_TYPE_HAND_MESH_MSFT

public static final int XR_TYPE_HAND_MESH_MSFT

Extends XrStructureType.

Enum values:

• TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT
• TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT
• TYPE_HAND_MESH_UPDATE_INFO_MSFT
• TYPE_HAND_MESH_MSFT
• TYPE_HAND_POSE_TYPE_INFO_MSFT

See Also:

• Constant Field Values

XR_TYPE_HAND_POSE_TYPE_INFO_MSFT

public static final int XR_TYPE_HAND_POSE_TYPE_INFO_MSFT

Extends XrStructureType.

Enum values:

• TYPE_SYSTEM_HAND_TRACKING_MESH_PROPERTIES_MSFT
• TYPE_HAND_MESH_SPACE_CREATE_INFO_MSFT
• TYPE_HAND_MESH_UPDATE_INFO_MSFT
• TYPE_HAND_MESH_MSFT
• TYPE_HAND_POSE_TYPE_INFO_MSFT

See Also:

• Constant Field Values

XR_HAND_POSE_TYPE_TRACKED_MSFT

public static final int XR_HAND_POSE_TYPE_TRACKED_MSFT

XrHandPoseTypeMSFT - Describe type of input hand pose

Enumerant Descriptions

• HAND_POSE_TYPE_TRACKED_MSFT represents a hand pose provided by actual tracking of the user’s hand.
• HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT represents a stable reference hand pose in a relaxed open hand shape.

The HAND_POSE_TYPE_TRACKED_MSFT input provides best fidelity to the user’s actual hand motion. When the hand tracking input requires the user to be holding a controller in their hand, the hand tracking input will appear as the user virtually holding the controller. This input can be used to render the hand shape together with the controller in hand.

The HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT input does not move with the user’s actual hand. Through this reference hand pose, an application can get a stable hand joint and mesh that has the same mesh topology as the tracked hand mesh using the same XrHandTrackerEXT, so that the application can apply the data computed from a reference hand pose to the corresponding tracked hand.

Although a reference hand pose does not move with user’s hand motion, the bone length and hand thickness may be updated, for example when tracking result refines, or a different user’s hand is detected. The application should update reference hand joints and meshes when the tracked mesh’s indexBufferKey is changed or when the isActive value returned from UpdateHandMeshMSFT changes from FALSE to TRUE. It can use the returned indexBufferKey and vertexUpdateTime from UpdateHandMeshMSFT to avoid unnecessary CPU or GPU work to process the neutral hand inputs.

See Also

XrHandMeshSpaceCreateInfoMSFT, XrHandMeshUpdateInfoMSFT, XrHandPoseTypeInfoMSFT, XrHandTrackerCreateInfoEXT, CreateHandTrackerEXT

See Also:

• Constant Field Values

XR_HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT

public static final int XR_HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT

XrHandPoseTypeMSFT - Describe type of input hand pose

Enumerant Descriptions

• HAND_POSE_TYPE_TRACKED_MSFT represents a hand pose provided by actual tracking of the user’s hand.
• HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT represents a stable reference hand pose in a relaxed open hand shape.

The HAND_POSE_TYPE_TRACKED_MSFT input provides best fidelity to the user’s actual hand motion. When the hand tracking input requires the user to be holding a controller in their hand, the hand tracking input will appear as the user virtually holding the controller. This input can be used to render the hand shape together with the controller in hand.

The HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT input does not move with the user’s actual hand. Through this reference hand pose, an application can get a stable hand joint and mesh that has the same mesh topology as the tracked hand mesh using the same XrHandTrackerEXT, so that the application can apply the data computed from a reference hand pose to the corresponding tracked hand.

Although a reference hand pose does not move with user’s hand motion, the bone length and hand thickness may be updated, for example when tracking result refines, or a different user’s hand is detected. The application should update reference hand joints and meshes when the tracked mesh’s indexBufferKey is changed or when the isActive value returned from UpdateHandMeshMSFT changes from FALSE to TRUE. It can use the returned indexBufferKey and vertexUpdateTime from UpdateHandMeshMSFT to avoid unnecessary CPU or GPU work to process the neutral hand inputs.

See Also

XrHandMeshSpaceCreateInfoMSFT, XrHandMeshUpdateInfoMSFT, XrHandPoseTypeInfoMSFT, XrHandTrackerCreateInfoEXT, CreateHandTrackerEXT

See Also:

• Constant Field Values

Method Details

nxrCreateHandMeshSpaceMSFT

public static int nxrCreateHandMeshSpaceMSFT(XrHandTrackerEXT handTracker,
 long createInfo,
 long space)

Unsafe version of: CreateHandMeshSpaceMSFT

xrCreateHandMeshSpaceMSFT

public static int xrCreateHandMeshSpaceMSFT(XrHandTrackerEXT handTracker,
 XrHandMeshSpaceCreateInfoMSFT createInfo,
 org.lwjgl.PointerBuffer space)

Create a space for hand mesh tracking.

C Specification

The application creates a hand mesh space using function CreateHandMeshSpaceMSFT. The position and normal of hand mesh vertices will be represented in this space.

 XrResult xrCreateHandMeshSpaceMSFT(
     XrHandTrackerEXT                            handTracker,
     const XrHandMeshSpaceCreateInfoMSFT*        createInfo,
     XrSpace*                                    space);

Description

A hand mesh space location is specified by runtime preference to effectively represent hand mesh vertices without unnecessary transformations. For example, an optical hand tracking system can define the hand mesh space origin at the depth camera’s optical center.

An application should create separate hand mesh space handles for each hand to retrieve the corresponding hand mesh data. The runtime may use the lifetime of this hand mesh space handle to manage the underlying device resources. Therefore, the application should destroy the hand mesh handle after it is finished using the hand mesh.

The hand mesh space can be related to other spaces in the session, such as view reference space, or grip action space from the pathname:/interaction_profiles/khr/simple_controller interaction profile. The hand mesh space may be not locatable when the hand is outside of the tracking range, or if focus is removed from the application. In these cases, the runtime must not set the SPACE_LOCATION_POSITION_VALID_BIT and SPACE_LOCATION_ORIENTATION_VALID_BIT bits on calls to LocateSpace with the hand mesh space, and the application should avoid using the returned poses or query for hand mesh data.

If the underlying XrHandTrackerEXT is destroyed, the runtime must continue to support LocateSpace using the hand mesh space, and it must return space location with SPACE_LOCATION_POSITION_VALID_BIT and SPACE_LOCATION_ORIENTATION_VALID_BIT unset.

The application may create a mesh space for the reference hand by setting XrHandPoseTypeInfoMSFT::handPoseType to HAND_POSE_TYPE_REFERENCE_OPEN_PALM_MSFT. Hand mesh spaces for the reference hand must only be locatable in reference to mesh spaces or joint spaces of the reference hand.

Valid Usage (Implicit)

• The XR_MSFT_hand_tracking_mesh extension must be enabled prior to calling CreateHandMeshSpaceMSFT
• handTracker must be a valid XrHandTrackerEXT handle
• createInfo must be a pointer to a valid XrHandMeshSpaceCreateInfoMSFT structure
• space must be a pointer to an XrSpace handle

Return Codes

On success, this command returns

• SUCCESS
• SESSION_LOSS_PENDING

On failure, this command returns

• ERROR_FUNCTION_UNSUPPORTED
• ERROR_VALIDATION_FAILURE
• ERROR_RUNTIME_FAILURE
• ERROR_HANDLE_INVALID
• ERROR_INSTANCE_LOST
• ERROR_SESSION_LOST
• ERROR_OUT_OF_MEMORY
• ERROR_LIMIT_REACHED
• ERROR_POSE_INVALID
• ERROR_FEATURE_UNSUPPORTED

See Also

XrHandMeshSpaceCreateInfoMSFT

Parameters:

handTracker - an XrHandTrackerEXT handle previously created with the CreateHandTrackerEXT function.

createInfo - the XrHandMeshSpaceCreateInfoMSFT used to specify the hand mesh space.

space - the returned XrSpace handle of the new hand mesh space.

nxrUpdateHandMeshMSFT

public static int nxrUpdateHandMeshMSFT(XrHandTrackerEXT handTracker,
 long updateInfo,
 long handMesh)

Unsafe version of: UpdateHandMeshMSFT

xrUpdateHandMeshMSFT

public static int xrUpdateHandMeshMSFT(XrHandTrackerEXT handTracker,
 XrHandMeshUpdateInfoMSFT updateInfo,
 XrHandMeshMSFT handMesh)

Update hand mesh buffers.

C Specification

The application can use the UpdateHandMeshMSFT function to retrieve the hand mesh at a given timestamp. The hand mesh’s vertices position and normal are represented in the hand mesh space created by CreateHandMeshSpaceMSFT with a same XrHandTrackerEXT.

 XrResult xrUpdateHandMeshMSFT(
     XrHandTrackerEXT                            handTracker,
     const XrHandMeshUpdateInfoMSFT*             updateInfo,
     XrHandMeshMSFT*                             handMesh);

Description

The application should preallocate the index buffer and vertex buffer in XrHandMeshMSFT using the XrSystemHandTrackingMeshPropertiesMSFT::maxHandMeshIndexCount and XrSystemHandTrackingMeshPropertiesMSFT::maxHandMeshVertexCount from the XrSystemHandTrackingMeshPropertiesMSFT returned from the GetSystemProperties function.

The application should preallocate the XrHandMeshMSFT structure and reuse it for each frame so as to reduce the copies of data when underlying tracking data is not changed. The application should use XrHandMeshMSFT::indexBufferChanged and XrHandMeshMSFT::vertexBufferChanged in XrHandMeshMSFT to detect changes and avoid unnecessary data processing when there is no changes.

Valid Usage (Implicit)

• The XR_MSFT_hand_tracking_mesh extension must be enabled prior to calling UpdateHandMeshMSFT
• handTracker must be a valid XrHandTrackerEXT handle
• updateInfo must be a pointer to a valid XrHandMeshUpdateInfoMSFT structure
• handMesh must be a pointer to an XrHandMeshMSFT structure

Return Codes

On success, this command returns

• SUCCESS
• SESSION_LOSS_PENDING

On failure, this command returns

• ERROR_FUNCTION_UNSUPPORTED
• ERROR_VALIDATION_FAILURE
• ERROR_RUNTIME_FAILURE
• ERROR_HANDLE_INVALID
• ERROR_INSTANCE_LOST
• ERROR_SESSION_LOST
• ERROR_SIZE_INSUFFICIENT
• ERROR_TIME_INVALID
• ERROR_FEATURE_UNSUPPORTED

See Also

XrHandMeshMSFT, XrHandMeshUpdateInfoMSFT, CreateHandMeshSpaceMSFT

Parameters:

handTracker - an XrHandTrackerEXT handle previously created with CreateHandTrackerEXT.

updateInfo - an XrHandMeshUpdateInfoMSFT which contains information to query the hand mesh.

handMesh - an XrHandMeshMSFT structure to receive the updates of hand mesh data.