Process for Locating and Sizing the Skeleton
Segment origins and alignment are not directly measured from the analyzed slice data when we create an actor skeleton. Instead, the following process occurs:
- The actor profile that you add into Backstage defines the standard proportions of joints and their size.
- After we detect the T-pose, we analyze the height of the user and apply a global scaling to the standard joints.
- The arms are further refined for sizing by specifically looking at the volume data near the arms.
- This initial skeleton placement defines the first locations where the system searches for the true segments of the user.
- The actual location of the user's segments in T-Pose including any slight imperfections in that user's stance will occur ~5 frames after acquisition.
Summary of Joint Axes and Locations
The end result of the sizing process is that joints are placed in an approximation of the following locations.
Anatomical Approximation of Origin
|Upper Arm||The center of the head of the humerus.||Since OpenStage approximates segments as straight , the Z-axis of the humerus segment runs from the head of the humerus straight to the endpoint which is approximately the midpoint between the medial and lateral epicondyles of the humerus. As a result, the OpenStage humerus segment is perfectly aligned with the body or shaft of the actual humerus bone in a human although this misalignment is much less severe than seen with the femur.|
|Lower Arm||The midpoint between the epicondyles of the humerus.||The root of the OpenStage segment for the lower arm is designed to align with the center of rotation of the elbow. As noted, this is more accurately the tip of the humerus. As a result, if you wish to estimate the location of the upper end of the lower arm, an approximation of the ulnar notch of the radius, you will need to offset from the segment origin along the Z axis.|
|Hand||The end of the radius where the lunate and scaphoid bones abutt|
|Femur||The center of the head of the femur.||Since OpenStage approximates segments as straight , the Z-axis of the femur segment runs from the head of the femur straight to the endpoint which is approximately the midpoint between the medial and lateral epicondyles of the femur. As a result, the OpenStage femur segment is not aligned with the body or shaft of the actual femur bone in a human.|
|Tibia||The midpoint between the epicondyles of the femur.||The root of the OpenStage segment for the tibia aligns with the center of rotation of the knee. As noted, this is more accurately the tip of the femur. As a result, if you wish to estimate the location of the upper end of the tibia from the OpenStage segment, you will need to offset from the segment origin along the Z axis.|
|Ankle||The center of the body of the talus where it engages the tibia to form the talocrural joint.|
|Foot||The foot origin is best approximated as the center of the calcaneus.|
Major Joint Axes
Anatomical Approximation of Axis
Extraction from OpenStage Matrix
|Hinge of Elbow||Axis between the medial and lateral epicondyles of the humerus||The current Y-Axis from the orientation of the lower arm.||At T-pose, the hinge of the elbow is vertical. i.e., Rotation of the elbow brings the hand toward the front of the capture volume.|
|Hinge of Knee||Axis between the medial and lateral epicondyles of the femur||The current Y-Axis from the orientation of the tibia segment.||AT T-pose, the hinge of the knee is horizontal. i.e., Knees are facing forward and rotation takes the foot toward the rear of the capture volume.|
Understanding the Math from XML or the SDK
There are two ways to get rotation information for an OpenStage segment for a given frame.
- Use the SDK to retrieve a 4x4 transformation matrix in real time.
- Write the data to file and read the information from that file offline.
This discussion focuses on the OpenStage XML files, but the fundamentals are the same in Matlab.
About Rotation Matrices
As a matrix, that would be the following indicating an identity rotation and a translation of (-0.03, -3.47e-18, 0.87):
[[ 1 0 0 0 ]
[ 0 1 0 0 ]
[ 0 0 1 0 ]
[ -0.03 -3.47e-18 0.87 1 ]]
Thus, if we define the origin, O, as the row vector (0,0,0,1) and that matrix as M, we can perform the multiply O*M = (-0.03, -3.47e-18, 0.87, 1). i.e., It will translate the origin to the origin of that segment which in this case is the Sacrum and the root of our actor.
About Segment Orientation
OpenStage segments are defined to extend from their origin along the positive Z-axis prior to any transformation. Additional axes are defined by the default skeleton orientation at T-pose. This can be retrieved via the ActorSlot::defaultSkeleton member in the SDK or by looking at the transform matrices in the ActorLayout section of an XML file. Continuing with the example file cited above, the following entry defines the LowerArm_Left segment.
As a matrix, that would be the following:
[[ 1 0 0 0 ]
[ 0 0 1 0 ]
[ 0 -1 0 0 ]
[ -0.03 -0.473394 1.441 1 ]]
The third row of this matrix shows us the length of the lower arm. In this case, it's the negative Y-axis of our capture volume. This makes sense as the Y-axis of the capture volume is defined by the long arm of the calibration triangle. Similarly, we see that the second row of the matrix contains the positive Z-axis of the capture volume. This indicates that at T-pose, the hinge axis of the elbow is vertical.