About Pose

Updated: 12/08/2022

What is the Pose Function?

Pose is a function that fades the switching of a single part among multiple similar parts based on motion.
Multiple similar parts are those that cause display inconsistencies when displayed at the same time, such as right hand A and right hand B.

This is also the stage of the process where the part opacity operations in motion are finally applied to the part.

We recommend that you read “Use Parameter IDs that Do Not Exist in the Model in the CubismModel Class” in “About Models” in advance for a better understanding of this page.

Flow of Part Opacity Operations from Motion

Manipulating part opacity in motion playback

To maintain consistency in part opacity, motion playback alone in the Original WorkFlow Framework does not operate directly on part opacity.

Part opacity operations in motion playback are instead replaced by overrides to the same ID parameter as the part.
At this time, parameter IDs that do not exist in the model are only retained as virtual parameters.
No fade processing is performed when switching between motions; only overwriting is performed.

The OW SDK assumes that steps are applied to the interpolation method for part opacity operations to motion.

// C++
	void CubismMotion::DoUpdateParameters(CubismModel* model, csmFloat32 timeSeconds, csmFloat32 fadeWeight, CubismMotionQueueEntry* motionQueueEntry)
    {
/* Omitted */
		for (; c < _motionData->CurveCount && curves[c].Type == CubismMotionCurveTarget_PartOpacity; ++c)
        {
            // Find parameter index.
            parameterIndex = model->GetParameterIndex(curves[c].Id);

            // Skip curve evaluation if no value in sink.
            if (parameterIndex == -1)
            {
                continue;
            }

            // Evaluate curve and apply value.
            value = EvaluateCurve(_motionData, c, time);

            model->SetParameterValue(parameterIndex, value);// Write to virtual parameters instead of part opacity.
        }
/* Omitted */
    }
// TypeScript
    public doUpdateParameters(model: CubismModel, userTimeSeconds: number, fadeWeight: number, motionQueueEntry: CubismMotionQueueEntry): void
    {
      
/* Omitted */

      for(; c < this._motionData.curveCount && curves.at(c).type == CubismMotionCurveTarget.CubismMotionCurveTarget_PartOpacity; ++c)
      {
        // Find parameter index.
        parameterIndex = model.getParameterIndex(curves.at(c).id);

        // Skip curve evaluation if no value in sink.
        if(parameterIndex == -1)
        {
          continue;
        }

        // Evaluate curve and apply value.
        value = evaluateCurve(this._motionData, c, time);

        model.setParameterValueByIndex(parameterIndex, value);
      }
/* Omitted */
    }
// Java
	public void doUpdateParameters(CubismModel model, float timeSeconds, float fadeWeight, CubismMotionQueueEntry motionQueueEntry){
      
/* Omitted */
      
		for (CubismMotionCurve curve : curves) {
          
			if (curve.type ! = CubismMotionCurveTarget.PARAMETER) {
			continue;
            }

          // Findparameter index.
          final int parameterIndex = model.getParameterIndex(curve.id);

          // Skip curve evaluation if no value.
          if (parameterIndex == -1) {
            continue;
          }
          
          // Evaluate curve and apply value.
          value = evaluateCurve(curve, time);
          
          ...
          
          model.setParameterValue(parameterIndex, v); // Write to virtual parameters instead of part opacity.
        }      
/* Omitted */
    }

Applying the Pose Function

By applying the Pose function during the last phase of the model update process, based on the information in .pose3.json, determine which parts to display for each group, referring to the value of the virtual parameter.

The API to apply Pose to a model is as follows.

// C++
CubismPose::UpdateParameters()
// TypeScript
CubismPose.updateParameters()
// Java
CubismPose.updateParameters();

The curves for part manipulation are recommended to be step curves, but if linear or other curves are selected, the display state is recognized when it becomes larger than 0.001.

When determining the displayed part, a proportional calculation is performed on the part opacity before operation and the difference time, and the new opacity calculated from the result of this proportional calculation is determined to be the linear interpolation.

// C++
	void CubismPose::DoFade(CubismModel* model, csmFloat32 deltaTimeSeconds, csmInt32 beginIndex, csmInt32 partGroupCount)
    {
        csmInt32    visiblePartIndex = -1;
        csmFloat32  newOpacity = 1.0f;

        const csmFloat32 Phi = 0.5f;
        const csmFloat32 BackOpacityThreshold = 0.15f;

        // Get the part that is currently in the display state
        for (csmInt32 i = beginIndex; i < beginIndex + partGroupCount; ++i)
        {
            csmInt32 partIndex = _partGroups[i].PartIndex;
            csmInt32 paramIndex = _partGroups[i].ParameterIndex;

            if (model->GetParameterValue(paramIndex) > Epsilon) // const csmFloat32 Epsilon= 0.001f;
            {
                if (visiblePartIndex >= 0)
                {
                    break;
                }

                visiblePartIndex = i;
                newOpacity = model->GetPartOpacity(partIndex);

                // Calculate new opacity
                newOpacity += (deltaTimeSeconds / _fadeTimeSeconds);

                if (newOpacity > 1.0f)
                {
                    newOpacity = 1.0f;
                }
            }
        }
/* Omitted */
    }
// TypeScript
	public doFade(model: CubismModel, deltaTimeSeconds: number, beginIndex: number, partGroupCount: number): void
    {
      let visiblePartIndex: number = -1;
      let newOpacity: number = 1.0;

      const phi: number = 0.5;
      const backOpacityThreshold: number = 0.15;

      // Get the part that is currently in the display state
      for(let i: number = beginIndex; i < beginIndex + partGroupCount; ++i)
      {
        const partIndex: number = this._partGroups.at(i).partIndex;
        const paramIndex: number = this._partGroups.at(i).parameterIndex;

        if(model.getParameterValueByIndex(paramIndex) > Epsilon)
        {
          if(visiblePartIndex >= 0)
          {
            break;
          }

          visiblePartIndex = i;
          newOpacity = model.getPartOpacityByIndex(partIndex);

          // Calculate new opacity
          newOpacity += (deltaTimeSeconds / this._fadeTimeSeconds);

          if(newOpacity > 1.0)
          {
            newOpacity = 1.0;
          }
        }
      }
/* Omitted */
    }
// Java
	public void doFade(CubismModel model,
                       float deltaTimeSeconds,
                       int beginIndex,
                       int partGroupCount)
	{
      int visiblePartIndex = -1;
      float newOpacity = 1.0f;
      
      // Get the part that is currently in the display state
      for (int i = beginIndex; i < beginIndex + partGroupCount; i++) {
        final int paramIndex = _partGroups.get(i).parameterIndex;
        
        if (model.getParameterValue(paramIndex) > EPSILON) {
          if (visiblePartIndex >= 0) {
            break;
          }
          
          // Calculate new opacity
          newOpacity = calculateOpacity(model, i, deltaTimeSeconds);
          visiblePartIndex = i;
        }
      }

/* Omitted */
  
}

After the display parts and their new opacity are determined, an opacity override is performed on the entire group.
The new opacity of the hidden parts will be reduced in relation to the opacity of the parts to be displayed at a level that does not allow the background to show through.

// C++
	void CubismPose::DoFade(CubismModel* model, csmFloat32 deltaTimeSeconds, csmInt32 beginIndex, csmInt32 partGroupCount)
    {
        csmInt32    visiblePartIndex = -1;
        csmFloat32  newOpacity = 1.0f;

        const csmFloat32 Phi = 0.5f;
        const csmFloat32 BackOpacityThreshold = 0.15f;
/* Omitted */

        if (visiblePartIndex < 0)
        {
            visiblePartIndex = 0;
            newOpacity = 1.0f;
        }

        // Set opacity for displayed and hidden parts.
        for (csmInt32 i = beginIndex; i < beginIndex + partGroupCount; ++i)
        {
            csmInt32 partsIndex = _partGroups[i].PartIndex;

            // Display parts settings
            if (visiblePartIndex == i)
            {
                model->SetPartOpacity(partsIndex, newOpacity); // Set first.
            }
            // Hidden parts settings
            else
            {
                csmFloat32 opacity = model->GetPartOpacity(partsIndex);
                csmFloat32 a1;          // Opacity obtained by calculation.

                if (newOpacity < Phi)
                {
                    a1 = newOpacity * (Phi - 1) / Phi + 1.0f; // Linear equation through (0,1),(phi,phi)
                }
                else
                {
                    a1 = (1 - newOpacity) * Phi / (1.0f - Phi); // Linear equation through (0,1),(phi,phi)
                }

                // When limiting the percentage of background visibility
                csmFloat32 backOpacity = (1.0f - a1) * (1.0f - newOpacity);

                if (backOpacity > BackOpacityThreshold)
                {
                    a1 = 1.0f - BackOpacityThreshold / (1.0f - newOpacity);
                }

                if (opacity > a1)
                {
                    opacity = a1; // If it is greater (darker) than the calculated opacity, increase the opacity.
                }

                model->SetPartOpacity(partsIndex, opacity);
            }
        }
    }
// TypeScript
	public doFade(model: CubismModel, deltaTimeSeconds: number, beginIndex: number, partGroupCount: number): void
    {
      let visiblePartIndex: number = -1;
      let newOpacity: number = 1.0;

      const phi: number = 0.5;
      const backOpacityThreshold: number = 0.15;

/* Omitted */

      if(visiblePartIndex < 0)
      {
        visiblePartIndex = 0;
        newOpacity = 1.0;
      }

      // Set opacity for displayed and hidden parts
      for(let i: number = beginIndex; i < beginIndex + partGroupCount; ++i)
      {
        const partsIndex: number = this._partGroups.at(i).partIndex;

        // Display parts settings
        if(visiblePartIndex == i)
        {
          model.setPartOpacityByIndex(partsIndex, newOpacity);   // Set first.
        }
        // Hidden parts settings
        else
        {
          let opacity: number = model.getPartOpacityByIndex(partsIndex);
          let a1: number; // Opacity obtained by calculation.

          if(newOpacity < phi)
          {
            a1 = newOpacity * (phi - 1) / phi + 1.0;    // Linear equation through (0,1),(phi,phi)
          }
          else
          {
            a1 = (1 - newOpacity) * phi / (1.0 - phi);  // Linear equation through (1,0),(phi,phi)
          }

          // When limiting the percentage of background visibility
          const backOpacity: number = (1.0 - a1) * (1.0 - newOpacity);

          if(backOpacity > backOpacityThreshold)
          {
            a1 = 1.0 - backOpacityThreshold / (1.0 - newOpacity);
          }

          if(opacity > a1)
          {
            opacity = a1;   // If it is greater (darker) than the calculated opacity, increase the opacity.
          }

          model.setPartOpacityByIndex(partsIndex, opacity);
        }
      }
    }
// Java
	public void doFade(CubismModel model,
                       float deltaTimeSeconds,
                       int beginIndex,
                       int partGroupCount)
	{
      int visiblePartIndex = -1;
      float newOpacity = 1.0f;
      
      /* Omitted */
      
      if (visiblePartIndex < 0) {
        visiblePartIndex = 0;
        newOpacity = 1.0f;
      }
      
      // Set opacity for displayed and hidden parts
      for (int i = beginIndex; i < beginIndex + partGroupCount; i++) {
        final int partsIndex = _partGroups.get(i).partIndex;
        
        // Display parts settings
        if (visiblePartIndex == i) {
          model.setPartOpacity(partsIndex, newOpacity);
        }
        // Hidden parts settings
        else {
          final float opacity = model.getPartOpacity(partsIndex);
          final float result = calcNonDisplayedPartsOpacity(opacity, newOpacity);
          model.setPartOpacity(partsIndex, result);
        }
      }
    }

Data structure of the Pose function

To speed up access to parameters and part information, the information handled by the Pose function is stored in a PartData structure that contains the index of the parameters and parts when accessed by part IDs.

Linked data is held as a child element of PartData as a Link.

// C++
	/**
    * @brief Manage data related to parts
    *
    * Manage various data related to the parts.
    */
    struct PartData
    {
/* Omitted */

        CubismIdHandle                     PartId;                 ///< Part ID
        csmInt32                            ParameterIndex;         ///< Parameter index
        csmInt32                            PartIndex;              ///< Part index
        csmVector<PartData>                 Link;                   ///< Linked parameters.
    };
// TypeScript
	/**
     * Manage data related to parts
     */
    export class PartData
    {
/* Omitted */
        
        partId: CubismIdHandle;   // Part ID
        parameterIndex: number; // Parameter index
        partIndex: number;  // Part index
        link: csmVector<PartData>;   // Linked parameters.
    }
// Java
	/**
    * Manage various data related to the parts.
    */
	public class PartData{
      /* Omitted */
      
      CubismId partId; // Part ID
      int parameterIndex; // Parameter index
      int partIndex; // Part index
      List<PartData> link // Linked parameters.
	}

CubismPose represents group information as a one-dimensional PartData array and an array of information on the number of pieces in each group.

// C++
	class CubismPose
    {
/* Omitted */

        csmVector<PartData>             _partGroups;                ///< Part groups
        csmVector<csmInt32>             _partGroupCounts;           ///< Number of each part group.
        csmFloat32                      _fadeTimeSeconds;           ///< Fade time [sec]
        csmFloat32                      _lastTimeSeconds;           ///< Last executed time [sec]
        CubismModel*                    _lastModel;                 ///< Previously operated model
    };
// TypeScript
	export class CubismPose
    {
/* Omitted */      

        _partGroups: csmVector<PartData>; // Part groups
        _partGroupCounts: csmVector<number>;         // Number of each part group.
        _fadeTimeSeconds: number;           // Fade time [sec]
        _lastModel: CubismModel;            // Previously operated model.
    }
// Java
	public class CubismPose{
/* Omitted */

      List<PartData> _partGroups; // Part groups
	 List<Integer> _partGroupCounts; // Number of each part group.
	 float _fadeTimeSeconds; // Fade time [sec]
	 float _lastTimeSeconds; // Last executed time [sec]
	 CubismModel _lastModel; // Previously operated model.
	}

The one-dimensional array PartData is processed as a group by passing the head position in the array and the number of elements to DoFade based on the information on the number of elements in each group.

// C++
	void CubismPose::UpdateParameters(CubismModel* model, csmFloat32 deltaTimeSeconds)
    {
        // Initialization is required when not the same as the previous model
        if (model ! = _lastModel)
        {
            // Initialize parameter index
            Reset(model);
        }

        _lastModel = model;

        // If the time is changed from the Settings, the elapsed time may be negative, so it is handled as 0 elapsed time.
        if (deltaTimeSeconds < 0.0f)
        {
            deltaTimeSeconds = 0.0f;
        }

        csmInt32 beginIndex = 0;

        for (csmUint32 i = 0; i < _partGroupCounts.GetSize(); i++)
        {
            csmInt32 partGroupCount = _partGroupCounts[i];

            DoFade(model, deltaTimeSeconds, beginIndex, partGroupCount);

            beginIndex += partGroupCount;
        }

        CopyPartOpacities(model);
    }
// TypeScript
	public updateParameters(model: CubismModel, deltaTimeSeconds: number): void
    {
      // Initialization is required if not the same as the previous model
      if(model ! = this._lastModel)
      {
        // Initialize parameter index
        this.reset(model);
      }

      this._lastModel = model;

      // If the time is changed from the Settings, the elapsed time may become negative, so it is handled as 0 elapsed time
      if(deltaTimeSeconds < 0.0)
      {
        deltaTimeSeconds = 0.0;
      }

      let beginIndex: number = 0;

      for(let i = 0; i < this._partGroupCounts.getSize(); i++)
      {
        const partGroupCount: number = this._partGroupCounts.at(i);

        this.doFade(model, deltaTimeSeconds, beginIndex, partGroupCount);

        beginIndex += partGroupCount;
      }

      this.copyPartOpacities(model);
    }
// Java
	public void updateParameters(final CubismModel model, float deltaTimeSeconds) {
      // Initialization is required when not the same as the previous model
      if (model ! = _lastModel) {
        reset(model);
      }
      _lastModel = model;
      
      // If the time is changed from the Settings, the elapsed time may be negative, so it is handled as 0 elapsed time.
      if (deltaTimeSeconds < 0.0f) {
        deltaTimeSeconds = 0.0f;
      }
      
      int beginIndex = 0;
      for (final int partGroupCount : _partGroupCounts) {
        doFade(model, deltaTimeSeconds, beginIndex, partGroupCount);
        beginIndex += partGroupCount;
      }
      copyPartOpacities(model);
    }

Linking opacity by Parent ID

The following function, called at the end of the Pose application process, propagates the value to the parts specified in the Link.

// C++
CubismPose::CopyPartOpacities()
// TypeScript
CubismPose.copyPartOpacities()
// Java
CubismPose.copyPartOpacities()

This Link is stored in an array of CubismIdHundle with the PartID expressed as Parent ID in OWViewer.
In the figure below, the Link of PartManteL001 would contain the CubismIdHundle of PartArmLB001 and PartArmLC001.

Note that although it is labeled Link, it is a parent-child hierarchy and will not be linked without an opacity operation on the parent part.
The children indicated in the Link are not linked to each other.

// C++
	void CubismPose::CopyPartOpacities(CubismModel* model)
    {
        for (csmUint32 groupIndex = 0; groupIndex < _partGroups.GetSize(); ++groupIndex)
        {
            PartData& partData = _partGroups[groupIndex];

            if (partData.Link.GetSize() == 0)
            {
                continue; // No linked parameters.
            }

            csmInt32    partIndex = _partGroups[groupIndex].PartIndex;
            csmFloat32  opacity = model->GetPartOpacity(partIndex);

            for (csmUint32 linkIndex = 0; linkIndex < partData.Link.GetSize(); ++linkIndex)
            {
                PartData&   linkPart = partData.Link[linkIndex];
                csmInt32    linkPartIndex = linkPart.PartIndex;

                if (linkPartIndex < 0)
                {
                    continue;
                }

                model->SetPartOpacity(linkPartIndex, opacity);
            }
        }
    }
// TypeScript
	public copyPartOpacities(model: CubismModel): void
    {
      for(let groupIndex: number = 0; groupIndex < this._partGroups.getSize(); ++groupIndex)
      {
        let partData: PartData = this._partGroups.at(groupIndex);

        if(partData.link.getSize() == 0)
        {
          continue;   // No linked parameters.
        }

        const partIndex: number = this._partGroups.at(groupIndex).partIndex;
        const opacity: number = model.getPartOpacityByIndex(partIndex);

        for(let linkIndex: number = 0; linkIndex < partData.link.getSize(); ++linkIndex)
        {
          let linkPart: PartData = partData.link.at(linkIndex);
          const linkPartIndex: number = linkPart.partIndex;

          if(linkPartIndex < 0)
          {
            continue;
          }

          model.setPartOpacityByIndex(linkPartIndex, opacity);
        }
      }
    }
// Java
	private void copyPartOpacities(CubismModel model) {
      for (PartData partData : _partGroups) {
        if (partData.linkedParameter == null) {
          continue;
        }
        
        final int partIndex = partData.partIndex;
        final float opacity = model.getPartOpacity(partIndex);
        
        for (PartData linkedPart : partData.linkedParameter) {
          final int linkedPartIndex = linkedPart.partIndex;
          
          if (linkedPartIndex < 0) {
            continue;
          }
          model.setPartOpacity(linkedPartIndex, opacity);
        }
      }
    }

Create an Instance (Import .pose3.json Files)

Poses are stored in a .pose3.json file and use the CubismPose class.
The following functions are used to create an instance.

// C++
CubismPose::Create()
// TypeScript
CubismPose.create()
// Java
CubismPose.create()

Implementation example

// C++
	csmString path = _modelSetting->GetPoseFileName();
    path = _modelHomeDir + path;

    buffer = CreateBuffer(path.GetRawString(), &size);
    CubismPose* pose = CubismPose::Create(buffer, size);
    DeleteBuffer(buffer, path.GetRawString());
// TypeScritp
	let path: string = _modelSetting.getPoseFileName();
    path = _modelHomeDire + path;

    fetch(path).then(
        (response) => 
        {
            return response.arrayBuffer();
        }
    ).then(
        (arrayBuffer) =>
        {
            let buffer: ArrayBuffer = arrayBuffer;
            let size: number = buffer.byteLength;

            let pose: CubismPose = CubismPose.create(buffer, size);
            deleteBuffer(buffer, path);
        }
    );
// Java
	String path = _modelSetting.getPoseFileName();
	path = _modelHomeDir + path;
	buffer = CreateBuffer(path);
	CubismPose pose = CubismPose.create(buffer);

Applying the Pose Function

The following functions are used to apply poses.

// C++
CubismPose::UpdateParameters()
// TypeScript
CubismPose.updateParameters()
// Java
CubismPose.updateParameters()

The application of motion and other virtual parameters must be completed in advance.

// C++
	void LAppModel::Update()
    {
        const csmFloat32 deltaTimeSeconds = LAppPal::GetDeltaTime();
        _userTimeSeconds += deltaTimeSeconds;

/* Omitted */

        //-----------------------------------------------------------------
        _model->LoadParameters(); // Load previously saved state.
        if (_motionManager->IsFinished())
        {
            // If there is no motion playback, playback is performed at random from among the standby motions.
            StartRandomMotion(MotionGroupIdle, PriorityIdle);
        }
        else
        {
            motionUpdated = _motionManager->UpdateMotion(_model, deltaTimeSeconds); // Update motion.
        }
        _model->SaveParameters(); // Save the state.
        //-----------------------------------------------------------------

/* Omitted */

        // Pose settings
        if (_pose ! = NULL)
        {
            _pose->UpdateParameters(_model, deltaTimeSeconds);
        }

        _model->Update();

    }
// TypeScript
	/**
     * Update
     */
    public update(): void
    {
        if(this._state ! = LoadStep.CompleteSetup) return;
        
        const deltaTimeSeconds: number = LAppPal.getDeltaTime();
        this._userTimeSeconds += deltaTimeSeconds;

/* Omitted */

        //--------------------------------------------------------------------------
        this._model.loadParameters();  // Load previously saved state.
        if(this._motionManager.isFinished())
        {
            // If there is no motion playback, playback is performed at random from among the standby motions.
            this.startRandomMotion(LAppDefine.MotionGroupIdle, LAppDefine.PriorityIdle);
            
        }
        else
        {
            motionUpdated = this._motionManager.updateMotion(this._model, deltaTimeSeconds);    // Update motion.
        }
        this._model.saveParameters(); // Save the state.
        //--------------------------------------------------------------------------

 /* Omitted */
        
        // Pose settings
        if(this._pose ! = null)
        {
            this._pose.updateParameters(this._model, deltaTimeSeconds);
        }

        this._model.update();
    }
// Java
	public void update() {
      final float deltaTimeSeconds = LAppPal.getDeltaTime();
      _userTimeSeconds += deltaTimeSeconds;

/* Omitted */
      
      // -------------------------------------------------
      _model.loadParameters(); // Load previously saved state.
      if (_motionManager.isFinished()) {
        // If there is no motion playback, playback is performed at random from among the standby motions.
        startRandomMotion(LAppDefine.MotionGroup.IDLE.getId(), LAppDefine.Priority.IDLE.getPriority());
      } else {
        isMotionUpdated = _motionManager.updateMotion(_model, deltaTimeSeconds);
      }
      _model.saveParameters(); // Save the state.
      // -------------------------------------------------
/* Omitted */

      if (_pose ! = null) {
        _pose.updateParameters(_model, deltaTimeSeconds);
      }
      _model.update();
    }

Discard

The CubismPose instance must also be destroyed at the time the model is released.

// C++
CubismPose::Delete(pose);
// TypeScript
CubismPose.delete(pose);

SDK for Java does not need to be disposed because garbage collection is responsible for deallocation.

Please let us know what you think about this article.