About Expression Motion

Updated: 10/06/2022

What is Expression Motion?

Expression Motion is an SDK feature that sets values for expressions relative to the current state, in addition to normal motion.
The Motion Manager’s fade function can be used to change over time from the current facial expression to a new facial expression setting that is played back.
Another feature not found in motion is the ability to specify the calculation method.

Since it inherits from the ACubismMotion class, it can be managed by the CubismMotionManager class just like normal motion.
To create expression data (.exp3.json file), the .motion3.json file must be loaded into Cubism Viewer (for OW) as an Expression Motion, converted and output.
When the .motion3.json file is loaded, the initial value of the motion is set as the target value of the facial expression.
“Create Facial Expressions in Animation View”
“Expression Settings and Export”

Expression has the following differences when compared to motion.

  • There is no change in value over time.
  • It cannot affect the parts.
  • You can specify the calculation method (add, multiply, or overwrite).

Create an Instance (Import .exp3.json)

Expression is reproduced using the CubismExpressionMotion class derived from the ACubismMotion class.
To create a CubismExpressionMotion instance, use the CubismExpressionMotion::Create function in Native (C++) or the CubismExpressionMotion.create function in Web (TypeScript) and Java.

// C++
	csmInt32 len = _modelSetting->GetExpressionCount();
    for (csmInt32 i = 0; i < len; ++i)
    {
        csmString name = _modelSetting->GetExpressionName(i);
        csmString path = _modelSetting->GetExpressionFileName(i);
        path = _modelHomeDir + path;

        buffer = CreateBuffer(path.GetRawString(), &size);
        ACubismMotion* motion = CubismExpressionMotion::Create(buffer, size);

        if (_expressions[name] ! = NULL)
        {
            ACubismMotion::Delete(_expressions[name]);
            _expressions[name] = NULL;
        }
        _expressions[name] = motion;

        DeleteBuffer(buffer, path.GetRawString());
    }
// TypeScript
	let len: number = _modelSetting.getExpressionCount();

    for(let i: number = 0; i < len; ++i)
    {
        let name: string = this._modelSetting.getExpressionName(i);
        let path: string = this._modelSetting.getExpressionFileName(i);
        path = this._modelHomeDir + path;

        fetch(path).then(
            (response) => 
            {
                return response.arrayBuffer();
            }
        ).then(
            (arrayBuffer) =>
            {
                let buffer: ArrayBuffer = arrayBuffer;
                let size: number = buffer.byteLength;

                let motion: ACubismMotion = CubismExpressionMotion.create(buffer, size);

                if(this._expressions.getValue(name) ! = null)
                {
                    ACubismMotion.delete(this._expressions.getValue(name));
                    this._expressions.setValue(name, null);
                }

                this._expressions.setValue(name, motion);

                deleteBuffer(buffer, path);
            }
        );
    }
// Java
    final int count = _modelSetting.getExpressionCount();

    for (int i = 0; i < count; i++) {
        String name = _modelSetting.getExpressionName(i);
        String path = _modelSetting.getExpressionFileName(i);
        path = _modelHomeDirectory + path;
      
        byte[] buffer = createBuffer(path);
        CubismExpressionMotion motion = loadExpression(buffer);
      
        if (_expression.get(name) ! = null) {
            _expressions.remove(name);
        }
        _expressions.put(name, motion);
    }

Create Instances for Expression Management

The CubismMotionManager class is used to apply facial expressions.
However, it must be created as a separate instance in order to use the normal motion and effect together.

// C++
    CubismMotionManager* motionManager = CSM_NEW CubismMotionManager();

    CubismMotionManager* expressionManager = CSM_NEW CubismMotionManager();
// TypeScript
    let motionManager: CubismMotionmanager = new CubismMotionManager();

    let expressionManager: CubismMotionManager = new CubismMotionManager();
// Java
    CubismMotionManager motionManager = new CubismMotionManager();

    CubismMotionManager expressionManager = new CubismMotionManager();

Playback of Facial Expressions and Its Mechanism

As with motion playback, facial expression playback involves issuing a playback command to the management class.

// C++
    ACubismMotion* motion = _expressions[expressionID];//loaded CubismExpression
    if (motion ! = NULL)
    {
        expressionManager->StartMotionPriority(motion, false, PriorityForce);
    }
// TypeScript
    let motion: ACubismMotion = _expressions[expressionID];//loaded CubismExpressionMotion
    if(motion ! = null)
    {
        expressionManager.startMotionPriority(motion, false, PriorityForce);
    }
// Java
    ACubismMotion motion = _expressions.get(expressionID); // loaded CubismExpression
    if (motion ! = null){
        expressionManager.startMotionPriority(motion, LAppDefine.Priority.FORCE.getPriority());
    }

The calculation method can be specified for each parameter of the expression.
There are three calculation methods: addition, multiplication, and overwrite.

Additive

This method is selected when the calculation method is indicated as “Add” or when no calculation method is specified.
The values shown in the .exp3.json file are added to the parameter values as they are when applied.
Use for parameters that are not often manipulated in motion or
when you want to affect them by sliding as is.
When exporting values in Cubism Viewer (for OW), the difference from the initial value to the set value of the parameter is set.

Multiply

This method is selected when the calculation method is indicated as “Multiply.”
The value of the parameter at the time of application is multiplied by the value shown in the .exp3.json file as it is.
Note that this multiplication has no effect on the addition or overwriting after it is applied.
This is useful for amplifying or reducing the motion as it is.
When exporting values in Cubism Viewer (for OW), the set values are set as they are.
If you multiply a -30 to 30 motion by 30, you end up with a -900 to 900 motion,
so be careful to set the combination of motion and facial expression values.

Overwrite

This cannot be set in Cubism Viewer (for OW), but can be used by specifying “Overwrite” as the calculation method.
The value of the parameter at the time of application is ignored and the value is overwritten as is.
This is useful when you want to ignore motion and other movements.

Differences in expression due to different arithmetic methods

This is an example of opening and closing both eyes with a set value of 0.4 and applying the Add, Multiply, and Overwrite expressions, respectively.
With Add, the eye remains closed for a while because it is -0.6 for every drawing, so it quickly reaches the lower limit of open/close and is overwritten to the lower limit value.
With Multiply, the eye blinking works nicely because the entire waveform is multiplied by the value.
Overwrite ignores eye blinking and does not blink.

When combined with motion, etc., nuanced expression can be set with a single calculation of facial expression.

Apply Facial Expressions

Facial expressions can be applied to the model using the CubismMotionManager::UpdateMotion function in Native (C++) of the management class, the CubismMotionManager.updateMotion function in the Web (TypeScript) and Java, the MotionQueueManager::DoUpdateMotion function in Native (C++), or the CubismMotionQueueManager.doUpdateMotion function in Web (TypeScript) and Java.

// C++
	expressionManager->UpdateMotion(_model, deltaTimeSeconds);
// TypeScript
	expressionManager.updateMotion(_model, deltaTimeSeconds);
// Java
    expressionManager.updateMotion(_model, deltaTimeSeconds);

The order of computation of motion, facial expression, and other parameter manipulation elements greatly affects the expression.

“About Parameters: The importance of the order in which parameters are calculated”

Pay close attention to the order of application.

Please let us know what you think about this article.