About Expression Motion
Updated: 01/26/2023
What is Expression Motion?
Expression Motion is an SDK feature that sets values for expressions relative to the current state, in addition to normal motion.
The Motion Manager’s fade function can be used to change over time from the current facial expression to a new facial expression setting that is played back.
Another feature not found in motion is the ability to specify the calculation method.
Since it inherits from the ACubismMotion class, it can be managed by the CubismMotionManager class just like normal motion.
To create expression data (.exp3.json file), the .motion3.json file must be loaded into Cubism Viewer (for OW) as an Expression Motion, converted and output.
When the .motion3.json file is loaded, the initial value of the motion is set as the target value of the facial expression.
“Create Facial Expressions in Animation View”
“Expression Settings and Export”
Expression has the following differences when compared to motion.
- There is no change in value over time.
- It cannot affect the parts.
- You can specify the calculation method (add, multiply, or overwrite).
Create an Instance (Import .exp3.json)
Expression is reproduced using the CubismExpressionMotion class derived from the ACubismMotion class.
To create a CubismExpressionMotion instance, use the CubismExpressionMotion::Create function in Native (C++) or the CubismExpressionMotion.create function in Web (TypeScript) and Java.
// C++
csmInt32 len = _modelSetting->GetExpressionCount();
for (csmInt32 i = 0; i < len; ++i)
{
csmString name = _modelSetting->GetExpressionName(i);
csmString path = _modelSetting->GetExpressionFileName(i);
path = _modelHomeDir + path;
buffer = CreateBuffer(path.GetRawString(), &size);
ACubismMotion* motion = CubismExpressionMotion::Create(buffer, size);
if (_expressions[name] ! = NULL)
{
ACubismMotion::Delete(_expressions[name]);
_expressions[name] = NULL;
}
_expressions[name] = motion;
DeleteBuffer(buffer, path.GetRawString());
}
// TypeScript
let len: number = _modelSetting.getExpressionCount();
for(let i: number = 0; i < len; ++i)
{
let name: string = this._modelSetting.getExpressionName(i);
let path: string = this._modelSetting.getExpressionFileName(i);
path = this._modelHomeDir + path;
fetch(path).then(
(response) =>
{
return response.arrayBuffer();
}
).then(
(arrayBuffer) =>
{
let buffer: ArrayBuffer = arrayBuffer;
let size: number = buffer.byteLength;
let motion: ACubismMotion = CubismExpressionMotion.create(buffer, size);
if(this._expressions.getValue(name) ! = null)
{
ACubismMotion.delete(this._expressions.getValue(name));
this._expressions.setValue(name, null);
}
this._expressions.setValue(name, motion);
deleteBuffer(buffer, path);
}
);
}
// Java
final int count = modelSetting.getExpressionCount();
for (int i = 0; i < count; i++) {
String name = modelSetting.getExpressionName(i);
String path = modelSetting.getExpressionFileName(i);
path = modelHomeDirectory + path;
byte[] buffer = createBuffer(path);
CubismExpressionMotion motion = loadExpression(buffer);
if (expression.get(name) ! = null) {
expressions.remove(name);
}
expressions.put(name, motion);
}
Create Instances for Expression Management
The CubismMotionManager class is used to apply facial expressions.
However, it must be created as a separate instance in order to use the normal motion and effect together.
// C++
CubismMotionManager* motionManager = CSM_NEW CubismMotionManager();
CubismMotionManager* expressionManager = CSM_NEW CubismMotionManager();
// TypeScript
let motionManager: CubismMotionmanager = new CubismMotionManager();
let expressionManager: CubismMotionManager = new CubismMotionManager();
// Java
CubismMotionManager motionManager = new CubismMotionManager();
CubismMotionManager expressionManager = new CubismMotionManager();
Playback of Facial Expressions and Its Mechanism
As with motion playback, facial expression playback involves issuing a playback command to the management class.
// C++
ACubismMotion* motion = _expressions[expressionID];//loaded CubismExpression
if (motion ! = NULL)
{
expressionManager->StartMotionPriority(motion, false, PriorityForce);
}
// TypeScript
let motion: ACubismMotion = _expressions[expressionID];//loaded CubismExpressionMotion
if(motion ! = null)
{
expressionManager.startMotionPriority(motion, false, PriorityForce);
}
// Java
ACubismMotion motion = expressions.get(expressionID); // loaded CubismExpression
if (motion ! = null){
expressionManager.startMotionPriority(motion, LAppDefine.Priority.FORCE.getPriority());
}
The calculation method can be specified for each parameter of the expression.
There are three calculation methods: addition, multiplication, and overwrite.
Additive
This method is selected when the calculation method is indicated as “Add” or when no calculation method is specified.
The values shown in the .exp3.json file are added to the parameter values as they are when applied.
Use for parameters that are not often manipulated in motion or
when you want to affect them by sliding as is.
When exporting values in Cubism Viewer (for OW), the difference from the initial value to the set value of the parameter is set.
Multiply
This method is selected when the calculation method is indicated as “Multiply.”
The value of the parameter at the time of application is multiplied by the value shown in the .exp3.json file as it is.
Note that this multiplication has no effect on the addition or overwriting after it is applied.
This is useful for amplifying or reducing the motion as it is.
When exporting values in Cubism Viewer (for OW), the set values are set as they are.
If you multiply a -30 to 30 motion by 30, you end up with a -900 to 900 motion,
so be careful to set the combination of motion and facial expression values.
Overwrite
This cannot be set in Cubism Viewer (for OW), but can be used by specifying “Overwrite” as the calculation method.
The value of the parameter at the time of application is ignored and the value is overwritten as is.
This is useful when you want to ignore motion and other movements.
Differences in expression due to different arithmetic methods
This is an example of opening and closing both eyes with a set value of 0.4 and applying the Add, Multiply, and Overwrite expressions, respectively.
With Add, the eye remains closed for a while because it is -0.6 for every drawing, so it quickly reaches the lower limit of open/close and is overwritten to the lower limit value.
With Multiply, the eye blinking works nicely because the entire waveform is multiplied by the value.
Overwrite ignores eye blinking and does not blink.
When combined with motion, etc., nuanced expression can be set with a single calculation of facial expression.
Apply Facial Expressions
Facial expressions can be applied to the model using the CubismMotionManager::UpdateMotion function in Native (C++) of the management class, the CubismMotionManager.updateMotion function in the Web (TypeScript) and Java, the MotionQueueManager::DoUpdateMotion function in Native (C++), or the CubismMotionQueueManager.doUpdateMotion function in Web (TypeScript) and Java.
// C++ expressionManager->UpdateMotion(_model, deltaTimeSeconds);
// TypeScript expressionManager.updateMotion(_model, deltaTimeSeconds);
// Java
expressionManager.updateMotion(model, deltaTimeSeconds);
The order of computation of motion, facial expression, and other parameter manipulation elements greatly affects the expression.
“About Parameters: The importance of the order in which parameters are calculated”
Pay close attention to the order of application.