Automatic eye-blinking

Updated: 01/26/2023

Identify Parameters Used for Eye Blinking

The automatic eye blinking effect feature can be used to apply a random blink behavior to the model.
The following processing is performed to apply the automatic eye blinking effect.

• Mapping of the automatic eye blinking effect values described in the .model3.json file to the parameters to be applied

• Setting the time interval for eye blinking and calling the update process

Of these, the information that maps parameters to Automatic eye blinking effects described in the .model3.json file can be obtained by using the CubismModelSettingJson class, which implements the ICubismModelSetting interface.

// C++
csmInt32 eyeBlinkCount = _modelSetting->GetEyeBlinkParameterCount();
for (csmInt32 i = 0; i < eyeBlinkCount; i++)
{
   CubismIdHandle eyeblinkParameter = _modelSetting->GetEyeBlinkParameterId(i);
}

// TypeScript
let eyeBlinkCount: number = _modelSetting.getEyeBlinkParameterCount();
for(let i: number = 0; i < eyeBlinkCount; i++)
{
    let eyeBlinkParameter: CubismIdHandle = _modelSetting.getEyeBlinkParameterId(i);
}

// Java
int eyeBlinkCount = modelSetting.getEyeBlinkParameterCount();
for(int i = 0; i < eyeBlinkCount; i++) {
    CubismId eyeBlinkParameter = modelSetting.getEyeBlinkParameterId(i);
}

Refer to “Eye Blinking Settings” for definitions in the .model3.json file.
After eye blinking and lip-sync settings are made in the Editor and then output, the .model3.json file will contain the following description.

{
    ... Omitted ...
       "Groups": [
                {
                        "Target": "Parameter",
                        "Name": "LipSync",
                        "Ids": [
                                "ParamMouthOpenY"
                        ]
                },
                {
                        "Target": "Parameter",
                        "Name": "EyeBlink",
                        "Ids": [
                                "ParamEyeLOpen",
                                "ParamEyeROpen"
                        ]
                }
        ]
}

Use CubismEyeBlink

To actually make the model blink, use the CubismEyeBlink class.
This class provides, via the ICubismModelSetting interface,
random interval eye blinking behavior for the eye blinking parameters
described in the .model3.json file.

Creating CubismEyeBlink instance

To create a CubismEyeBlink instance, use the CubismEyeBlink::Create function in Native (C++) or the CubismEyeBlink.create function in Web (TypeScript) and Java.
By passing a CubismModelSettingJson instance as an argument, the settings in the .model3.json file are automatically retrieved.
If omitted, the eye blinking ID will be unspecified.

// C++
if (_modelSetting->GetEyeBlinkParameterCount() > 0)
{
    _eyeBlink = CubismEyeBlink::Create(_modelSetting);
}

// TypeScript
if(_modelSetting.getEyeBlinkParameterCount() > 0)
{
    _eyeBlink = CubismEyeBlink.create(_modelSetting);
}

// Java
if(modelSetting.getEyeBlinkParameterCount() > 0) {
    eyeBlink = CubismEyeBlink.create(modelSetting);
}

Eye blinking settings

The eye blinking time can be specified using the CubismEyeBlink::SetBlinkingInterval function in Native (C++), CubismEyeBlink.setBlinkingInterval function in Web (TypeScript) and Java, CubismEyeBlink::SetBlinkingSettings function in Native (C++), or CubismEyeBlink.setBlinkingSettings function in Web (TypeScript)and Java.
The time between blinks set by the SetBlinkingInterval function is random, ranging from 0 seconds to twice the set time.

// C++
	_eyeBlink->SetBlinkingInterval(6.0f);
    _eyeBlink->SetBlinkingSettings(0.8f, 0.2f, 0.8f);

// TypeScript
	_eyeBlink.setBlinkingInterval(8.0);
	_eyeBlink.SetBlinkingSettings(0.8, 0.2, 0.8);

// Java
    eyeBlink.setBlinkingInterval(8.0f);
    eyeBlink.setBlinkingSetting(0.8f, 0.2f, 0.8f);

You can also use the CubismEyeBlink::SetParameterIds function in Native (C++), CubismEyeBlink.setParameterIds function in Web (TypeScript) and Java, CubismEyeBlink:: GetParameterIds function in Native (C++), or CubismEyeBlink.getParameterIds function in Web (TypeScript) and Java to change the eye blinking parameters later.

// C++
csmVector<CubismIdHandle> eyeBlinkList = _eyeBlink->GetParameterIds();
 
// Exclude parameters for right eye open/close (ParamEyeROpen)
CubismIdHandle removeTarget = CubismFramework::GetIdManager()->GetId("ParamEyeROpen");
for (csmUint32 i = 0; i < eyeBlinkList.GetSize(); ++i)
{
    if(eyeBlinkList[i] == removeTarget)
    {
        eyeBlinkList.Remove(i);
        break;
    }
}
 
// Add parameter for third eye open/close (ParamEyeThirdOpen)
eyeBlinkList.PushBack(CubismFramework::GetIdManager()->GetId("ParamEyeThirdOpen"));
 
// Overwrite to parameters for eye blinking.　The right eye no longer opens and closes, and the newly created third eye blinks instead.
_eyeBlink->SetParameterIds(eyeBlinkList);

// TypeScript
let eyeBlinkList: csmVector<CubismIdHandle> = _eyeBlink.getParamterIds();
 
// Exclude parameters for right eye open/close (ParamEyeROpen)
let removeTarget: CubismIdHandle = CubismFramework.getIdManager().getId("ParamEyeROpen");
for(let i: number = 0; i < eyeBlinkLIst.getSize(); ++i)
{
    if(eyeBlinkList[i] == removeTarget)
    {
        eyeBlinkList.remove(i);
        break;
    }
}
 
// Add parameter for third eye open/close (ParamEyeThirdOpen)
eyeBlinkList.pushBack(CubismFramework.getIdManager().getId("ParamEyeThirdOpen"));
 
// Overwrite to parameters for eye blinking.　The right eye no longer opens and closes, and the newly created third eye blinks instead.
_eyeBlink.setParameterIds(eyeBlinkList);

// Java
List<CubismId> eyeBlinkList = new ArrayList<CubismId>(eyeBlink.getParameterIds());

// Exclude parameters for right eye open/close (ParamEyeROpen)
CubismId removeTarget = CubismFramework.getIdManager().getId("ParamEyeROpen");
boolean isRemoved = eyeBlinkList.remove(removeTarget);

// If the parameter for right eye open/close exists and was correctly removed, add a parameter for third eye open/close (ParamEyeThirdOpen)
if (isRemoved) {
    eyeBlinkList.add(CubismFramework.getIdManager().getId("ParamEyeThirdOpen"));
}

// Overwrite to parameters for eye blinking. The right eye no longer opens and closes, and the newly created third eye blinks instead.
eyeBlink.setParameterIds(eyeBlinkList);

Apply to Models

To apply the eye blinking, use the CubismEyeBlink::UpdateParameters function in Native (C++) or CubismEyeBlink.updateParameters function in Web (TypeScript) and Java, and put the target model in the first argument and the difference time since the last update in the second argument.

// C++
if (_eyeBlink ! = NULL)
{
    _eyeBlink->UpdateParameters(_model, deltaTimeSeconds); 
}

// TypeScript
if(_eyeBlink ! = null)
{
    _eyeBlink.updateParameters(_model, deltaTimeSeconds);
}

// Java
if(eyeBlink ! = null) {
    eyeBlink.updateParameters(model, deltaTimeSeconds);
}

Discard

The CubismEyeBlink instance must also be destroyed at the time the model is released.

// C++
CubismEyeBlink::Delete(_eyeBlink);

// TypeScript
CubismEyeBlink.delete(_eyeBlink);

Java has a policy of letting garbage collection do the deallocation, so there is no need for a disposal procedure.