Automatic eye-blinking
Updated: 01/26/2023
Identify Parameters Used for Eye Blinking
The automatic eye blinking effect feature can be used to apply a random blink behavior to the model.
The following processing is performed to apply the automatic eye blinking effect.
• Mapping of the automatic eye blinking effect values described in the .model3.json file to the parameters to be applied
• Setting the time interval for eye blinking and calling the update process
Of these, the information that maps parameters to Automatic eye blinking effects described in the .model3.json file can be obtained by using the CubismModelSettingJson class, which implements the ICubismModelSetting interface.
// C++ csmInt32 eyeBlinkCount = _modelSetting->GetEyeBlinkParameterCount(); for (csmInt32 i = 0; i < eyeBlinkCount; i++) { CubismIdHandle eyeblinkParameter = _modelSetting->GetEyeBlinkParameterId(i); }
// TypeScript let eyeBlinkCount: number = _modelSetting.getEyeBlinkParameterCount(); for(let i: number = 0; i < eyeBlinkCount; i++) { let eyeBlinkParameter: CubismIdHandle = _modelSetting.getEyeBlinkParameterId(i); }
// Java int eyeBlinkCount = modelSetting.getEyeBlinkParameterCount(); for(int i = 0; i < eyeBlinkCount; i++) { CubismId eyeBlinkParameter = modelSetting.getEyeBlinkParameterId(i); }
Refer to “Eye Blinking Settings” for definitions in the .model3.json file.
After eye blinking and lip-sync settings are made in the Editor and then output, the .model3.json file will contain the following description.
{ ... Omitted ... "Groups": [ { "Target": "Parameter", "Name": "LipSync", "Ids": [ "ParamMouthOpenY" ] }, { "Target": "Parameter", "Name": "EyeBlink", "Ids": [ "ParamEyeLOpen", "ParamEyeROpen" ] } ] }
Use CubismEyeBlink
To actually make the model blink, use the CubismEyeBlink class.
This class provides, via the ICubismModelSetting interface,
random interval eye blinking behavior for the eye blinking parameters
described in the .model3.json file.
Creating CubismEyeBlink instance
To create a CubismEyeBlink instance, use the CubismEyeBlink::Create function in Native (C++) or the CubismEyeBlink.create function in Web (TypeScript) and Java.
By passing a CubismModelSettingJson instance as an argument, the settings in the .model3.json file are automatically retrieved.
If omitted, the eye blinking ID will be unspecified.
// C++ if (_modelSetting->GetEyeBlinkParameterCount() > 0) { _eyeBlink = CubismEyeBlink::Create(_modelSetting); }
// TypeScript if(_modelSetting.getEyeBlinkParameterCount() > 0) { _eyeBlink = CubismEyeBlink.create(_modelSetting); }
// Java if(modelSetting.getEyeBlinkParameterCount() > 0) { eyeBlink = CubismEyeBlink.create(modelSetting); }
Eye blinking settings
The eye blinking time can be specified using the CubismEyeBlink::SetBlinkingInterval function in Native (C++), CubismEyeBlink.setBlinkingInterval function in Web (TypeScript) and Java, CubismEyeBlink::SetBlinkingSettings function in Native (C++), or CubismEyeBlink.setBlinkingSettings function in Web (TypeScript)and Java.
The time between blinks set by the SetBlinkingInterval function is random, ranging from 0 seconds to twice the set time.
// C++ _eyeBlink->SetBlinkingInterval(6.0f); _eyeBlink->SetBlinkingSettings(0.8f, 0.2f, 0.8f);
// TypeScript _eyeBlink.setBlinkingInterval(8.0); _eyeBlink.SetBlinkingSettings(0.8, 0.2, 0.8);
// Java eyeBlink.setBlinkingInterval(8.0f); eyeBlink.setBlinkingSetting(0.8f, 0.2f, 0.8f);
You can also use the CubismEyeBlink::SetParameterIds function in Native (C++), CubismEyeBlink.setParameterIds function in Web (TypeScript) and Java, CubismEyeBlink:: GetParameterIds function in Native (C++), or CubismEyeBlink.getParameterIds function in Web (TypeScript) and Java to change the eye blinking parameters later.
// C++ csmVector<CubismIdHandle> eyeBlinkList = _eyeBlink->GetParameterIds(); // Exclude parameters for right eye open/close (ParamEyeROpen) CubismIdHandle removeTarget = CubismFramework::GetIdManager()->GetId("ParamEyeROpen"); for (csmUint32 i = 0; i < eyeBlinkList.GetSize(); ++i) { if(eyeBlinkList[i] == removeTarget) { eyeBlinkList.Remove(i); break; } } // Add parameter for third eye open/close (ParamEyeThirdOpen) eyeBlinkList.PushBack(CubismFramework::GetIdManager()->GetId("ParamEyeThirdOpen")); // Overwrite to parameters for eye blinking. The right eye no longer opens and closes, and the newly created third eye blinks instead. _eyeBlink->SetParameterIds(eyeBlinkList);
// TypeScript let eyeBlinkList: csmVector<CubismIdHandle> = _eyeBlink.getParamterIds(); // Exclude parameters for right eye open/close (ParamEyeROpen) let removeTarget: CubismIdHandle = CubismFramework.getIdManager().getId("ParamEyeROpen"); for(let i: number = 0; i < eyeBlinkLIst.getSize(); ++i) { if(eyeBlinkList[i] == removeTarget) { eyeBlinkList.remove(i); break; } } // Add parameter for third eye open/close (ParamEyeThirdOpen) eyeBlinkList.pushBack(CubismFramework.getIdManager().getId("ParamEyeThirdOpen")); // Overwrite to parameters for eye blinking. The right eye no longer opens and closes, and the newly created third eye blinks instead. _eyeBlink.setParameterIds(eyeBlinkList);
// Java List<CubismId> eyeBlinkList = new ArrayList<CubismId>(eyeBlink.getParameterIds()); // Exclude parameters for right eye open/close (ParamEyeROpen) CubismId removeTarget = CubismFramework.getIdManager().getId("ParamEyeROpen"); boolean isRemoved = eyeBlinkList.remove(removeTarget); // If the parameter for right eye open/close exists and was correctly removed, add a parameter for third eye open/close (ParamEyeThirdOpen) if (isRemoved) { eyeBlinkList.add(CubismFramework.getIdManager().getId("ParamEyeThirdOpen")); } // Overwrite to parameters for eye blinking. The right eye no longer opens and closes, and the newly created third eye blinks instead. eyeBlink.setParameterIds(eyeBlinkList);
Apply to Models
To apply the eye blinking, use the CubismEyeBlink::UpdateParameters function in Native (C++) or CubismEyeBlink.updateParameters function in Web (TypeScript) and Java, and put the target model in the first argument and the difference time since the last update in the second argument.
// C++ if (_eyeBlink ! = NULL) { _eyeBlink->UpdateParameters(_model, deltaTimeSeconds); }
// TypeScript if(_eyeBlink ! = null) { _eyeBlink.updateParameters(_model, deltaTimeSeconds); }
// Java if(eyeBlink ! = null) { eyeBlink.updateParameters(model, deltaTimeSeconds); }
Discard
The CubismEyeBlink instance must also be destroyed at the time the model is released.
// C++ CubismEyeBlink::Delete(_eyeBlink);
// TypeScript CubismEyeBlink.delete(_eyeBlink);
Java has a policy of letting garbage collection do the deallocation, so there is no need for a disposal procedure.