Expression (Cocos Creator)
Updated: 03/14/2023
Summary
Expression is a function that handles Cubism’s Expression Motion, which can be set by adding or multiplying the value of a parameter for an expression to the value of the current parameter.
Animator, Cocos Creator’s standard motion playback function, does not have Multiply as a Blend Mode that can be set for layers, so it is not possible to express this.
The settings file for the expression function is exported in .exp3.json format.
Click here for more information on the mechanism of facial expressions.
To use Expression, it is necessary to “How to Set Up UpdateController” and “Save/Restore the Values to Be Manipulated” in Prefab in advance.
Click here for the corresponding tutorial article.
To play back the Expression Motion, perform the following steps.
- Create [.exp3.asset]
- Create [.expressionList.asset] and add [.exp3.asset]
- Play back Expression Motion
- Calculate and apply Expression Motion
Create [.exp3.asset]
[.exp3.asset] is a ScriptableObject asset converted from [exp3.json].
Normal operation cannot be guaranteed if [.exp3.asset] is changed.
To convert [exp.json] to [.exp3.asset], do the following.
- Parse [exp.json]
- Create CubismExpressionData
- Create [.exp3.asset]
Parse [exp.json]
To parse [exp3.json], use CubismExp3Json.LoadFrom(string exp3Json) or CubismExp3Json.LoadFrom(TextAsset exp3JsonAsset).
Create CubismExpressionData
CubismExpressionData is a class that records information for the parsed .exp3.json and holds the following data.
Field | Type | Description |
---|---|---|
Type | string | Type of json file. For .exp3.json, it is set to “Live2D Expression.” |
FadeInTime | number | Time for the expression to fade in. The unit of value is seconds. |
FadeOutTime | number | Time for the expression to fade out. The unit of value is seconds. |
Parameters | SerializableExpressionParameter[] | ID of the parameter to which the expression is applied, the value to be applied, and the calculation method. |
Use CubismExpressionData.CreateInstance(CubismExp3Json json) to create a new [.exp3.asset].
Use CubismExpressionData.CreateInstance(CubismExpressionData expressionData, CubismExp3Json json) to overwrite [.exp3.asset].
- CubismExp3Json json: CubismExp3Json is parsed data of [exp3.json].
- CubismExpressionData expressionData: CubismExpressionData to be overwritten.
Create [.exp3.asset]
Create [.exp3.asset] as follows.
const jsonSrc = readFileSync(asset.source, UTF8); const json = CubismExp3Json.loadFrom(jsonSrc); if (json == null) { console.error('CubismExp3Json.loadFrom() is failed.'); return false; } const data = CubismExpressionData.createInstance(json); const serialized = EditorExtends.serialize(data); writeFileSync(outputFilePath, jsonSrc); asset.saveToLibrary('.json', serialized); refresh(outputFilePath);
This is done by CubismExpression3JsonImporter.import().
Note: For runtime, it is not necessary to .asset CubismExpressionData.
Create [.expressionList.asset] and add [.exp3.asset]
[.expressionList.asset] is an asset that lists references to [.exp3.asset] for each model.
This is used in CubismExpressionController to retrieve the expression motion to be played from CurrentExpressionIndex.
The order of the [.exp3.asset] list is added in the import order of [exp3.json].
Creating [.expressionList.asset]
The [.expressionList.asset] is created in the same hierarchy as the model prefab.
const source = JSON.parse(readFileSync(asset.source, UTF8)); const expDataUuidsSource = Array.isArray(source.cubismExpressionObjects) ? (source.cubismExpressionObjects as any[]) : undefined; if (expDataUuidsSource == null) { console.error('CubismExpressionListImporter.import(): parse error.'); return false; } const expDataUuids = purseCubismExpressionObjects(expDataUuidsSource); if (expDataUuids == null) { console.error('CubismExpressionListImporter.import(): CubismExpressionObjects parse error.'); return false; } const cubismExpressionObjects = new Array<CubismExpressionData>(expDataUuids.length); for (let i = 0; i < cubismExpressionObjects.length; i++) { const uuid = expDataUuids[i]; // @ts-ignore cubismExpressionObjects[i] = EditorExtends.serialize.asAsset( uuid, CubismExpressionData ) as CubismExpressionData; } const list = new CubismExpressionList(); list.cubismExpressionObjects = cubismExpressionObjects; await asset.saveToLibrary('.json', EditorExtends.serialize(list));
Note: This item is processed by CubismExpressionListImporter.import().
Adding [.exp3.asset]
To add [.exp3.asset] to [.expressionList.asset], do the following.
const jsonSrc = readFileSync(asset.source, 'utf8'); const json = CubismExp3Json.loadFrom(jsonSrc); if (json == null) { console.error('CubismExp3Json.loadFrom() is failed.'); return false; } const data = CubismExpressionData.createInstance(json); const serialized = EditorExtends.serialize(data); asset.saveToLibrary('.json', serialized); refresh(outputFilePath); const dirPath = Path.join(Path.dirname(asset.source), '../'); const dirName = Path.basename(dirPath); if (dirPath.length == 0) { console.warn('CubismExpressionDataImporter.import(): Not subdirectory.'); return true; } const expressionListFilePath = Path.join( dirPath, dirName + `.${CubismExpressionListImporter.extension}` ); updateExpressionListFile(expressionListFilePath, asset.uuid); refresh(expressionListFilePath);
Note: This item is processed by CubismExpressionDataImporter.import().
Play back Expression Motion
Expression motions can be played or changed by setting the index of the expression motion to be played to CubismExpressionController.CurrentExpressionIndex.
The following process is used to play back the expression motion.
- Set the end time of the expression motion currently playing
- Initialize new expression motions and add them to the playback list of expression motions
Set the end time of the expression motion currently playing
If there is an expression motion in playback, set the end time so that the expression motion in playback ends.
const playingExpressions = this._playingExpressions; if (playingExpressions.length > 0) { const playingExpression = playingExpressions[playingExpressions.length - 1]; playingExpression.expressionEndTime = playingExpression.expressionUserTime + playingExpression.fadeOutTime; playingExpressions[playingExpressions.length - 1] = playingExpression; }
This is done by CubismExpressionController.startExpression().
Initializing new expression motions and adding them to the playback list of expression motions
Create a CubismPlayingExpression with playback information for the facial expression motion.
CubismPlayingExpression holds the following information.
Field | Type | Description |
---|---|---|
Type | string | Type of json file. For .exp3.json, it is set to “Live2D Expression.” |
FadeInTime | number | Time for the expression to fade in. The unit of value is seconds. |
FadeOutTime | number | Time for the expression to fade out. The unit of value is seconds. |
Weight | number | Expression weights. Values range from 0 to 1. |
ExpressionUserTime | number | Elapsed time since the start of facial expression playback. The unit of value is seconds. |
ExpressionEndTime | number | The time at which the playback of the facial expression ends. The unit of value is seconds. |
Destinations | CubismParameter[] | An array of parameters to which the expression applies. |
Value | number[] | The value that the expression applies to the parameter. The number of elements and the order are the same as for Destinations. |
Blend | CubismParameterBlendMode[] | A calculation method in which expressions are applied to parameters. The number of elements and the order are the same as for Destinations. The calculation methods set are Override, Additive, and Multiply. |
CubismPlayingExpression can be created using CubismPlayingExpression.create(CubismModel model, CubismExpressionData expressionData).
- CubismModel model: A model that plays back expression motion.
- CubismExpressionData expressionData: The [exp3.asset] data of the expression motion to be played.
The created expression motion is added to the playback list as follows.
const playingExpression = CubismPlayingExpression.create( this._model, expressionsList.cubismExpressionObjects[this.currentExpressionIndex] ); if (playingExpression == null) { return; } // Add to PlayingExList. playingExpressions.push(palyingExpression);
This is done by CubismExpressionController.startExpression().
Calculate and apply Expression Motion
Expression calculation and application is called from CubismExpressionController.onLateUpdate().
Note: The expression must be applied after ParameterStore.saveParameters().
Expression is a function that applies a new value for expression each frame to the current value.
If applied before ParameterStore.saveParameters(), the result applied by the expression will be saved, so the values of the parameters for the expression will be added or multiplied indefinitely.