How to Use CubismWebFramework Directly

Updated: 01/30/2020

The sample project included in the Cubism SDK for Web handles Cubism models using the processes under the “/Framework” directory of the SDK package.
When creating a project that handles Cubism models, it is best to use the sample project as a basis, but there are cases where you may want to use the CubismWebFramework API directly for implementation in an existing application or in your own engine.
However, the included sample project is highly functional, so it is not easy to understand and isolate the structure just by referring to it.

The following is a small snippet that directly calls CubismFramework to handle models for users like those mentioned above.

CubismFramework Cycle

The procedure for working with CubismFramework is as follows:

1. Initialize CubismFramework
2. Get the path to the model file
3. Load the model
4. Update process
5. Discard the model
6. Exit process of CubismFramework

Initialize CubismFramework

The initialization process of CubismFramework is as follows:

CubismFramework is initialized using CubismFramework.startUp().
Cubism SDK for Native takes an allocator (ICubismAllocator) as its first argument, but Cubism SDK for Web does not use this.

    // Set logging and other options.
    _cubismOption: Csm_Option;

    // Message output function.
    public static printMessage(message: string): void

The contents of printMessage() should be implemented by referring to the function of the same name in the sample.

import {Live2DCubismFramework as live2dcubismframework, Option as Csm_Option, LogLevel} from "../../../../Framework/live2dcubismframework";
import Csm_CubismFramework = live2dcubismframework.CubismFramework;    
・
・
・
    // Set the level of log output. If LogLevel_Verbose, create a detailed log output.
    this._cubismOption.logFunction = printMessage;
    this._cubismOption.loggingLevel = LogLevel.LogLevel_Verbose;
    
    // Sets the parameters required to initialize CubismNativeFramework.
    Csm_CubismFramework.startUp(this._cubismOption);
    
    // Initialize CubismFramework.
    Csm_CubismFramework.initialize();

Obtain Model Files

The set of data for Cubism’s built-in data is described in .model3.json with their relative paths, and byte data is loaded by referring to this file.
It is recommended to parse .model3.json to obtain the path to model files, etc. and load them.

Parsing of .model3.json uses CubismFramework’s CubismModelSettingJson class.

The .moc3 files and textures can also be directly specified and loaded.
If you want to specify a file directly and have it read, code modification is required.

In the sample, fetch() is used in LAppModel.loadAssets() to retrieve a file from the resource path and read the byte data to get the Json content.

fetch(path).then(
    (response) =>
    {
        return response.arrayBuffer();
    }
).then(
    (arrayBuffer) =>
    {
        let buffer: ArrayBuffer = arrayBuffer;
        let size = buffer.byteLength;
        // Obtain model data paths, etc. described in Json
        let setting: ICubismModelSetting = new CubismModelSettingJson(buffer, size);

        // Save the result
        this.setupModel(setting);
    }
);

Load the model

The model is loaded through the CubismModel interface.

The CubismModel instance is held by CubismUserModel._model in CubismNativeFramework, and model data is retrieved and stored using CubismUserModel.loadModel().

When using CubismModel, it is recommended to handle it from a class that inherits from CubismUserModel.
In the sample, the LAppModel class inherits from and handles CubismUserModel.

Resource management for textures, motion, facial expression motion, etc. can also be done externally.

In the sample, fetch() is used in LAppModel.setupModel() to load model data from a file path stored in ICubismModelSetting.

// Get the model file path stored in ICubismModelSetting
let path: string = this._modelSetting.getModelFileName();
path = this._modelHomeDir + path;

fetch(path).then(
    (response) =>
    {
        return response.arrayBuffer();
    }
).then(
    (arrayBuffer) =>
    {
        buffer = arrayBuffer;
        // Load model data
        this.loadModel(buffer);
        deleteBuffer(buffer, path);
    }
);
this._state = LoadStep.WaitLoadModel;

The path to the .moc3 file to be loaded can be obtained in .model3.json as described above. Similarly, the path to facial expressions, physics, poses, eye blinking, lip-sync, user data, and motion can be obtained from .model3.json.
The SDK sample loads these at the same time as the model is loaded.
Please refer to the following for each loading method.

• Facial expression
• Physics
• Pose
• Eye blinking
• Lip-sync
• User data
• Motion

Update process

The update process for a Cubism model is accomplished by setting the target model by passing the parameters and values to be updated through the CubismModel interface, such as CubismModel.setParameterValueById(), and call CubismModel.update() to update.

Calling CubismModel.update() causes Cubism Core to perform an update process, updating vertex information and other information based on the values of parameters and parts that have been set up to that point.

By passing this updated vertex information, etc. to the renderer, the Cubism model can be drawn on the screen.

The following code directly manipulates model parameters in a class with _model(CubismModel) and calls Cubism Core’s update process to apply the operation.

public update(): void
{
    // Parameter operation.
    // Move ParamAngleX to value
    this._model.setParameterValueById(CubismFramework.getIdManager().getId("ParamAngleX"), value);

    // Part opacity operation.
    // Set Opacity of PartArmL to opacity
    this._model.setPartOpacityById(CubismFramework.getIdManager().getId("PartArmL"), opacity);

    // Update the model
    this._model.update();
}

The update process always performs eye tracking, physics, motion playback, etc. before CubismModel.update().

Manipulation of parameter values after CubismModel.update() is not reflected.
CubismModel.update() can be called again later to reflect the changes, but this process is highly burdensome, so we recommend that you combine them into one.

    // Reflected in vertices.
    _model.setParameterValue(CubismFramework.getIdManager().getId("ParamAngleX"), value1);
    
    // Update model vertex information.
    _model.update();
    
    // Not reflected in vertices.
    _model.setParameterValue(CubismFramework.getIdManager().getId("ParamAngleX"), value);

CubismMotionManager.updateMotion(), which plays the motion, overwrites any ID parameter values used in the motion to be played.
Therefore, any manipulation of parameter values prior to this process will be overwritten by CubismMotionManager.updateMotion().
It is recommended that value operations such as eye tracking and physics calculations be performed first, followed by motion playback processing.

    // Reflect the motion being played to the model.
    _motionManager.updateMotion(_model, deltaTimeSeconds);
    
    // Value operations such as eye tracking and physics processing
    
    // Update model vertex information.
    _model.update();

Also, if not all parameters are used in the motion to be played back, or if the parameter values are not manipulated because playback of the motion is stopped, the result of the value manipulation done in the previous frame will remain.
Therefore, the results of subsequent relative parameter value manipulations may be unintended.
By calling CubismModel.loadParameter() and CubismModel.saveParameter() before and after the process of applying motion to the model, this can reset any subsequent relative value operations.

    // Restore the values of all parameters to the state at the time of saving.
    _modelloadParameters();
    
    // Reflect the motion being played to the model.
    _motionManager.updateMotion(_model, deltaTimeSeconds);
    
    // Save the values of all parameters.
    _model.saveParameters();
    
    
    // Relative value manipulation processing
    
    
    // Update model vertex information.
    _model.update();

Please refer to the respective pages for information on eye tracking, physics calculations, motion playback, and other processes.

• Facial expression
• Physics
• Pose
• Eye blinking
• Lip-sync
• User data
• Motion

By passing this updated vertex information, etc. to the renderer, the Cubism model can be drawn on the screen.

Discard the model

To destroy the model, destroy the instance of the generated CubismUserModel derived class.
This destroys the motion, facial expressions, physics, and other information held by this model from the destructor of the CubismUserModel.

    // Discard model data.
    this._model.release();
    this._model = null;

Exit process of CubismFramework

To release common part resources allocated by CubismFramework, call CubismFramework.dispose().
Do not call CubismFramework.dispose() before calling CubismFramework.initialize().

    // Destroy CubismFramework.
    CubismFramework.dispose();