About Models (Web)

Updated: 11/07/2019

Edit Model Information

Basically, model information is created in Modeler.
The movement of vertices and other objects relative to the parameters is recorded in the .moc3 file.
Other physics operations and user data attached to the ArtMesh are output as separate files. The .model3.json file is what keeps track of all file references related to the model.

You can output .moc3 files from Modeler all at once. “Exporting Data for Embedded Use”
You can also use Cubism Viewer (for OW) to add built-in settings for motion and facial expression files and pose files.

Import from .model3.json File Using Framework

When loading using the Framework, it is assumed that the necessary information for the model is extracted from the .model3.json file and an instance inheriting from the CubismUserModel class is maintained for use.

// TypeScript
const dir: string = "example/";
const fileName: string = "example.model3.json";
const path: string = dir + fileName;

fetch(path).then(
    (response) =>
    {
        return response.arrayBuffer();
    }
).then(
    (arrayBuffer) =>
    {
        let buffer: ArrayBuffer = arrayBuffer;
        let size = buffer.byteLength;
        let setting: ICubismModelSetting = new CubismModelSettingJson(buffer, size);
        deleteBuffer(buffer, path);
// Omitted
    }

Each element extracted in the ICubismModelSetting class can be loaded with the CubismUserModel.load~~ system function.

// TypeScript

// buffer is .model3.json’s ArrayBuffer
this.loadModel(buffer);

The overall loading flow from the .model3.json file can be easily grasped by following the flow from the LAppModel.loadAssets function in the sample to LAppModel.setupModel function, CubismUserModel.createRenderer function, and LAppModel.setupTextures function.

Create an Instance (Import .moc3 Files)

First, load the .moc3 file into memory.
Pass the read buffer and size to the CubismMoc.create function to first create a CubismMoc instance.
Next, call the CubismMoc.createModel function to create a CubismModel instance.
From this CubismModel instance, the user manipulates parameters and acquires information for drawing.

// TypeScript

// buffer is an ArrayBuffer of .moc3
this._moc = CubismMoc.create(buffer);
this._model = this._moc.createModel();

Graphic Environment Associations

The Framework uses classes derived from the CubismRenderer class to make the model texture information independent of the graphics API.
All graphics-related information associated with the model is managed by classes derived from the CubismRenderer class.
By generating classes derived from the CubismRenderer class for the graphics API and registering a CubismModel instance with the CubismRenderer_WebGL.initialize function, CubismRenderer and CubismModel instances are linked.

// TypeScript
let renderer: CubismRenderer_WebGL = new CubismRenderer_WebGL();

renderer.initialize(model);

Associate Textures

The textures that models have in the Framework are managed by a class derived from the CubismRenderer class.
However, that functionality avoids dependence on the graphics API, so its methods are not registered in the CubismRenderer class.
Note the change in method specifications depending on the target graphics API.
The WebGL process uses the CubismRenderer_WebGL.bindTexture function in the CubismRenderer_WebGL class, which is derived from the CubismRenderer class, to register the textures.
The first argument is the texture number of the model, identified in the Editor by the number of the texture atlas.
The second argument is the texture’s WebGL management number.

The example below shows texture loading in WebGL.

// TypeScript
// Load texture into WebGL texture unit
let texturePath = this._modelSetting.getTextureFileName(modelTextureNumber);
texturePath = this._modelHomeDir + texturePath;

img[modelTextureNumber] = new Image();
img[modelTextureNumber].onload = () =>
{
    // Create texture objects.
    let tex: WebGLTexture = gl.createTexture();

    // Select texture.
    gl.bindTexture(gl.TEXTURE_2D, tex);

    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_LINEAR);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

    // Let Premult process
    if(usePremultiply)
    {
        gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, 1);
    }

    // Write pixels to texture.
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA,
                  gl.UNSIGNED_BYTE, img[modelTextureNumber]);

    // Generate mipmap.
    gl.generateMipmap(gl.TEXTURE_2D);

    this.getRenderer().bindTexture(modelTextureNumber, tex);
}
img[modelTextureNumber].src = texturePath; 

this.getRenderer().setIsPremultipliedAlpha(usePremultiply);

Specify Display Position and Size

With the previous settings, the model can be drawn, but in many cases, the display position and scale are too different to be shown on the screen as is.

See “DrawableVertexPositions Range” for the vertex range returned by the csmGetDrawableVertexPositions function.
To adjust the size, use the CubismModelMatrix class and the CubismModel.getCanvasWidth and getCanvasHeight functions.

// TypeScript
let modelMatrix: CubismModelMatrix =
  new CubismModelMatrix(this._model.getCanvasWidth(),
                        this._model.getCanvasHeight());

This matrix is multiplied by the Projection matrix before drawing and passed to the renderer as an MVP matrix.

// TypeScript
let projectionMatrix: CubismMatrix44 = new CubismMatrix44();
projectionMatrix.scale(1, window.width / window.height);
projectionMatrix.multiplyByMatrix(modelMatrix);

this.getRenderer().setMvpMatrix(projectionMatrix);
this.getRenderer().drawModel();

Update Vertices

Execute the CubismModel.update function to reflect parameter operations in the CubismModel instance to the vertices of the ArtMesh.

// TypeScript
model.update();

Draw

Drawing is not executed from the CubismModel instance, but by commanding the renderer associated with it.

// TypeScript
let projectionMatrix: CubismMatrix44 = new CubismMatrix44();
projectionMatrix.scale(1, window.width / window.height);
projectionMatrix.multiplyByMatrix(modelMatrix);

this.getRenderer().setMvpMatrix(projectionMatrix);

this.getRenderer().drawModel();

Use Parameter IDs that Do Not Exist in the Model in the CubismModel Class

In the operations of the CubismModel class parameters and part opacity, there is a functionality to handle IDs that do not exist in the .moc3 file.
This functionality is used by the CubismMotion class, the CubismPose class, and other classes, and if creating a new mechanism using this function, be careful to use an ID that does not conflict with existing functions.

Also, when reading the Framework, keep in mind that it may be linked to other functions using created parameters.
Note: Accessing a non-existent maximum, minimum, or initial value will result in an error.

// TypeScript
// CubismModel
public getParameterIndex(parameterId: CubismIdHandle): number
{
    let parameterIndex: number;
    const idCount: number = this._model.parameters.count;

    for(parameterIndex = 0; parameterIndex < idCount; ++parameterIndex)
    {
        if(parameterId ! = this._parameterIds.at(parameterIndex))
        {
            continue;
        }

        return parameterIndex;
    }

    // If not present in the model, search in the non-existent parameter ID list and return its index.
    if(this._notExistParameterId.isExist(parameterId))
    {
        return this._notExistParameterId.getValue(parameterId);
    }

    // Add a new element if it is not in the non-existent parameter ID list
    parameterIndex = this._model.parameters.count + this._notExistParameterId.getSize();

    this._notExistParameterId.setValue(parameterId, parameterIndex);
    this._notExistParameterValues.appendKey(parameterIndex);

    return parameterIndex;
}