About Models (Java)

Updated: 01/26/2023

Edit Model Information

Model information is typically created in the Model Workspace of the Cubism Editor.
The movement of vertices and other objects relative to the parameters is recorded in the .moc3 file.
Other physics operations and user data attached to the ArtMesh are output as separate files. The .model3.json file is what keeps track of all file references related to the model.

These can be output simultaneously when outputting a .moc3 file. “Exporting Data for Embedded Use”
You can also use Cubism Viewer (for OW) to add built-in settings for motion and facial expression files and pose files.

Import from .model3.json File Using Framework

When loading using the Framework, it is assumed that the necessary information for the model is extracted from the .model3.json file and an instance inheriting from the CubismUserModel class is maintained for use.

// Java
final String dir = "example/";
final String fileName = "example.model3.json";
final String jsonpath = dir + fileName;

byte[] buffer = createBuffer(jsonpath);
ICubismModelSetting setting = new CubismModelSettingJson(buffer);

Each element extracted in the ICubismModelSetting class can be loaded with the CubismUserModel.load~~ system function.

// Java
// Cubism Model
if (setting.getModelFileName() == ""){
    String path = setting.getModelFileName();
    path = dir + path;
  
    if (debugMode){
        LAppPal.printLog("[APP]create model" + setting.getModelFileName());
    }
  
    byte[] buffer = createBuffer(path);
    loadModel(buffer);
}

The overall loading flow from the .model3.json file can be easily grasped by following the flow from the LAppmodel.loadAssets function in the sample to the functions below.

• LAppModel.setupModel function
• CubismUserModel.createRenderer function
• LAppModel.setupTextures function

Create an Instance (Import .moc3 Files)

First, load the .moc3 file into memory.
Pass the read buffer to the CubismMoc.create function to first create a CubismMoc instance.
Next, call the CubismMoc.createModel function to create a CubismModel instance.
From this CubismModel instance, the user manipulates parameters and acquires information for drawing.

The CubismMoc.createModel function counts the number of models created by the function inside CubismMoc, and when discarding a CubismMoc, all CubismModels generated from all CubismMocs must be discarded.

// Java
String path = "example.moc3";
path = dir + path;

byte[] buffer = createBuffer(path);
CubismMoc moc = CubismMoc.create(buffer);
CubismModel model = moc.createModel();

Graphic Environment Associations

The Framework uses classes derived from the CubismRenderer class to make the model texture information independent of the graphics API.
All graphics-related information associated with the model is managed by classes derived from the CubismRenderer class.
As of SDK for Java R1 alpha 1, there is no need for the derived classes to manage the information since they only support Android. However, this specification is made in consideration of the possibility of adding a graphics API in the future and to unify specifications with existing SDKs.

First, specify the type of graphics environment you want to generate with the RendererType enumeration and call it as an argument to the CubismUserModel.createRenderer function.
(In SDK for Java R1 alpha 1, only Android is supported.)
Inside the createRenderer function, a derived class for the graphics API corresponding to the argument is created, and by registering a CubismModel instance with the CubismRenderer.initialize function, the CubismRenderer instance and the CubismModel instance are linked.

// Java
public void setupRenderer(CubismRenderer renderer, int maskBufferCount) {
    this.renderer = renderer;
 
    this.renderer.initialize(model, maskBufferCount);
}

Associate Textures

The textures that models have in the Framework are managed by a class derived from the CubismRenderer class.
The Android OpenGL ES 2.0 process uses the CubismRendererAndroid.bindTexture function in the CubismRendererAndroid class, which is derived from the CubismRenderer class, to register the textures.
The first argument is the texture number of the model, identified in the Editor by the number of the texture atlas.
The second argument is the texture’s OpenGL ES 2.0 management number.

// Java
// Load texture into Android OpenGL ES 2.0 texture unit.
String texturePath = modelSetting.getTextureFileName(modelTextureNumber);
texturepath = modelHomeDir + texturePath;
LAppTextureManager.textureInfo texture = LAppDelegate.getInstance().getTextureManager().createTextureFromPngFile(texturePath);
final int glTextureNumber = texture.id;

this.<CubismRendererAndroid>getRenderer().bindTexture(modelTextureNumber, glTextureNumber);

Specify Display Position and Size

With the previous settings, the model can be drawn, but in many cases, the display position and scale are too different to be shown on the screen as is.
See “DrawableVertexPositions Range” for the vertex range returned by the CubismModel.getDrawableVertexPositions function.
To adjust the size, use the CubismModelMatrix class and the CubismModel.getCanvasWidth and CubismModel.getCanvasHeight functions.

// Java
// modelMatrix is defined as the CubismModelMatrix type in the LAppModel class
modelMatrix = CubismModelMatrix.create(model.getCanvasWidth(), model.getCanvasHeight());

This matrix is multiplied by the Projection matrix before drawing and passed to the renderer as an MVP matrix.

// Java
public void onUpdate() {
    int width = LAppDelegate.getInstance().getWindowWidth();
    int height = LAppDelegate.getInstance().getWindowHeight();
    ...
    CubismMatrix4x4 projectionMatrix = CubismMatrix4x4.create();
    ...
    projectionMatrix.scale(1.0f, (float) height / (float) width);
    ...
    model.draw(projectionMatrix);
}

// Java
public void draw(CubismMatrix4x4 matrix) {
    ...
    matrix.multiplyByMatrix(modelMatrix);
    this.<CubismRendererAndroid>getRenderer().setMvpMatrix(matrix);
    ...
}

Update Vertices

Execute the CubismModel.update function to reflect parameter operations in the CubismModel instance to the vertices of the ArtMesh.

// Java)
model.update()

Draw

Drawing is not executed from the CubismModel instance, but by commanding the renderer associated with it.

// Java
this.<CubismRendererAndroid>getRenderer.drawModel();

Discard

Destroying an instance requires destroying all CubismModels and then destroying CubismMoc.
To preserve this order, CubismModel is deleted by the CubismMoc.deleteModel function.
If this destruction is not performed via this function, a warning will be issued because the number of models checked inside CubismMoc does not match.

// Java
protected void delete() {
    if (moc == null || model == null) {
        return;
    }
    moc.deleteModel(model);

    moc.delete();
    model.close();
    renderer.close();

    moc = null;
    model = null;
    renderer = null;
}

// Java
public void delete() {
    assert (modelCount == 0);
    if (moc ! = null) {
        moc.close();
    }
    moc = null;
}

public void deleteModel(CubismModel model) {
    model.close();
    modelCount--;
}

Use Parameter IDs that Do Not Exist in the Model in the CubismModel Class

In the operations of the CubismModel class parameters and part opacity, there is a functionality to handle IDs that do not exist in the .moc file.
This functionality is used by the CubismMotion class, the CubismPose class, and other classes, and if creating a new mechanism using this function, be careful to use an ID that does not conflict with existing functions.
Also, when reading the Framework, keep in mind that it may be linked to other functions using created parameters.
Note: Accessing a non-existent maximum, minimum, or initial value will result in an error.

When generating a parameter index from a parameter ID, parameters that do not exist are newly created

// Java
public int getParameterIndex(CubismId parameterId) {
    final CubismParameterView parameterView = model.findParameterView(parameterId.getString());
    if (parameterView ! = null) {
        return parameterView.getIndex();
    }

    // If the parameter does not exist in the model, it searches for it in the non-existent parameter ID list and returns its index.
    if (notExistParameterIds.containsKey(parameterId)) {
        final Integer index = notExistParameterIds.get(parameterId);
        assert index ! = null;
        return index;
    }

    // If the parameter does not exist in the non-existent parameter ID list, add newly the element.
    final int parameterCount = parameterValues.length;
    final int parameterIndex = parameterCount + notExistParameterIds.size();
    notExistParameterIds.put(parameterId, parameterIndex);
    notExistParameterIndices.add(parameterIndex);

    float[] tmp = new float[notExistParameterIndices.size()];
    System.arraycopy(notExistParameterValues, 0, tmp, 0, notExistParameterIndices.size() - 1);
    tmp[notExistParameterIndices.size() - 1] = 0.0f;
    notExistParameterValues = new float[notExistParameterIndices.size()];
    System.arraycopy(tmp, 0, notExistParameterValues, 0, notExistParameterIndices.size());

    return parameterIndex;
}