Manipulate the Display Position and Zoom Rate of the Model

Updated: 12/08/2022

Since Live2D uses polygons for drawing, it uses matrices as in 3DCG when drawing.
This article describes how to handle matrices in Cubism SDK sample programs.
The matrix itself is not explained.

Summary

To perform matrix calculations on the Cubism SDK, use CubismMatrix44 or a class that inherits it.

CubismMatrix44 internally holds an array for 4*4 matrices.
The order of array elements conforms to OpenGL.

CubismMatrix44 and its inherited classes do not support rotation, Z-axis movement, or zoom rate operations.

However, if the data is converted to GMatrix44 when passed to the renderer, it can be processed by a third-party matrix calculation library.

CubismMatrix44

Get array for matrix

To obtain the array for the matrix in CubismMatrix44, use one of the following functions:

• CubismMatrix44::GetArray function in Native (C++)
• CubismMatrix44.getAray function in Web (TypeScript)
• CubismMatrix44.getAray function in Java

CubismMatrix44::GetArray and CubismMatrix44.getAray functions are used to get an array of matrices to be passed to the renderer.

csmFloat32* matrixArray = cubismMatrix44->GetArray();

let matrixArray = mubismmatrix44.getAray();

float[] matrixArray = CubismMatrix.getArray();

Set array for matrix

To overwrite the array for the matrix in CubismMatrix44 with an externally generated matrix, use one of the following functions:

• CubismMatrix44::SetMatrix function in Native (C++)
• CubismMatrix44.setMatrix function in Web (TypeScript)
• CubismMatrix44.setMatrix function in Java

csmFloat32* matrixArray;

cubismMatrix44->SetMatrix(matrixArray);

let matrixArray: Float32Array;

cubismMatrix44.setMatrix(matrixArray);

CubismMatrix44 cubismMatrix44 = CubismMatrix.create();
float[] matrixArray = new float[16]

for (int i = 0; i < 16; i++) {
  matrixArray[i] = 0.5f;
}

cubismMatrix44.setMatrix(matrixArray);

Move display position

To translate the display position, use one of the following functions:

• CubismMatrix44::Translate function or CubismMatrix44::TranslateRelative function in Native (C++)
• CubismMatrix44.translate function or CubismMatrix44.translateRelative function in Web (TypeScript)
• CubismMatrix44.translate function or CubismMatrix44.translateRelative function in Java

The coordinates set in the arguments are all in the screen coordinate system.

CubismMatrix44::Translate and CubismMatrix44.translate functions overwrite the current coordinates with their arguments.
CubismMatrix44::TranslateRelative and CubismMatrix44.translateRelative functions translate relative to the current coordinates by the argument.

cubismMatrix44->Translate(translateX, translateY);

cubismMatrix44->TranslateRelative(translateX, translateY);

cubismMatrix44.translate(translateX, translateY);

cubismMatrix44.translateRelative(translateX, translateY);

cubismMatrix44.translate(translateX, translateY);

cubismMatrix44.translateRelative(translateX, translateY);

Zoom in/out on display magnification

To scale the display magnification, use one of the following functions:

• CubismMatrix44::Scale function or CubismMatrix44::ScaleRelative function in Native (C++)
• CubismMatrix44.scale function or CubismMatrix44.scaleRelative function in Web (TypeScript)
• CubismMatrix44.scale function or CubismMatrix44.scaleRelative function in Java

CubismMatrix44::Scale and CubismMatrix44.scale functions overwrite the current display magnification with an argument.
CubismMatrix44::ScaleRelative and CubismMatrix44.scaleRelative functions set the current display magnification relative to the value of the argument.

If 1 is set for the display magnification, it will be the original size of the model.

cubismMatrix44->Scale(scaleX, scaleY);

cubismMatrix44->ScaleRelative(scaleX, scaleY);

cubismMatrix44.scale(scaleX, scaleY);

cubismMatrix44.scaleRelative(scaleX, scaleY);

cubismMatrix44.scale(scaleX, scaleY);

cubismMatrix.scaleRelative(scaleX, scaleY);

Obtaining coordinates

To get the transformed coordinates by matrix, use one of the following functions:

• CubismMatrix44::TransformX function or CubismMatrix44::TransformY function in Native (C++)
• CubismMatrix44.transformX function or CubismMatrix44.transformY function in Web (TypeScript)
• CubismMatrix44.transformX function or CubismMatrix44.transformY function in Java

It is used, for example, to obtain the position on the screen from the vertex coordinates of a model in the local coordinate system.

csmFloat32 positionX = cubismMatrix44->TransformX(localX);
csmFloat32 positionY = cubismMatrix44->TransformY(localY);

let positionX: number = cubismMatrix44.transformX(localX);
let positionY: number = cubismMatrix44.transformY(localY);

float positionX = cubismMatrix44.transformX(localX);
float positionY = cubismMatrix44.transformY(localY);

To get the inverted transformed coordinates by matrix, use one of the following functions:

• CubismMatrix44::InvertTransformX function or CubismMatrix44::InvertTransformY function in Native (C++)
• CubismMatrix44.invertTransformX function or CubismMatrix44.invertTransformY function in Web (TypeScript)
• CubismMatrix44.invertTransformX function or CubismMatrix44.invertTransformY function in Java

Used to obtain the coordinates in the model’s local coordinate system from the coordinates entered in the screen coordinate system, such as in collision decision.

csmFloat32 localX = cubismMatrix44->InvertTransformX(inputPositionX);
csmFloat32 localY = cubismMatrix44->InvertTransformY(inputPositionY);

let localX: number = cubismMatrix44.invertTransformX(inputPositionX);
let localY: number = cubismMatrix44.invertTransformY(inputPositionY);

float localX = cubismMatrix44.invertTransformX(inputPositionX);
float localY = cubismMatrix44.invertTransformY(inputPositionY);

Matrix-to-Matrix calculations

To compute matrices against each other, use one of the following functions:

• CubismMatrix44::Multiply function or CubismMatrix44::MultiplyByMatrix function in Native (C++)
• CubismMatrix44.multiply function or CubismMatrix44.multiplyByMatrix function in Web (TypeScript)
• CubismMatrix44.multiply function or CubismMatrix44.multiplyByMatrix function in Java

CubismMatrix44* matrixA, matrixB;

// The following code performs the same processing in all the languages.

CubismMatrix44::Multiply(matrixA->GetArray(), matrixB->GetArray(), matrixA->GetArray());

matrixA->MultiplyByMatrix(matrixB);

let matrixA, matrixB: CubismMatrix44;

// The following code performs the same processing in all the languages.

CubismMatrix44.multiply(matrixA->GetArray(), matrixB->GetArray(), matrixA->GetArray());

matrixA.multiplyByMatrix(matrixB);

CubismMatrix44 matrixA = CubismMatrix44.create();
CubismMatrix44 matrixB = CubismMatrix44.create();

// The following code performs the same processing in all the languages.

CubismMatrix44.multiply(matrixA.getArray(), matrixB.getArray(), matrixA.getArray());

matrixA.multiplyByMatrix(matrixB);

CubismModelMatrix

This class inherits from CubismMatrix44.
It is intended for setting model coordinates, allowing the display position to be set from an edge or center point, and manipulating the display magnification by height and width.

The Cubism SDK sample displays the height of the Cubism model canvas across the entire screen.

Creating an instance

CubismModelMatrix requires that the model canvas size be set at instantiation.

CubismModel* cubismModel;
CubismModelMatrix* cubismModelMatrix;

cubismModelMatrix = new CubismModelMatrix(cubismModel->GetCanvasWidth(), cubismModel->GetCanvasHeight());

let cubismModel: CubismModel;
let cubismModelMatrix: CubismModelMatrix;

cubismModelMatrix = new CubismModelMatrix(cubismModel.getCanvasWidth(), cubismModel.getCanvasHeight());

// CubismModel generation process is omitted.
CubismModelMatrix cubismModelMatrix;

cubismModelMatrix = CubismModelMatrix.create(cubismModel.getCanvasWidth(), cubismModel.getCanvasHeight());

Apply Layout in model3.json

The layout information for the model can be described in model3.json.
To set the model position and scaling using this method, use one of the following functions:

• CubismModelMatrix::SetupFromLayout function in Native (C++)
• CubismModelMatrix.setupFromLayout function in Web (TypeScript)
• CubismModelMatrix.setupFromLayout function in Java

CubismModelSettingJson* model3Json;

csmMap<csmString, csmFloat32> layout;
model3Json->GetLayoutMap(layout);

// Set the model layout.
cubismModelMatrix->SetupFromLayout(layout);

let model3Json: CubismModelSettingJson;

let layout: csmMap<string, number>
model3Json.getLayoutMap(layout);

// Set the model layout.
cubismModelMatrix.setupFromLayout(layout);

CubismModelSettingJson model3Json;
Map<String, Float> layout = new HashMap<String, Float>();
model3Json.getLayoutMap(layout);
 
// Set the model layout.
cubismModelMatrix.setupFromLayout(layout);

Manipulate model display position

CubismModelMatrix allows you to set coordinates based on each edge, center, etc. of the canvas.

// Arrange the model so that the top edge of the model canvas is at the coordinates of the Y axis specified by the argument. This position cannot be the same as CubismModelMatrix::Bottom() and is overwritten by the position set later.
void Top(csmFloat32 y);

// Arrange the model so that the bottom edge of the model canvas is at the coordinates of the Y axis specified by the argument. This position cannot be the same as CubismModelMatrix::Top() and is overwritten by the position set later.
void Bottom(csmFloat32 y);


// Arrange the model so that the right side of the model canvas (on the right side when facing the screen) is at the coordinates of the X axis specified by the argument. This position cannot be the same as CubismModelMatrix::Left() and is overwritten by the position set later.
void Right(csmFloat32 x);

// Arrange the model so that the left side of the model canvas is at the coordinates of the X axis specified by the argument. This position cannot be the same as CubismModelMatrix::Right() and is overwritten by the position set later.
void Left(csmFloat32 x);


// Arrange the model so that the origin of the model is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
void SetPosition(csmFloat32 x, csmFloat32 y);

// Arrange the model so that the center of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates. The height and width of the canvas must be set before calling this.
void SetCenterPosition(csmFloat32 x, csmFloat32 y);


// Arrange the model so that the center of the Y axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
void CenterX(csmFloat32 x);

// Arrange the model so that the center of the X axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
void SetY(csmFloat32 y);

// Arrange the model so that the top edge of the model canvas is at the coordinates of the Y axis specified by the argument. This position cannot be the same as CubismModelMatrix::Bottom() and is overwritten by the position set later.
public top(csmFloat32 y): void

// Arrange the model so that the bottom edge of the model canvas is at the coordinates of the Y axis specified by the argument. This position cannot be the same as CubismModelMatrix::Top() and is overwritten by the position set later.
public bottom(csmFloat32 y): void


// Arrange the model so that the right side of the model canvas (on the right side when facing the screen) is at the coordinates of the X axis specified by the argument. This position cannot be the same as CubismModelMatrix::Left() and is overwritten by the position set later.
public right(csmFloat32 x): void

// Arrange the model so that the left side of the model canvas is at the coordinates of the X axis specified by the argument. This position cannot be the same as CubismModelMatrix::Right() and is overwritten by the position set later.
public left(csmFloat32 x): void


// Arrange the model so that the origin of the model is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public setPosition(csmFloat32 x, csmFloat32 y): void

// Arrange the model so that the center of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates. The height and width of the canvas must be set before calling this.
public setCenterPosition(csmFloat32 x, csmFloat32 y): void


// Arrange the model so that the center of the Y axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public centerX(csmFloat32 x): void

// Arrange the model so that the center of the X axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public setY(csmFloat32 y): void

// Arrange the model so that the top edge of the model canvas is at the coordinates of the Y axis specified by the argument. This position cannot be the same as CubismModelMatrix.bottom() and is overwritten by the position set later.
public void top(float y) {...}

// Arrange the model so that the bottom edge of the model canvas is at the coordinates of the Y axis specified by the argument. This position cannot be the same as CubismModelMatrix.top() and is overwritten by the position set later.
public void bottom(float y) {...}

// Arrange the model so that the right side of the model canvas (on the right side when facing the screen) is at the coordinates of the X axis specified by the argument. This position cannot be the same as CubismModelMatrix.left() and is overwritten by the position set later.
public void right(float y) {...}

// Arrange the model so that the left side of the model canvas (on the left side when facing the screen) is at the coordinates of the X axis specified by the argument. This position cannot be the same as CubismModelMatrix.right() and is overwritten by the position set later.
public void left(float y) {...}

// Arrange the model so that the origin of the model is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public void setPosition(float x, float y) {...}

// Arrange the model so that the center of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates. The height and width of the canvas must be set before calling this.
public void setCenterPosition(float x, float y) {...}

// Arrange the model so that the center of the X axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public void centerX(float x) {...}

// Arrange the model so that the X axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public void setX(float x) {...}

// Arrange the model so that the center of the Y axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public void centerY(float y) {...}

// Arrange the model so that the Y axis of the model canvas is the coordinates of the argument. The coordinates passed in the argument are treated as world coordinates.
public void setY(float y) {...}

Manipulate model display size

CubismModelMatrix has a scaling factor that can be manipulated while maintaining the model’s aspect ratio.

// Scale the width of the canvas to the value specified by the argument while maintaining the aspect ratio of the model.
cubismModelMatrix->SetWidth(w);

// Scale the height of the canvas to the value specified by the argument while maintaining the aspect ratio of the model.
cubismModelMatrix->SetHeight(h);

// Scale the width of the canvas to the value specified by the argument while maintaining the aspect ratio of the model.
cubismModelMatrix.setWidth(w);

// Scale the height of the canvas to the value specified by the argument while maintaining the aspect ratio of the model.
cubismModelMatrix.setHeight(h);

// Scale the width of the canvas to the value specified by the argument while maintaining the aspect ratio of the model.
cubismModelMatrix.setWidth(w);

// Scale the height of the canvas to the value specified by the argument while maintaining the aspect ratio of the model.
cubismModelMatrix.setHeight(h)

CubismViewMatrix

This class inherits from CubismMatrix44.
It is intended for changing camera position and zooming, etc.

The Cubism SDK sample creates a matrix from the size of the drawing area.

The transformation matrices handled within CubismViewMatrix are assumed to be parallel projections.

Move cameras

To move the camera (screen), use one of the following functions:

• CubismViewMatrix::AdjustTranslate function in Native (C++)
• CubismViewMatrix.adjustTranslate function in Web (TypeScript)
• CubismViewMatrix.adjustTranslate function in Java

The amount of movement passed as an argument is the screen coordinates.

cubismViewMatrix->AdjustTranslate(x, y);

cubismViewMatrix.adjustTranslate(x, y);

cubismViewMatrix.adjustTranslate(x, y);

Zoom in and out of camera

To scale using the camera’s magnification, use one of the following functions:

• CubismViewMatrix::AdjustScale function in Native (C++)
• CubismViewMatrix.adjustScale function in Web (TypeScript)
• CubismViewMatrix.adjustScale function in Java

Scale around the coordinates passed as the first and second arguments.

cubismViewMatrix->AdjustScale(centerX, centerY, scale);

cubismViewMatrix.adjustScale(centerX, centerY, scale);

cubismViewMatrix.adjustScale(centerX, centerY, scale);