Mask preprocessing method (Web)

[Last updated: 09/03/2019]

To maintain drawing speed on smartphones and other devices, the Live2D Cubism SDK for Web allows for the use of
the “pre-processing method” in which all mask shapes are drawn on a single mask buffer at the beginning of the model drawing process.

In the principle drawing method, the mask shape is drawn each time when a Drawable that requires a mask is drawn (see figure).
This method allows for the representation of high-definition masks, but it is relatively costly in terms of switching render targets, clearing buffers, etc. each time Drawable needs a mask.
This may cause slow rendering speeds on smartphones and other devices.

However, simply preparing masks in advance requires multiple mask buffers, which can overwhelm memory.
To solve this problem, the following processing can be performed on a single mask buffer to minimize memory pressure while treating it as if multiple mask buffers were used.

 

Mask Integration

Since all masks are generated in advance, Drawables with the same mask specification can use the same mask image to reduce the number of masks to be generated.

This is done in the CubismRenderer_WebGL.initialize function call
by the CubismClippingManager_WebGL.initialize function.

 

 

Separation by color information

The mask buffer is an RGBA video array, just like a normal texture buffer, etc.
The normal mask process uses only this A channel to apply the mask, but not the RGB channels.
Therefore, by having separate mask data for RGBA, one mask buffer can be treated as four mask buffers.

 

 

Partitioning

When 4 mask images are not enough, the number of masks can be increased by handling the mask buffer in 2, 4, or 9 divisions.
There is also a division by color information so that up to 36 different masks of 4x9 can be held.

Also, to prevent the mask image from being crushed, the mask is drawn on all Drawable rectangles to which the mask is applied.
This requires range generation, mask generation, and matrix generation with the use of masks.

 

 

 

Rectangle Confirmation

In the first step of mask generation, for each mask, check the rectangle that fits all the mask application destinations.

 

 

Layout determination subject to color separation and divisional separation

Defines the color channel and division position of the mask buffer that belongs to each mask.

 

 

Mask drawing, matrix generation using masks

Prepare transformation matrices for mask generation and mask use based on the rectangle area and the location of the rectangle examined before drawing.

 

 

Dynamic resizing of mask buffer

CubismRenderer_WebGL provides an API to resize the mask buffer at runtime.
Currently, the size of the mask buffer is set to 256 * 256 (pixels) as the initial value, but if you want to cut the mask generation area into 9 sheets,
a mask shape drawn in a rectangular area of 85*85 (pixels) is further enlarged and used as a clipping area.
As a result, the edges of the clipping result are blurred or blotchy.
As a solution to this problem, an API is provided to change the size of the mask buffer at program execution time.

For example, if the mask buffer size is 256*256 => 1024*1024 and the mask generation area is cut into 9 pieces, the mask shape can be drawn in a rectangular area of 341*341, so
when enlarged and used as a clipping area, it eliminates edge blurring or blurring of the clipping result.

* Increase the size of the mask buffer => The more pixels to be processed, the slower the speed, but the cleaner the drawing result.
* Reduce the size of the mask buffer => Faster speed because fewer pixels are processed, but the drawing result will be dirtier.

 

 

© 2010 - 2022 Live2D Inc.