How to use CubismNativeFramework directly

[Last updated : 01/30/2020]

The sample project included in the Cubism SDK for Native uses the CubismNativeFramework (under /Framework/ in the SDK package) to handle Cubism models.
When creating a project that handles Cubism models, it is best to use the sample project as a basis, but there are cases where you may want to use the CubismNativeFramework API directly for implementation in an existing application or in your own engine.
However, the included sample project is highly functional, so it is not easy to understand and isolate the structure just by referring to it.

The following is a minimal snippet that directly calls CubismFramework to handle models for users like those listed above.


CubismFramework Cycle

The procedure for working with CubismFramework is as follows:

  1. Initialization of CubismFramework
  2. Get the path to the model file
  3. Loading the model
  4. Update process
  5. Model Destruction
  6. Exit process of CubismFramework


Initialization of CubismFramework

The initialization process of CubismFramework is as follows:

CubismFramework allocates memory using the allocator passed as the first argument to CubismFramework::StartUp().
Cubism SDK for Native includes allocator samples in CubismNativeSamples.
The allocator used in CubismFramework also requires an implementation that specifies and allocates an alignment.
However, if the user wishes to customize the memory allocation to his/her own, he/she can replace it with his/her own implementation or modify the sample to incorporate it.

For specific implementation of allocators, please click here, and for notes on creating custom memory allocators, please click here.


CubismFramework::Initialize() is called only once at initialization, after which successive calls will be skipped unless CubismFramework is destroyed.
However, once the CubismFramework has been destroyed, CubismFramework::Initialize() is called to initialize it again.



Get the path to the model file

The set of data for Cubism embedding is described in model3.json with their relative paths.
You can directly specify .moc3 files, textures, etc. to be loaded, but basically we recommend that you get the path from model3.json and load them.

Parsing of model3.json uses CubismFramework's CubismModelSettingJson class.



Loading the model

The model is loaded through the CubismModel interface.

Instances of CubismModel are held by CubismUserModel::_model in CubismNativeFramework.
If you use this, we recommend that you handle it from a class inheriting from CubismUserModel.

Resource management for textures, motion, facial expression motion, etc. can also be done externally.

As an example, we will use CubismUserModelExtend, which inherits from CubismUserModel.


The path to the moc3 file to be loaded can be obtained in model3.json as described above. Similarly, the path to facial expressions, physics, poses, blinks, lip-sync, user data, and motion can be obtained from model3.json.
The SDK sample loads these at the same time as the model is loaded.
Please refer to the following for each loading method.



The snippet above is based on the assumption that a single model will be displayed on the screen.
To display multiple models on the screen at the same time, create instances of the same number of CubismUserModel derived classes as the models to be displayed.


Update process

The update process of a Cubism model is performed by calling CubismModel::Update(), which is the interface of CubismModel, as well as model initialization.
Calling CubismModel::Update() updates the Cubism Core and updates vertex information from the values of parameters and parts that have been set so far.

The update process always performs eye tracking, physics, motion playback, etc. before Update().

Manipulation of parameter values after CubismModel::Update() is not reflected.
After that, call CubismModel::Update() again to reflect the change, but this process is highly burdensome, so we recommend that you combine them into one.


Cubism MotionManager::UpdateMotion(), which plays the motion, overwrites any ID parameter values used in the motion to be played.
Therefore, any manipulation of parameter values prior to this process will be overwritten by Cubism MotionManager::UpdateMotion().
It is recommended that value operations such as line-of-sight tracking and physics operations be performed first, followed by motion playback processing.


Also, if not all parameters are used in the motion to be played back, or if the parameter values are not manipulated because playback of the motion is stopped, the result of the value manipulation done in the previous frame will remain.
Therefore, the results of subsequent relative parameter value manipulations may be unintended.
This can be done by calling CubismModel::LoadParameter() and CubismModel::SaveParameter() before and after the process of applying motion to the model, thereby resetting any subsequent relative value operations.


Please refer to the respective documents for information on eye tracking, physics operations, motion playback, and other processes themselves.

By passing this updated vertex information, etc. to the renderer, the Cubism model can be drawn on the screen.


Model Destruction

To destroy the model, destroy the instance of the generated CubismUserModel derived class.
This destroys the motion, facial expressions, physics, and other information held by this model from the destructor of the CubismUserModel.



Termination process of Cubism Framework

To release common part resources allocated by CubismFramework, call CubismFramework::Dispose().
Do not call CubismFramework::Dispose() before calling CubismFramework::Initialize().

Also, when calling CubismFramework::Dispose(), all models must be destroyed beforehand.
This is because the process of destroying the model uses the allocator Deallocate().


Data initialized in CubismFramework is placed in a static area and does not depend on model data.
Therefore, it can be used across multiple models, and there is no need to call CubismFramework::Dispose() if you just want to switch models.

© 2010 - 2022 Live2D Inc.