Motion-sync during motion playback (Unity)

Updated: 11/28/2024

As of the Cubism 5 SDK MotionSync Plugin for Unity R2, it is now possible to perform motion-sync during motion playback using the motion-sync settings associated with the motion in the .model3.json file as well as the audio associated with the motion.

Prerequisites

To apply motion-sync during motion playback, it is assumed that the motion information exported to .model3.json has the appropriate audio and motion-sync settings.

See “Motion Settings” in the Editor manual for specific setting methods.

Mechanism

The following items have been added in Cubism 5 SDK MotionSync Plugin for Unity R2 and later.

• CubismMotionSyncLinkData
  • An asset that associates motion with audio and motion-sync settings on import
• CubismMotionSyncLinkList
  • An asset with references to CubismMotionSyncLinkData
• CubismMotionSyncLinkController
  • A component that plays the audio associated with the played motion and applies the motion-sync settings

The CubismMotionSyncLinkData asset and CubismMotionSyncLinkList asset, which is a list with references to the former asset, are generated during model import and re-import.

You can use CubismMotionController.AnimationBeginHandler to detect motion playback by applying the newly added CubismMotionSyncLinkController component to the model and using the CubismMotionController component to play back motion. You can also use motion-sync by playing the associated audio.

You can use CubismMotionController.AnimationBeginHandler to detect motion playback by applying the newly added CubismMotionSyncLinkController component to the model. You can also use motion-sync by playing the associated audio.

The CubismMotionController component is used to play back motion.
For more information on motion playback with the CubismMotionController component, see “Motion” in the SDK manual.

To use a model that has already been imported

If your models have already been set and imported into a project using the Cubism 5 SDK MotionSync Plugin for Unity R1 or earlier, update it to the Cubism 5 SDK MotionSync Plugin for Unity R2 or later and re-import the models. CubismMotionSyncLinkData and CubismMotionSyncLinkList assets will be generated.

Creating a scene to work on

1. Prepare a model for which settings have been completed according to “Motion Settings” in the Editor manual.

2. Place the model in the scene and configure the settings in “Usage in a Scene (Unity)” in the SDK manual except for adding the audio file.

3. Attach the CubismMotionSyncLinkController component to the root of the model. At this time, the CubismMotionController component is also automatically attached.

4. Link the AudioSource component to be used to the AudioSource of the CubismMotionSyncLinkController component and link the CubismMotionSyncLinkList asset of the model to be used to the MotionSyncLinkList.

5. This time, create the following script.

using UnityEngine;
using Live2D.Cubism.Framework.Motion;

public class MotionLinkTest : MonoBehaviour
{
    private CubismMotionController _motionController;
    public AnimationClip Clip;

    // Start is called before the first frame update
    private void Start()
    {
        _motionController = GetComponent<CubismMotionController>();
    }

    // Update is called once per frame
    private void Update()
    {
        if (_motionController.IsPlayingAnimation())
        {
            return;
        }

        if (Input.GetKeyDown(KeyCode.F))
        {
            _motionController.PlayAnimation(Clip, isLoop: false);
        }
    }
}

6. Attach the script you created to the model and set Clip to a motion that has already been associated with audio and motion-sync.

7. Play the scene and press the F key on the keyboard to play the motion, which will play the audio and perform motion-sync.