LookAt (Cocos Creator)

Updated: 03/14/2023

Summary

LookAt is a function that manipulates the value of the desired parameter to follow a specific coordinate.
By customizing the coordinates to be tracked by the user, Cubism models can be made to track specific GameObjects and other objects.
Click here for more information on how to use LookAt.

LookAt in the Cubism SDK for Cocos Creator consists of three types of elements.

1. Components for specifying parameters to be tracked
2. Components that apply values to each parameter
3. Manipulation of the values applied in number 2

1. Components for Specifying Parameters to Be Tracked

To specify parameters to be followed by LookAt, use CubismLookParameter.

CubismLookParameter is used by attaching it to GameObjects placed under [Prefab root]/Parameters/.
The parameter with the same ID as the GameObject to which it is attached is treated as a parameter for eye tracking.

CubismLookParameter has two setting items, Axis and Factor.

/** Look axis. */
@property({ type: Enum(CubismLookAxis), serializable: true, visible: true })
public axis: CubismLookAxis = CubismLookAxis.X;

/** Factor. */
@property({ type: CCFloat, serializable: true, visible: true })
public factor: number = 0;

• Axis

Sets which axis values of the input coordinates will be used.
There are three items that can be set: X, Y, and Z.

/** Look axis. */
enum CubismLookAxis {
  /** X axis. */
  X,
  /** Y axis. */
  Y,
  /** Z axis. */
  Z,
}
export default CubismLookAxis;

• Factor

Sets the correction value to the value to be applied.
The coordinates entered will be processed to a value in the range of -1.0f to 1.0f when applied.
The value set for Factor is a multiplier that multiplies the input values to match the minimum and maximum values of each parameter.

public tickAndEvaluate(targetOffset: math.Vec3): number {
  const result =
    this.axis == CubismLookAxis.X
      ? targetOffset.x
      : this.axis == CubismLookAxis.Z
      ? targetOffset.z
      : targetOffset.y;
  return result * this.factor;
}

CubismLookParameter is also used as a marker for CubismLookController to obtain references.

2. Components that Apply Values to Each Parameter

Use CubismLookController to apply eye tracking to each parameter.
When using CubismLookController, attach it to the root of Cubism’s Prefab.

Gets a reference to all CubismLookParameters attached to Prefab during initialization of CubismLookController.
When parameters for eye tracking are added/removed during execution, CubismLookController.Refresh() is called to get the references again.

/** Refreshes the controller. Call this method after adding and/or removing {@link CubismLookParameter}s. */
public refresh(): void {
  const model = CoreComponentExtensionMethods.findCubismModel(this);
  if (model == null) {
    return;
  }
  if (model.parameters == null) {
    return;
  }

  // Catch sources and destinations.

  this.sources = ComponentExtensionMethods.getComponentsMany(
    model.parameters,
    CubismLookParameter
  );
  this.destinations = new Array<CubismParameter>(this.sources.length);

  for (let i = 0; i < this.sources.length; i++) {
    this.destinations[i] = this.sources[i].getComponent(CubismParameter);
  }

  // Get cubism update controller.
  this.hasUpdateController = this.getComponent(CubismUpdateController) ! = null;
}

...

/** Called by Cocos Creator. Makes sure cache is initialized. */
protected start(): void {
  // Default center if necessary.
  if (this.center == null) {
    this.center = this.node;
  }

  // Initialize cache.
  this.refresh();
}

CubismLookController applies the coordinates returned by the object set to CubismLookController.Target to the parameters marked by CubismLookParameter at the timing of LateUpdate() in every frame.

// Update position.
let position = this.lastPosition;

const inverseTransformPoint = this.node.inverseTransformPoint(
  new math.Vec3(),
  target.getPosition()
);
this.goalPosition = math.Vec3.subtract(
  new math.Vec3(),
  inverseTransformPoint,
  this.center.position
);
if (position ! = this.goalPosition) {
  const temp = MathExtensions.Vec3.smoothDamp(
    position,
    this.goalPosition,
    this.velocityBuffer,
    this.damping
  );
  position = temp[0];
  this.velocityBuffer = temp[1];
}

// Update sources and destinations.
for (let i = 0; i < this.destinations.length; i++) {
  CubismParameterExtensionMethods.blendToValue(
    this.destinations[i],
    this.blendMode,
    this.sources[i].tickAndEvaluate(position)
  );
}

// Store position.
this.lastPosition = position;

3. Manipulation of the Values Applied in Number 2

As described in “2. Components that Apply Values to Each Parameter,” the coordinates that CubismLookController applies to the parameters for eye tracking return the object set in CubismLookController.Target.

This is an implementation of the ICubismLookTarget interface, which can be implemented by the user to track the model to desired coordinates.

/** Target to look at. */
interface ICubismLookTarget {
  readonly [ICubismLookTarget.SYMBOL]: typeof ICubismLookTarget.SYMBOL;

  /**
   * Gets the position of the target.
   *
   * @returns The position of the target in world space.
   */
  getPosition(): math.Vec3;

  /**
   * Gets whether the target is active.
   *
   * @returns true if the target is active; false otherwise.
   */
  isActive(): boolean;
}
export default ICubismLookTarget;

• GetPosition()

Returns the coordinates to be followed.
The coordinates returned here are treated as world coordinates.

• IsActive()

Returns whether the tracking is valid or not.
Follows only when true is returned.

CubismLookTargetBehaviour is included in the SDK as an example implementation of ICubismLookTarget.
CubismLookTargetBehaviour will be a sample that returns the coordinates of the GameObject to which it is attached.

/** Straight-forward {@link ICubismLookTarget} {@link Component}. */
@ccclass('CubismLookTargetBehaviour')
export default class CubismLookTargetBehaviour extends Component implements ICubismLookTarget {
  readonly [ICubismLookTarget.SYMBOL]: typeof ICubismLookTarget.SYMBOL = ICubismLookTarget.SYMBOL;

  //#region Implementation of ICubismLookTarget

  /**
   * Gets the position of the target.
   * @returns The position of the target in world space.
   */
  public getPosition(): Readonly<math.Vec3> {
    return this.node.worldPosition;
  }

  /**
   * Gets whether the target is active.
   * @returns true if the target is active; false otherwise.
   */
  public isActive(): boolean {
    return this.enabledInHierarchy;
  }

  //#endregion
}