Raycasting (Cocos Creator)

Updated: 03/14/2023

Summary

Raycasting is a function that determines whether a user-specified Cubism mesh intersects with the set coordinates.
It is possible to obtain the clicked/tapped coordinates or a collision detection between any object in the scene and a specified mesh.
Click here for more information on how to use Raycasting.

Raycasting in the Cubism SDK for Cocos Creator consists of two major elements.

1. Components for specifying the ArtMesh to be judged
2. Components that perform collision detection between input coordinates and the specified ArtMeshes

1. Components for Specifying the ArtMesh to Be Judged

Use CubismRaycastable to specify the mesh to be used for collision detection.

CubismRaycastable is used by attaching it to Node placed under [Prefab root]/Drawables/.
The ArtMesh with the same ID as the Node to which it is attached is used for collision detection.

CubismRaycastable has a parameter that sets the accuracy of its mesh collision detection.

@ccclass('CubismRaycastable')
export default class CubismRaycastable extends Component {
  /** The precision. */
  @property({
    type: Enum(CubismRaycastablePrecision),
    serializable: true,
    visible: true,
    readonly: false,
  })
  public precision: CubismRaycastablePrecision = CubismRaycastablePrecision.boundingBox;
}

CubismRaycastable.Precision has two values that can be set, each as follows.

• BoundingBox
  A rectangle with four horizontal or vertical sides that encloses the mesh is used for the collision detection.
  Depending on the mesh shape, it may also take collisions at coordinates outside the mesh, but it offers better performance than Triangles.
• Triangles
  The shape of the mesh is used as the range of the collision detection.
  It is less performant than the BoundingBox, but it can make judgments within an accurate range.

enum CubismRaycastablePrecision {
  /** Cast against bounding box. */
  boundingBox,

  /** Cast against triangles. */
  triangles,
}

2. Components that Perform Collision Detection between Input Coordinates and the Specified ArtMeshes

CubismRaycaster is used to obtain the actual collision detection.
When using CubismRaycaster, it is attached to the root of Cubism’s Prefab.

Gets a reference to all CubismRaycastables attached to the Prefab during initialization of the CubismRaycaster.
If you add/remove a mesh for collision detection during execution, call CubismRaycaster.refresh() to get the reference again.

  /** Refreshes the controller. Call this method after adding and/or removing {@link CubismRaycastable}. */
  private refresh(): void {
    const candidates = ComponentExtensionMethods.findCubismModel(this)?.drawables ?? null;
    if (candidates == null) {
      console.error('CubismRaycaster.refresh(): candidates is null.');
      return;
    }

    // Find raycastable drawables.
    const raycastables = new Array<CubismRenderer>();
    const raycastablePrecisions = new Array<CubismRaycastablePrecision>();

    for (var i = 0; i < candidates.length; i++) {
      // Skip non-raycastables.
      if (candidates[i].getComponent(CubismRaycastable) == null) {
        continue;
      }
      const renderer = candidates[i].getComponent(CubismRenderer);
      console.assert(renderer);
      raycastables.push(renderer!) ;

      const raycastable = candidates[i].getComponent(CubismRaycastable);
      console.assert(raycastable);
      console.assert(raycastable!.precision);
      raycastablePrecisions.push(raycastable!.precision!) ;
    }

    // Cache raycastables.
    this.raycastables = raycastables;
    this.raycastablePrecisions = raycastablePrecisions;
  }

  /** Called by Cocos Creator. Makes sure cache is initialized. */
  protected start(): void {
    // Initialize cache.
    this.refresh();
  }

To get a collision detection from the coordinates, use CubismRaycaster.raycast1() or CubismRaycaster.raycast2().

  public raycast1(
    origin: Vector3,
    direction: Vector3,
    result: CubismRaycastHit[],
    maximumDistance: number = Number.POSITIVE_INFINITY
  ): number {
    return this.raycast2(
      geometry.Ray.create(origin.x, origin.y, origin.z, direction.x, direction.y, direction.z),
      result,
      maximumDistance
    );
  }

  /**
   * Casts a ray.
   * @param ray
   * @param result  The result of the cast.
   * @param maximumDistance [Optional] The maximum distance of the ray.
   * @returns
   * true in case of a hit; false otherwise.
   *
   * The numbers of drawables had hit
   */
  public raycast2(
    ray: geometry.Ray,
    result: CubismRaycastHit[],
    maximumDistance: number = Number.POSITIVE_INFINITY
  ): number {
    // Cast ray against model plane.
    const origin = Vector3.from(ray.o);
    const direction = Vector3.from(ray.d);
    const intersectionInWorldSpace = origin.add(
      direction.multiplySingle(direction.z / origin.z)
    );
    let intersectionInLocalSpace = Vector3.from(
      this.node.inverseTransformPoint(new math.Vec3(), intersectionInWorldSpace.toBuiltinType())
    );
    intersectionInLocalSpace = intersectionInLocalSpace.copyWith({ z: 0 });
    const distance = intersectionInWorldSpace.magnitude();
    // Return non-hits.
    if (distance > maximumDistance) {
      return 0;
    }
    // Cast against each raycastable.
    let hitCount = 0;
    console.assert(this.raycastables);
    const raycastables = this.raycastables!;
    console.assert(this.raycastablePrecisions);
    const raycastablePrecisions = this.raycastablePrecisions!;
    for (let i = 0; i < raycastables.length; i++) {
      const raycastable = raycastables[i];
      const raycastablePrecision = raycastablePrecisions[i];
      // Skip inactive raycastables.
      console.assert(raycastable.meshRenderer);
      if (!raycastable.meshRenderer!.enabled) {
        continue;
      }
      const bounds = raycastable.mesh.calculateBounds();

      // Skip non hits (bounding box)
      if (!bounds.contains(intersectionInLocalSpace)) {
        continue;
      }

      // Do detailed hit-detection against mesh if requested.
      if (raycastablePrecision == CubismRaycastablePrecision.triangles) {
        if (!this.containsInTriangles(raycastable.mesh, intersectionInLocalSpace)) {
          continue;
        }
      }

      result[hitCount] = new CubismRaycastHit({
        drawable: raycastable.getComponent(CubismDrawable),
        distance: distance,
        localPosition: intersectionInLocalSpace,
        worldPosition: intersectionInWorldSpace,
      });

      ++hitCount;

      // Exit if result buffer is full.
      if (hitCount == result.length) {
        break;
      }
    }

    return hitCount;
  }

CubismRaycaster.raycast() returns the number of meshes that get collision detection in the return value.
In addition, information on the mesh from which the collision detection was obtained is set to the instance of the CubismRaycastHit[] type passed as an argument.

      result[hitCount] = new CubismRaycastHit({
        drawable: raycastable.getComponent(CubismDrawable),
        distance: distance,
        localPosition: intersectionInLocalSpace,
        worldPosition: intersectionInWorldSpace,
      });

      ++hitCount;

CubismRaycaster.raycast() retrieves up to the number of elements of an instance of CubismRaycastHit[] type if multiple meshes overlap at the same coordinates.
If there are more overlapping meshes than the number of elements, the results will not be retrieved for the overlapping meshes.

CubismRaycastHit is a class with information about the mesh from which the collision detection was obtained.

  /** The hit {@link CubismDrawable} */
  public readonly drawable: CubismDrawable | null;

  /** The distance the ray traveled until it hit the {@link CubismDrawable}. */
  public readonly distance: number;

  /** The hit position local to the {@link CubismDrawable}. */
  public readonly localPosition: Vector3;

  /** The hit position in world coordinates. */
  public readonly worldPosition: Vector3;

• drawable
  This is a reference to the ArtMesh from which the collision detection was obtained.
• distance
  The distance from the specified coordinates.
  The linear distance between origin or ray.origin passed as an argument to CubismRaycaster.Raycast() and Transform.position of the Drawable.
• localPosition
  These are the local coordinates of the ArtMesh from which the collision detection was obtained.
• worldPosition
  These are the world coordinates of the ArtMesh from which the collision detection was obtained.