Interoperability - A Way Forward

[NateTheGreatt]

Interoperability - A Way Forward

Integrating existing libraries with bitECS has been a pain point due to all state being stored in TypedArrays. I am starting this discussion to flesh out a new way to interop with any object in JavaScript and still maintain the option of reaping the performance benefits of cache-friendly SoA (Structure of Array) iteration.

Proxies

In the current bitECS documentation I demonstrate a way to construct proxy-like objects to interface with the SoA data of bitECS components. This idea was recently expanded upon in a new Three.js integration POC that I put together for a discussion going on over at the ThirdRoom project.

Sandboxed here (thx @SupremeTechnopriest 🍻)

The gist of the POC:

  const world = createWorld()

  const mesh = new THREE.Mesh( geometry, material )

  const e = addEntity(world)

  // store the eid and component SoA store on the relevant object property
  Object.defineProperty(mesh.position, 'eid', { get: () => e })
  Object.defineProperty(mesh.position, 'store', { get: () => TransformComponent.position })

  // redefine x/y/z in-place with getters/setters which proxy data straight to the SoA store
  Object.defineProperty(mesh.position, 'x', {
    get () { return this.store.x[this.eid] },
    set (n) { this.store.x[this.eid] = n }
  })
  Object.defineProperty(mesh.position, 'y', {
    get () { return this.store.y[this.eid] },
    set (n) { this.store.y[this.eid] = n }
  })
  Object.defineProperty(mesh.position, 'z', {
    get () { return this.store.z[this.eid] },
    set (n) { this.store.z[this.eid] = n }
  })

function update() {

  requestAnimationFrame(update)

  TransformComponent.position.x[e] += 0.01
  TransformComponent.position.y[e] += 0.01
  TransformComponent.position.z[e] += 0.01

  // mesh will move via modification of the SoA component, no need to touch the mesh object itself
  // Three.js will pull data from SoA stores via getters
  renderer.render( scene, camera )

}

Using this method, we no longer have to manually sync up Three's Object3D.position, because the x, y, and z properties are redefined in-place as getters/setters which read/write data straight to the SoA component data stores. This can be done with any property, but some properties come with caveats which can be navigated around. E.g. rotation needs a rotation._onChangeCallback to be called for the changes to be picked up and rendered.

Now, all Three.js API methods will read and write directly to and from the SoA component data stores, which means you can fluidly switch from object-syntax to SoA syntax at any time! Write your gameplay logic using the high-level Three.js API methods, and then selectively optimize systems by writing lower-level transformations in SoA syntax.

The best part is that this is not limited to Three.js! Bindings can be created to any object in JavaScript from any library!

Preliminary benchmarks show that the performance benefits of being able to (re)write any data transformation as a cache-aware iteration over SoA stores far outweigh potential performance impacts induced by the proxy gets/sets on the objects. This also eliminates full loops that iterate over entities to sync their properties up with objects.

Bonus: technically this gives bitECS serialization the ability to serialize data from any object in JS!

I would like to leave this discussion open for alternative ideas for interoperability, as well as fleshing out this proposal and any potential downsides.

[robertlong]

This looks great! Thanks for starting this!

Here's my edits on what you've got so far:

import {
  createWorld,
  pipe,
  addEntity,
  defineComponent,
  addComponent,
  IWorld,
  Component,
  Types,
  defineQuery,
} from "bitecs";
import {
  Scene,
  PerspectiveCamera,
  Camera,
  WebGLRenderer,
  Object3D,
} from "three";

interface Time {
  last: number;
  delta: number;
  elapsed: number;
}

type MapComponent<T> = Component & {
  get: (eid: number) => T | undefined;
  set: (eid: number, value: T | undefined) => void;
};

function defineMapComponent<T>(): MapComponent<T> {
  const component = defineComponent({}) as MapComponent<T>;
  const store: Map<number, T | undefined> = new Map();
  component.get = (eid: number) => store.get(eid);
  component.set = (eid: number, value: T | undefined) => {
    store.set(eid, value);
  };
  return component;
}

const RendererComponent = defineMapComponent<WebGLRenderer>();
const TimeComponent = defineMapComponent<Time>();
const Object3DComponent = defineMapComponent<Object3D>();

const { f32 } = Types;

export const Vector3Schema = { x: f32, y: f32, z: f32 };
export const QuaternionSchema = { x: f32, y: f32, z: f32, w: f32 };

export const TransformComponent = defineComponent({
  position: Vector3Schema,
  rotation: QuaternionSchema,
  scale: Vector3Schema,
});

type WorldOptions = {
  scene?: Scene;
  camera?: Camera;
  renderer?: WebGLRenderer;
  time?: Time;
};

type World = IWorld & { eid: number; scene: number; camera: number };

type Object3DEntity = Object3D & { eid: number };

function addObject3DEntity(world: World, obj: Object3D): number {
  const eid = addEntity(world);

  Object.defineProperty(obj, "eid", { get: () => eid });

  Object.defineProperties(obj.position, {
    eid: { get: () => eid },
    store: { get: () => TransformComponent.position },
    x: {
      get() {
        return this.store.x[this.eid];
      },
      set(n) {
        this.store.x[this.eid] = n;
      },
    },
    y: {
      get() {
        return this.store.y[this.eid];
      },
      set(n) {
        this.store.y[this.eid] = n;
      },
    },
    z: {
      get() {
        return this.store.z[this.eid];
      },
      set(n) {
        this.store.z[this.eid] = n;
      },
    },
  });

  Object.defineProperties(obj.rotation, {
    eid: { get: () => eid },
    store: { get: () => TransformComponent.rotation },
    _x: {
      get() {
        return this.store.x[this.eid];
      },
      set(n) {
        this.store.x[this.eid] = n;
      },
    },
    _y: {
      get() {
        return this.store.y[this.eid];
      },
      set(n) {
        this.store.y[this.eid] = n;
      },
    },
    _z: {
      get() {
        return this.store.z[this.eid];
      },
      set(n) {
        this.store.z[this.eid] = n;
      },
    },
  });

  return eid;
}

function createThreeWorld(opts: WorldOptions = {}): World {
  const world = createWorld() as World;
  world.eid = addEntity(world);
  addComponent(world, RendererComponent, world.eid);
  RendererComponent.set(world.eid, opts.renderer || new WebGLRenderer());
  addComponent(world, TimeComponent, world.eid);
  TimeComponent.set(
    world.eid,
    opts.time || {
      last: 0,
      delta: 0,
      elapsed: 0,
    }
  );
  world.scene = addObject3DEntity(world, opts.scene || new Scene());
  world.camera = addObject3DEntity(
    world,
    opts.camera || new PerspectiveCamera()
  );
  return world;
}

function TimeSystem(world: World) {
  const now = performance.now();
  world.time.delta = now - world.time.last;
  world.time.elapsed += world.time.delta;
  world.time.last = now;
  return world;
}

function RenderSystem(world: World) {
  const renderer = RendererComponent.get(world.eid)!;
  const scene = Object3DComponent.get(world.scene)!;
  const camera = Object3DComponent.get(world.camera)!;
  renderer.render(scene, camera as Camera);
  return world;
}

const movementQuery = defineQuery([TransformComponent]);

function MovementSystem(world: World) {
  const { delta, elapsed } = TimeComponent.get(world.eid)!;
  const ents = movementQuery(world);
  for (let i = 0; i < ents.length; i++) {
    const e = ents[i];
    const obj3d = world.objects.get(e);

    TransformComponent.rotation.x[e] += 0.0001 * delta;
    TransformComponent.rotation.y[e] += 0.003 * delta;
    TransformComponent.rotation.z[e] += 0.0005 * delta;
    obj3d!.rotation._onChangeCallback();

    TransformComponent.position.x[e] += (Math.sin(elapsed / 1000) / 30) * delta;
    TransformComponent.position.y[e] += (Math.cos(elapsed / 1000) / 30) * delta;
    TransformComponent.position.z[e] += (Math.cos(elapsed / 1000) / 30) * delta;
  }
  return world;
}

const canvas = document.getElementById("canvas") as HTMLCanvasElement;
const renderer = new WebGLRenderer({ canvas, antialias: true });
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);

window.addEventListener("resize", () => {
  const camera = Object3DComponent.get(world.camera)! as PerspectiveCamera;
  const renderer = RendererComponent.get(world.eid)!;

  if (camera.isPerspectiveCamera) {
    camera.aspect = window.innerWidth / window.innerHeight;
    camera.updateProjectionMatrix();
  }

  renderer.setSize(window.innerWidth, window.innerHeight);
});

const camera = new PerspectiveCamera(
  70,
  window.innerWidth / window.innerHeight,
  1,
  1000
);

const world = createThreeWorld({ renderer, camera });

const pipeline = pipe(TimeSystem, MovementSystem, RenderSystem);

renderer.setAnimationLoop(() => {
  pipeline(world);
});

The main difference is just using a MapComponent instead of adding things to the world. I think this makes things more modular, especially when you're using Typescript and don't want to deal with extending the type of your world for each property you want to add.

I think the big initial focus should be on interoperability of the Object3D methods and properties. add(), remove(), parent, and children being some of the more challenging ones to deal with if you want to enforce any sort of consistency between adding/removing entities and components. As well as being able to traverse the scene graph from inside bitECS. Which could lend itself to some major performance gains!

Excited to jump in more!