Introduction

When we were developing WebVR applications such as a-painter, a-blast, and a-saturday-night that were designed to work across different hardware, we realized the importance of having a convenient way to map inputs to application-specific actions. It used to be that Vive controllers were the only hand input available so our logic was hard wired to that. As Oculus, Google, and Microsoft started releasing their platforms, we needed a scalable and convenient solution to support multiple input methods. Inspired by the Steam Controller API (This talk is a great introduction), I developed a more convenient way to map inputs to application-specific actions.

Today, A-Frame supports Vive, Oculus Touch, Microsoft Mixed Reality, GearVR, and Daydream controllers. You don’t have to deal with controller IDs or identify which indices correspond to which button. We still need to manually map particular actions to each of the controller inputs. Here’s an example from a-painter showing how tedious and error prone that can be:

function setActionListener() {
  el.addEventListener('controllerconnected', (event) => {
    const controller = event.detail.type;
    if (controller === 'vive-controls') {
      el.addEventListener('buttonViveDown', this.doAction());
    } else if (controller === 'oculus-touch-controls') {
      el.addEventListener('buttonOculusDown', this.doAction());
    } else if (controller === 'microsoft-motion-controls') {
      el.addEventListener('buttonMicrosoftDown', this.doAction());
    }
    ...
  });
}

We have to listen for the controllerconnected event and then add a listener depending on the button we want to use for a specific action. More info about interactions & controllers can be found on the A-Frame.io docs.

Introducing the input-mapping system

Components should be able to communicate with each other using high level action events instead of specific hardware events from the controllers like triggerdown, trackpadchanged, or gripup. For example, the “paint-controls” component on a-painter listens for a triggerdown event to start painting. Wouldn’t it make more sense to listen for a paint event instead that is agnostic from the input method? For the paint action from a-painter makes sense to use the trigger button that is available in all controllers. It is a rare scenario where all controllers have the same button available and remapping action can get complicated.

This will help simplify our code a lot and make it more clean and readable. For example compare the old version of the code to toggle the menu in a-painter:

init: function () {
    this.el.addEventListener('controllerconnected', function (evt) {
      self.controller = {
        name: evt.detail.name,
        hand: evt.detail.component.data.hand
      }
}

  addToggleEvent: function () {
    var el = this.el;

    if (this.controller.name === 'oculus-touch-controls') {
      if (this.controller.hand === 'left') {
        el.addEventListener('xbuttondown', this.toggleMenu);
      } else {
        el.addEventListener('abuttondown', this.toggleMenu);
      }
    } else if (this.controller.name === 'vive-controls' || this.controller.name === 'windows-motion-controls') {
      el.addEventListener('menudown', this.toggleMenu);
    }
  },

  removeToggleEvent: function () {
    var el = this.el;

    if (this.controller.name === 'oculus-touch-controls') {
      if (this.controller.hand === 'left') {
        el.removeEventListener('xbuttondown', this.toggleMenu);
      } else {
        el.removeEventListener('abuttondown', this.toggleMenu);
      }
    } else if (this.controller.name === 'vive-controls' || this.controller.name === 'windows-motion-controls') {
      el.removeEventListener('menudown', this.toggleMenu);
    }
  },

...with the version using input-mapping:

  addToggleEvent: function () {
    this.el.addEventListener('toggleMenu', this.toggleMenu);
  },

  removeToggleEvent: function () {
    this.el.removeEventListener('toggleMenu', this.toggleMenu);
  },

How to use it

Include the script in your HTML:

<script  src="https://rawgit.com/fernandojsg/aframe-input-mapping-component/master/dist/aframe-input-mapping-component.min.js"></script>

Define a mapping:

var mappings = {
  default: {
    'vive-controls': {
      menudown: 'toggleMenu'
    },

    'oculus-touch-controls': {
      abuttondown: 'toggleMenu',
      xbuttondown: 'toggleMenu'
    },

    'windows-motion-controls': {
      menudown: 'toggleMenu'
    }
};

And register it:

AFRAME.registerInputMappings(mappings);

As you may guess if we want to add a new controller we just need to modify the mapping without touching the actual menu code.

Mapping format

You can organize mappings in groups, ideally representing different states of our application. An experience could have a playing state where the trigger is used to grab things and another state called menu where the same button should be used to select an item from the settings menu. In the following example we have two groups: default (As you may guess is the one active by default) and painting. Each group contains a list of controllers and the reserved keywords common (to define common mappings for all the controllers) and keyboard (for mappings events on key down, up and press).

  • Controllers: Each controller has a mapping between button events and actions that our application understands. For example, on vive-controls we could map trackpaddown to teleport and gripdown to undo.
  • common: Some mappings between buttons and actions might be common across all controllers. For example, triggerdown is emitted by vive, oculus and microsoft controllers, so we could include just one map from triggerdown to grab instead of duplicating it on each controller’s section.
  • keyboard: Mapping between keyboard keys and actions. The event names should start with the key value and the suffix _up, _down and _press. For example we could map s_up to save.
{
  default: {
    'vive-controls': {
      trackpaddown: 'teleport'
    },

    'oculus-touch-controls': {
      xbuttondown: 'teleport'
    },
    keyboard: {
      's_up': 'save'
    }
  },
  paint: {
    common: {
      triggerdown: 'paint'
    },
  
    'vive-controls': {
      menudown: 'toggleMenu'
    },

    'oculus-touch-controls': {
      abuttondown: 'toggleMenu'
    }
  }
}

Mapping API

Registering mappings

The input-mapping system exposes a global function to register your application’s mappings:

AFRAME.registerInputMappings(mappings, override)

Where mappings is the mappings object as described below and with override set to true it will override the previous registered mappings with the current one.

We could have more than one mappings group registered:

var mappingsApainter = {
    default: {
      ‘vive-controls’: {
        triggerdown: 'paint’
      }
    erasing: {
      'vive-controls': {
        triggerdown: 'erase',
      }
    }
  };

var mappingsTeleport = {
    default: {
      'vive-controls': {
        trackpaddown: 'aim’,
        trackpadup: ‘teleport’,
        triggerdown: ‘aim’,
      }
    }
  };

AFRAME.registerInputMappings(mappingsApainter);
AFRAME.registerInputMappings(mappingsTeleport);

It will combine both mappings, giving priority to the latest mapping values in case of conflict:

{
    default: {
      'vive-controls': {
        triggerdown: `paint’,
        trackpadup: ‘teleport',
        trackpaddown: 'aim'
      },
    }
    erasing: {
      'vive-controls': {
        triggerdown: 'erase',
    }
}

On the other hand if we choose to override the second mapping it will discard all the mappings from mappingsApainter as if we had never registered it.

AFRAME.registerInputMappings(mappingsApainter);
AFRAME.registerInputMappings(mappingsTeleport, true);

Activate a mapping group

To activate one group of mappings we just need to set the AFRAME.currentMapping variable:

AFRAME.currentMapping = ‘erasing';

Updating your demos and components

I recommend anyone to start using mappings and stop registering listeners for each combination of buttons and controllers. For example, I’ve modified (PR #30) aframe-teleport-controls to listen to events instead of buttons for aim and teleport, with this change I don’t need to update the code if new webvr compatible controller appears. I’ll just need to include a new mapping on my app (or just leave it in common if the mapping is similar to an already existing controller). If you want to see a migration use case using the component please take a look at the changes in A-Painter (PR #227) and A-Blast (PR #128).

Perhaps we could…

Some ideas and features:

  • Create a component that opens a configuration panel to show and let the user modify the mappings, storing them locally using LocalStorage.
  • Include a common state that contains all the mapping that are common for all the states.
  • Use the mapping data to auto generate tooltips for the controllers in our application.

tooltips

Final words

Currently this is maintained as a third-party component at https://github.com/fernandojsg/aframe-input-mapping-component It will likely ship with A-Frame by default in the 0.8.0 release (PR #3164). Please give it a try with your current project and send me any feedback. I hope this system will help you create more responsive VR experiences and components.