Introduction

When we were developing WebVR applications such as a-painter, a-blast, and a-saturday-night that were designed to work across different hardware, we realized the importance of having a convenient way to map inputs to application-specific actions. Previously, Vive controllers were the only hand inputs available, so our logic was hardwired to them. As Oculus, Google, and Microsoft started releasing their platforms, we needed a scalable and convenient solution to support multiple input methods. Inspired by the Steam Controller API (This talk is a great introduction), I developed a system to map inputs to application-specific actions.

Today, A-Frame supports Vive, Oculus Touch, Microsoft Mixed Reality, GearVR, and Daydream controllers. You no longer have to deal with controller IDs or identify which indices correspond to which buttons. However, we still need to manually map specific actions to each controller input. Here’s an example from a-painter showing how tedious and error-prone that can be:

function setActionListener() {
  el.addEventListener('controllerconnected', (event) => {
    const controller = event.detail.type;
    if (controller === 'vive-controls') {
      el.addEventListener('buttonViveDown', this.doAction());
    } else if (controller === 'oculus-touch-controls') {
      el.addEventListener('buttonOculusDown', this.doAction());
    } else if (controller === 'microsoft-motion-controls') {
      el.addEventListener('buttonMicrosoftDown', this.doAction());
    }
    ...
  });
}

We have to listen for the controllerconnected event and then add a listener depending on the button we want to use for a specific action. More info about interactions and controllers can be found in the A-Frame.io docs.

Introducing the input-mapping system

Components should communicate using high-level action events instead of specific hardware events from the controllers, like triggerdown, trackpadchanged, or gripup. For example, the “paint-controls” component on a-painter listens for a triggerdown event to start painting. Wouldn’t it make more sense to listen for a paint event instead that is agnostic from the input method? For the paint action in A-Painter, it makes sense to use the trigger button, which is available on all controllers. It's rare for all controllers to have the same button, and remapping actions can get complicated.

This will help simplify our code and make it more clean and readable. For example, compare the old version of the code to toggle the menu in A-Painter:

init: function () {
    this.el.addEventListener('controllerconnected', function (evt) {
      self.controller = {
        name: evt.detail.name,
        hand: evt.detail.component.data.hand
      }
}

  addToggleEvent: function () {
    var el = this.el;

    if (this.controller.name === 'oculus-touch-controls') {
      if (this.controller.hand === 'left') {
        el.addEventListener('xbuttondown', this.toggleMenu);
      } else {
        el.addEventListener('abuttondown', this.toggleMenu);
      }
    } else if (this.controller.name === 'vive-controls' || this.controller.name === 'windows-motion-controls') {
      el.addEventListener('menudown', this.toggleMenu);
    }
  },

  removeToggleEvent: function () {
    var el = this.el;

    if (this.controller.name === 'oculus-touch-controls') {
      if (this.controller.hand === 'left') {
        el.removeEventListener('xbuttondown', this.toggleMenu);
      } else {
        el.removeEventListener('abuttondown', this.toggleMenu);
      }
    } else if (this.controller.name === 'vive-controls' || this.controller.name === 'windows-motion-controls') {
      el.removeEventListener('menudown', this.toggleMenu);
    }
  },

...with the version using input-mapping:

  addToggleEvent: function () {
    this.el.addEventListener('toggleMenu', this.toggleMenu);
  },

  removeToggleEvent: function () {
    this.el.removeEventListener('toggleMenu', this.toggleMenu);
  },

How to use it

Include the script in your HTML:

<script  src="https://rawgit.com/fernandojsg/aframe-input-mapping-component/master/dist/aframe-input-mapping-component.min.js"></script>

Define a mapping:

var mappings = {
  default: {
    'vive-controls': {
      menudown: 'toggleMenu'
    },

    'oculus-touch-controls': {
      abuttondown: 'toggleMenu',
      xbuttondown: 'toggleMenu'
    },

    'windows-motion-controls': {
      menudown: 'toggleMenu'
    }
};

And register it:

AFRAME.registerInputMappings(mappings);

As you might guess, if we want to add a new controller, we just need to modify the mapping without touching the menu code.

Mapping format

You can organize mappings in groups, ideally representing different states of our application. An experience could have a 'playing' state where the trigger is used to 'grab' things and a 'menu' state where the same button is used to 'select' an item from the settings menu. In the following example, we have two groups: 'default' (which is active by default) and 'painting'. Each group contains a list of controllers and the reserved keywords common (to define common mappings for all the controllers) and keyboard (for mappings events on key down, up and press).

  • Controllers: Each controller has a mapping between button events and actions that our application understands. For example, on vive-controls we could map trackpaddown to teleport and gripdown to undo.
  • common: Some mappings between buttons and actions may be common across all controllers. For example, 'triggerdown' is emitted by Vive, Oculus, and Microsoft controllers, so we can include a single map from 'triggerdown' to 'grab' instead of duplicating it in each controller's section.
  • keyboard: Mapping between keyboard keys and actions. The event names should start with the key value and have a '_up', '_down', or '_press' suffix. For example, we could map 's_up' to 'save'.
{
  default: {
    'vive-controls': {
      trackpaddown: 'teleport'
    },

    'oculus-touch-controls': {
      xbuttondown: 'teleport'
    },
    keyboard: {
      's_up': 'save'
    }
  },
  paint: {
    common: {
      triggerdown: 'paint'
    },

    'vive-controls': {
      menudown: 'toggleMenu'
    },

    'oculus-touch-controls': {
      abuttondown: 'toggleMenu'
    }
  }
}

Mapping API

Registering mappings

The input-mapping system exposes a global function to register your application’s mappings:

AFRAME.registerInputMappings(mappings, override)

Where 'mappings' is the mappings object as described below. If 'override' is set to true, it will override the previously registered mappings with the current one.

We can have more than one mappings group registered:

var mappingsApainter = {
    default: {
      ‘vive-controls’: {
        triggerdown: 'paint’
      }
    erasing: {
      'vive-controls': {
        triggerdown: 'erase',
      }
    }
  };

var mappingsTeleport = {
    default: {
      'vive-controls': {
        trackpaddown: 'aim’,
        trackpadup: ‘teleport’,
        triggerdown: ‘aim’,
      }
    }
  };

AFRAME.registerInputMappings(mappingsApainter);
AFRAME.registerInputMappings(mappingsTeleport);

It will combine both mappings, giving priority to the latest mapping values in case of conflict:

{
    default: {
      'vive-controls': {
        triggerdown: `paint’,
        trackpadup: ‘teleport',
        trackpaddown: 'aim'
      },
    }
    erasing: {
      'vive-controls': {
        triggerdown: 'erase',
    }
}

On the other hand, if we choose to override the second mapping, it will discard all the mappings from 'mappingsApainter' as if we had never registered them.

AFRAME.registerInputMappings(mappingsApainter);
AFRAME.registerInputMappings(mappingsTeleport, true);

Activate a mapping group

To activate a group of mappings, we just need to set the 'AFRAME.currentMapping' variable:

AFRAME.currentMapping = ‘erasing';

Updating your demos and components

I recommend that everyone start using mappings and stop registering listeners for each combination of buttons and controllers. For example, I’ve modified (PR #30) aframe-teleport-controls to listen to events instead of buttons for aim and teleport. With this change, I don’t need to update the code if a new WebVR-compatible controller appears. I’ll just need to include a new mapping in my app (or leave it in 'common' if the mapping is similar to an existing controller). If you want to see a migration use case using the component please take a look at the changes in A-Painter (PR #227) and A-Blast (PR #128).

Perhaps we could…

Some ideas and features:

  • Create a component that opens a configuration panel to show and let the user modify the mappings, storing them locally using 'localStorage'.
  • Include a 'common' state that contains all the mappings that are common to all states.
  • Use the mapping data to auto-generate tooltips for the controllers in our application.

tooltips

Final words

Currently, this is maintained as a third-party component at https://github.com/fernandojsg/aframe-input-mapping-component. It will likely ship with A-Frame by default in the 0.8.0 release (PR #3164). Please give it a try with your current project and send me any feedback. I hope this system will help you create more responsive VR experiences and components.