A-Painter

WebVR, A-Frame, three.js

VR paint application on the web

Try it! Source code Launch article Performance optimizations article

A-Painter

Introduction

I believe the first application I tried in my first 6DOF VR headset, the HTC Vive, was Tilt Brush by Google. It was a very pleasant experience, being able to draw in the 3D space, and with all these different brushes some of them with special effects.

I thought it could be great to try to replicate that experience on the web using WebVR and A-Frame, so I started by creating a simple application to draw colored lines:

A-Painter

I included support for the Vive's touchpad so with one controller you could change the brush shape and size, and with the other one the color:

It was a bit tricky to implement thick lines as I was using triangle strips for the meshes but three.js had not good support for them back then, so I needed to compute mostly everything myself from normals, to bbox, UVs and so

A-Painter

But few days after I had the initial prototype up and running:

Making it prettier

Just right after I finished my prototype we hired Diego Fernandez as technical artist and he modelled a custom vive controller with some helpers to show the current line width, color and the tip where the stroke will come from.

Right after that we included support for the Oculus Touch controls and we improved the UI panel where you can choose color, width, brush type or save your creation.

I also included my aframe-teleport-component so you could easily move around the room to build even bigger pieces of art:

The last feature I implemented before the release was to serialize the drawing to a JSON and push it to our Uploadcare account, so users could receive a link and share their creations.

Initial release

Here you can read the article I wrote when we release the first version of A-Painter. We shipped it with the following list of features:

  • Paint in 3D using VR controllers
  • Share drawings by copying and pasting an URL.
  • View 3D drawings anywhere both with and without a headset.
  • Paint on top of other people’s drawings and make them your own.
  • Drag and drop images and OBJ models from your desktop to the browser, for a template or starting point to paint over.
  • Save and load local binary files of your drawings.
  • Over 30 brushes with a custom A-Painter Brush API to easily create new ones.

Reception

The feedback we got from the community was super positive, and we even got featured in some digital magazines too. And it was nice to see it being featured in many presentations about WebVR and VR in general too.

I was also quite happy of the reception the demo had in our first Mozilla All-hands after we released it.

It was remarkable that our Chief of R&D, Sean White, used an A-Painter video as a background on his presentation at the plenary, and praised the work we did.

A-Painter

Kip also showcased* A-Painter in his presentation to talk about the state of the WebVR API in Gecko.

*Actually it was Diego who was doing a live painting exhibition, drawing the Mozilla VR fog logo

A-Painter

Improving A-Painter

Custom brushes

One of the first thing I worked on after the initial release was to add a simple API so people could build custom brushes, and it ended up being quite simple and we got several PRs with new brushes.

OBJ exporter

In the initial release I was using a custom JSON format to store the drawing but I thought it could be cool to store it to a common 3D file format as OBJ. So I built a custom exporter to OBJ which ended up being a bit tedious as I was using triangle strip and I needed to convert them to triangle lists instead before exporting.

Obj exporter

glTF exporter

Although OBJ is a really easy to use format, it has a lot of limitations. After a while I started working on a glTF Exporter for three.js and A-Frame and I believe the file format brings a lot of benefits to the workflow and the 3D web ecosystem. Once the components for A-Frame and three.js was working it was pretty straightforward to export the whole scene as glTF:

// Export the whole scene as glTF when the user press the key
if (event.keyCode === 71) {
  // Export to GTF when the user press the "g" key
  var drawing = document.querySelector('.a-drawing');
  self.sceneEl.systems['gltf-exporter'].export(drawing);
}

Obj exporter

Performance improvements

I wrote an article with a lot optimizations for A-Painter.

You can read the article to know more about the details but as summary please take a look at the following screenshot of the "before" and "after" loading the same scene:

  • FPS: ~22fps => ~60fps
  • DrawCalls: 3990 => 6
  • Textures: 13 => 3
  • Geometries: 4002 => 8
  • Entities: 4090 => 90
  • Load time: 554 => 311

Optimizations