Dynamically textured animations in the browser

It’s probably best to let you play with this little demo first. The animation is stored only once and the texture is applied dynamically. Keep in mind that this is only a techdemo. Somebody more familiar with animation could use this technique for something truely great (think along the lines of you uploading an image of yourself and suddenly you’re appearing in a movie, thanks to the combined power of the video tag and this).

The technology is not the limiting factor here, just my artistic skills.

The concept

.. .is actually pretty simple. An animation is rendered with a placeholder, instead of a real texture. This placeholder contains the position in the texture (encoded as color). Specifically, the model was rendered with this texture, using a planar projection for UV coordinates.

When the animation is shown, the pixels get replaced again with pixels from some texture that the user or the developer specifies.

All this can be done with SVG filters, specifically feDisplacementMap. If you want to know in detail how it’s done, have a look here. The only real difference here is that instead of a static map, a series of images is composed into an animation.
You may also want to look at the source of the demo. Everything needed to recreate the effect shown here is marked with the keyword IMPORTANT.

Can only Firefox 3.5 do this? And why aren’t you using the video tag?

Most modern browsers can do this, but there are a few differences that you should be aware of: To make this work, your video has to be stored in a lossless format.

For Gecko based browsers that’s usually APNG in an IMG or maybe at a later time Theora (the current encoders don’t allow for for lossless YUV compression, but that’s likely a limitation of the current tools, not the format, as we’ve seen with JPEG or h.264) via VIDEO. For Webkit it’s lossless h.264 and for legacy browsers it may be a greyscale GIF containing the color channels side by side, so they can be recombined by the filter.

Older browsers also need the filter applied inside an SVG document, so you’ll have to load an SVG in an iframe to do the rendering.

To sum it up: This demo only works in Firefox 3.5 because I didn’t want to spend more time optimizing it. I wanted to show you what can be done and Firefox 3.5 made it easiest for me. But you can make it work. For a few pointers, see this file, which implements a similar solution in pure SVG.