Video tag user to Apple: Get your act together.

I’m currently trying to get a video of the last demo online and all things considered, it has been a good experience… except for Apple’s Safari browser. It’s nice that they support the video tag, but their implementation is pretty much useless:

  • Don’t say that you support a feature when you really don’t.

    Apple’s implementation lacks support for the controls attribute, which tells the browser to display a native interface for video playback. I don’t really have a problem with this, but why the heck does Safari tell me that it does support it?

    The controls attribute is treated as if it was supported with the controls property mirroring the value of the controls attribute. So myVideoElement.controls is true or false instead of the undefined I’d expect for an unknown attribute.

    In the end, that means that I have to do browser sniffing to load my own interface for Safari and I thought that we had all agreed by now that this is a bad thing

  • Don’t say that you support h.264 when you really only support is your own special flavour.

    There’s a lot discussion going on over which codec to use for video, but something that’s often overlooked is that Apple’s h.264 decoder is a proprietary mess that will only accept Quicktime created MP4s. Since video in Safari uses the Quicktime backend that means that you won’t be able to actually use h.264 unless you fork some cash over to Apple for Quicktime Pro, even when there are many better and cheaper alternatives. This is exactly the lock-in that shouldn’t happen with video.

    The only real alternative for now is using the MPEG-4 Advanced Simple Profile (better known as DivX or XviD) and suddenly all the better quality you were promised in exchange for having to use a proprietary format is gone.

Fake Voxels with filters

Click the image above to get to the demo

Video

Download/View in external player

Voxels are a way to display 3D data that dates back to the days before 3D accelerators became popular. Voxels (VOLume piXELS) don’t store models as triangles or polygons, but instead
function much like a traditional bitmap, only these bitmaps have 3 dimensions, much like a video (where the third dimension is time). Today you find them mostly in medical scanners.

This demo however doesn’t use real voxel data (we get to the rendering in a second, this is just about the data itself), but a variant where there is only a two dimensional bitmap and a third channel
that contains the Z position of the pixel. The benefit is that there’s a lot less data, but there can only be one color for each pixel at all depths, which makes it well suited for terrains, but not so
much for anything else. This is the way most games used voxels and it even has found its way into modern games as "parallax mapping" for textures, which is really just voxel rendering
implemented as a shader.

The rendering itself isn’t real voxel rendering either. That would require a 3D buffer that’s currently not available to filters. Instead, a series of perspective renderings is just put
on top of each other, so these voxels don’t have any side surfaces, just the top. At steep angles this would pose a problem, but since we’re looking at it from (roughly) a 45 degree
angle, it works well enough.

Code-wise this is really just a simple evolution of the MarioKart demo and if you
want to know how the perspective rendering is done you should look there.

The only real difference is that this uses 5 different perspective renderings, stacked upon each other. Then there’s a bit of fairly simple brightness/contrast adjustments: The bottom layers
are made a little darker and the alpha channel gets higher contrast and a brightness adjustments depending on its height. The top ones are made to display only where the alpha
channel is entirely opaque/very bright, while the bottom ones pretty much ignore the alpha channel.

Frankly, I didn’t expect this to work, primarily because the filter is very, very long. It needs to process around 30 operations to generate each frame (4 operations for each of the 5 layers + 5 for the texture + 3 for the texture transformation)
, and that’s with a 768×512 surface. It’s not very difficult to understand, but there’s simply a lot of processing needed and the speed at which Firefox renders this is nothing short of amazing.

Font work: schmucker-sans-nothrills-rev0

Click to get to the SVG.

When font-face made it’s appearance in all major browsers I was really, really excited. Until I learned that I wouldn’t be able to use any commercial fonts and even most free fonts. So I’ve started working on my own. It’s terribly basic and isn’t even complete yet, but I thought that maybe some people could even have use for an incomplete font. It is currently available as an SVG containing the letters A-Z, a-z and the numbers 0-9.

schmucker-sans-nothrills-rev0 is built from first principles: Lay out a grid for your letters, create one arc for the round parts and try to assemble all letters from these parts. It’s not exciting, but it should be easily readable so you can also use it for longer text paragraphs, not just headlines.

The most interesting thing is probably the license, which allows for pretty much anything. I’m putting this into the public domain.

So, create fonts or use it in your logos or basically do whatever you like. Would be nice if you sent me a mail though.

Dynamically textured animations in the browser

It’s probably best to let you play with this little demo first. The animation is stored only once and the texture is applied dynamically. Keep in mind that this is only a techdemo. Somebody more familiar with animation could use this technique for something truely great (think along the lines of you uploading an image of yourself and suddenly you’re appearing in a movie, thanks to the combined power of the video tag and this).

The technology is not the limiting factor here, just my artistic skills.

The concept

.. .is actually pretty simple. An animation is rendered with a placeholder, instead of a real texture. This placeholder contains the position in the texture (encoded as color). Specifically, the model was rendered with this texture, using a planar projection for UV coordinates.

When the animation is shown, the pixels get replaced again with pixels from some texture that the user or the developer specifies.

All this can be done with SVG filters, specifically feDisplacementMap. If you want to know in detail how it’s done, have a look here. The only real difference here is that instead of a static map, a series of images is composed into an animation.
You may also want to look at the source of the demo. Everything needed to recreate the effect shown here is marked with the keyword IMPORTANT.

Can only Firefox 3.5 do this? And why aren’t you using the video tag?

Most modern browsers can do this, but there are a few differences that you should be aware of: To make this work, your video has to be stored in a lossless format.

For Gecko based browsers that’s usually APNG in an IMG or maybe at a later time Theora (the current encoders don’t allow for for lossless YUV compression, but that’s likely a limitation of the current tools, not the format, as we’ve seen with JPEG or h.264) via VIDEO. For Webkit it’s lossless h.264 and for legacy browsers it may be a greyscale GIF containing the color channels side by side, so they can be recombined by the filter.

Older browsers also need the filter applied inside an SVG document, so you’ll have to load an SVG in an iframe to do the rendering.

To sum it up: This demo only works in Firefox 3.5 because I didn’t want to spend more time optimizing it. I wanted to show you what can be done and Firefox 3.5 made it easiest for me. But you can make it work. For a few pointers, see this file, which implements a similar solution in pure SVG.

Smooth image filtering for MSIE6+ and Firefox 2+

Sometimes working on a production website reminds you of problems you thought that were long gone. In this case, I needed smooth image scaling for the majority of browsers and yes, that sadly still includes MSIE6 and Firefox2.

The solution for MSIE6 is pretty well known, so I’ll keep this short: You need to apply the AlphaImageLoader with sizingMethod scale:

<img src="blank.gif"
   style="
      filter:
         progid:DXImageTransform.Microsoft.AlphaImageLoader(
            src='myImage.jpg',
            sizingMethod='scale'
      );
   "
/>

Firefox2 is actually more difficult here. There are no hidden properties or anything that will magically render your image smoothly. But there’s another way to do it. Namely, the SVG renderer, which uses smooth image rendering by default. So you simply need to link in an SVG via the object tag instead of the image:

<?xml?>
<svg
  xmlns="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink"
>
   <image xlink:href="myImage.jpg"
     width="100%" height="100%"
     preserveAspectRatio="none"
   />
</svg>

If you want to automate the whole thing, the easiest way is using data urls, because it means that you don’t have to create separate documents. Instead you just let a script run over the page and replace the imgs with objects. The only real difference with data urls is that you have to urlencode the characters, which makes the final object tag something like this (without any linebreaks):

<object data="data:image/svg+xml,%3C?xml%20?%3E%3Csvg%20
xmlns=%22http://www.w3.org/2000/svg%22%20
xmlns:xlink=%22http://www.w3.org/1999/xlink%22%3E
%3Cimage%20xlink:href=%22test.jpg%22%20
width=%22100%25%22%20height=%22100%25%22
%20preserveAspectRatio=%22none%22%20/%3E
%3C/svg%3E" type="image/svg+xml" />

That’s pretty much it. Now I’m off to investigate Safari and Opera.