Carmageddon

This doesn’t have anything with code or Javascript, but I just love playing the games I played when I was younger. The Settlers, Incubation and Sam and Max are at the top of the list, but in the racing section Carmageddon is just below Screamer and Ignition and even pulls ahead of Twisted Metal 2 and Destruction Derby.

And now they’re making a new game. In case you do not know: Carmageddon is a mixture of car racing and arena fighting, much like the better known Twisted Metal series. It’s also a very tasteless game where you get bonus points for running over pedestrians… you’ve got to realize that this game was made when the discussion about how violence in games influences people in real life was at an all time high. Carmageddon made fun of the whole discussion by making it clear that games have nothing to do with reality: Ironically it was censored in Germany with flying bombs instead of pedestrians because the consors were unable to see that.

Personally I think that the discussion about violence in games was (and still is) heading in entirely the wrong direction: Instead of educating people and teaching them that a computer is not a magic box we still act as if any of that stuff happening on screen was real.
The pixels we see on the screen have no more to do with real persons than children playing cops and robbers with a real gunfight. It’s just polygons, pixels and a bit of math… nothing more. That’s what we should get across and Carmageddon was a not-so-subtle nudge in that direction.

Anyway, they’re still trying to get the funding over at kickstarter.com and while I think the concept of Kickstarter will have to evolve at some point beyond the non-monetary-reward thing they’re doing now, I think in this case there’s enough talent (and they have invested enough of their own money) that it’s likely that they’ll get the game done. They’re not asking for much… $15 for what amounts to preordering the game and I think that’s fair. I’ve given $25 because I want access to the beta, which I still think is very reasonable.

Kickstarter Project Page

Scripting the windows commandline with Spidermonkey made easy

I frequently have to automate really simple tasks, like moving files with a certain filename to another directory and the Spidermonkey shell that now comes with XULRunner (thank you for that Mozilla, building it yourself was time-consuming and annoying) has become an invaluable tool.

Few people know how easy it is to use any Mozilla-JS based program (yes, that includes XULRunner and Firefox) to work with commandline programs. As with any other programming environment you just need to be able to call the popen function of the OS, which runs any command and returns the output as a stream.

Mozilla-JS does not include POPEN. However it does support CTYPES, a system for calling libraries. On Windows POPEN and everything else you need is in MSVCRT.

Opening MSVCRT with CTYPES is easy:

var msvcrt = ctypes.open("msvcrt");

Now you just need to declare what functions you need (_popen, _pclose, feof and fgetc in our case) and what types these require (first parameter is the name, second the interface type, third is the return type and everything else is the type of each argument):

var popen=msvcrt.declare("_popen",ctypes.winapi_abi,
    ctypes.void_t.ptr,ctypes.char.ptr,ctypes.char.ptr);
var pclose=msvcrt.declare("_pclose",ctypes.winapi_abi,
    ctypes.int,ctypes.void_t.ptr);
var feof=msvcrt.declare("feof",ctypes.winapi_abi,
    ctypes.int,ctypes.void_t.ptr);
var fgetc=msvcrt.declare("fgetc",ctypes.winapi_abi,
    ctypes.jschar,ctypes.void_t.ptr);

With this you can very easily build something like C’s SYSTEM call, just with the difference that it will return everything the program outputs through STDOUT:

function system(cmd,raw){
/* Open the program (Windows automatically
   uses cmd.exe for that), use raw mode
   if requested. */
    var file=popen(cmd,"r"+(raw?"b":""));

    var o=""; // STDOUT content
    var end=false; // End of STDOUT reached

/* Loop trying to get a character from STDOUT
   until feof informs us that the end of the
   stream has been reached */
    do{
        var c=fgetc(file);
        if(!(end=feof(file))) 
             o+=c; //Append current char
    }while(!end);

    pclose(file); // Close pipe
    return o; // Return 
}

And that’s really all you need (in this case to call DIR, split by newline and output the first result):

var dirs=system('DIR /B /S');
dirs=dirs.split("\n");

if(dirs.length)
	print(dirs[0].trim());

Zerg Rush Easter Egg

It’s incredibly funny, especially if you know that “Zerg Rush” is a very popular method for rooting Google’s Android OS. I wonder why that connection hasn’t so far popped up in any reports I’ve found about the Google Zerg Rush easter egg

Tip: Relaxing JSON strictness

I’ve decided to start posting about the little things I encounter day by day. Post #1: I needed less strict JSON parsing than Firefox usually does and a little RegEx seems to do the trick for now (it also allows for single word strings ├á la PHP, e.g. FOO becomes “FOO”:

var findRelevantToken=/(?:"(?:|.*?[^\\](?:\\\\)*)"|([\s,\:\{\[]|^)([a-z\_\$][a-z\_\$0-9]*))/gi;
   str=str.replace(findRelevantToken,function(m,ssep,nqs){
   if(nqs) return ssep+'"'+nqs+'"';
   else return m;
});

References vs. Pointers in Javascript

I’m as guilty as the next guy when it comes to the confusion surrounding the terminology used for variables that get assigned objects, but I think it’s time to get my act together and define a standard at least for myself:

If “new Object()” is an OBJECT, then “var obj=new Object()” makes obj a POINTER. Why is that?

The important thing here is considering what it is not. It is definitely not the actual OBJECT, because if that variable were the actual OBJECT (even if only at that particular moment), then “var obj2=obj” would have to make obj2 its own OBJECT too. But it doesn’t: Instead it points to the same OBJECT as obj does.

So that means that obj can only be a POINTER or a REFERENCE, since it doesn’t actually contain the OBJECT like it does for strings or numbers.

It’s tempting to call it a REFERENCE and I wouldn’t object to it if it weren’t for the fact that Javascript actually has real references, meaning variables that if set will cause another variable to change its value. This happens every time you access the arguments of a function:

(function(myArg){
    alert(myArg+" "+arguments[0]);
    arguments[0]="SET-BY-REF";
    alert(myArg+" "+arguments[0]);
    myArg="SET-BY-VARIABLE";
    alert(myArg+" "+arguments[0]);
})("INITIAL");

Which will print out “INITIAL INITIAL”, “SET-BY-REF SET-BY-REF”, “SET-BY-VARIABLE SET-BY-VARIABLE” in that order.

To be honest, this isn’t really something that you’ll encounter very often and it rarely is of much use, but it definitely makes myArg a REFERENCE of arguments[0], (meaning that any change to variable 1 will be reflected on variable2 – almost. In this case a change to a property will be reflected on a variable and vice-versa, but it’s so very close that it’s difficult to imagine that you’d use the term for variables pointing to an object, but not for this).

So, if obj can’t rightly be called a REFERENCE or an OBJECT, it has to be something else. And pointer seems to fit the bill very nicely, so that’s what I’m going to call it. Let’s see if it catches on ­čśë

Adding icons to bookmarklets … again

Show simple Demo

Yesterday I got a question on how to make my Bookmarklet-with-icon code work and sadly, it seems to be broken now. So I set out to find another method and strangely enough the solution that works with modern Firefox versions is quite simple and elegant:

Open the bookmarklet in "page mode" once after the user has added it as bookmarklet. This may sound like something that’s terribly hard to detect, but with a few simple tricks it’s not much of an issue: Mainly because you can make it so that the user barely notices if it fires to often.

  1. Just like the original bookmarklet-with-icon code, adding icons still relies on the behavior that a javascript: url which returns a string will cause the browser to display that string as a HTML document. So, the first step (again) is saving the HTML string somewhere. Of course, you can still add a bit of randomness, but for simplicity’s sake we’ll leave it out for now:

    <script>
      top["bookmarkletv2sample@tapper-ware.net"] = ''
        +'<!DOCTYPE html>'
        +'<html>'
          +'<head>'
            +'<title>My Bookmarklet</title>'
          +'</head>'
          +'<body></body>'
        +'</html>'
      ;
    </script>

  2. Now, we add the magic favicon sauce:

    <script>
      top["bookmarkletv2sample@tapper-ware.net"] = ''
        +'<!DOCTYPE html>'
        +'<html>'
          +'<head>'
            +'<title>My Bookmarklet</title>'
            +'<link rel="icon" href="http://www.my.domain/favicon.ico" />'
          +'</head>'
          +'<body></body>'
        +'</html>'
      ;
    </script>

  3. Like I said, we want that page to only appear very briefly, so we add a history.back(); call that will take us right back to the previous page. I’ve added a timeout as well, which is probably not necessary, but with hacks like that it’s better to err on the side of caution. It’s also a good idea to split the closing script tag to prevent any browser from misinterpreting it as the end of the current script block:

    <script>
      top["bookmarkletv2sample@tapper-ware.net"] = ''
        +'<!DOCTYPE html>'
        +'<html>'
          +'<head>'
            +'<title>My Bookmarklet</title>'
            +'<link rel="icon" href="http://www.my.domain/favicon.ico" />'
          +'</head>'
          +'<body>'
            +'<script>'
              +'window.onload=function(){'
                +'window.setTimeout(function(){'
                  +'history.back();'
                +'},250);'
              +'};'
            +'</scr'+'ipt>'
          +'</body>'
        +'</html>'
      ;
    </script>

  4. And that’s pretty much it for our magic assign-icon-from-within-bookmarklet page. Now we just have to change the actual bookmarklet to return that HTML string instead of running the bookmarklet code whenever a thusly-named HTML string exists on the current page. Assuming our bookmarlet is a plain old alert("This is my bookmarklet");void(0);, we have to wrap it into a little if-block like this:

    if(top["bookmarkletv2sample@tapper-ware.net"]){
       top["bookmarkletv2sample@tapper-ware.net"];
    }else{	
       alert('This is my bookmarklet');
      void(0);
    }


    which, written as a bookmarklet one-liner, turns into

    <a href="javascript:if(top['bookmarkletv2sample@tapper-ware.net']){top['bookmarkletv2sample@tapper-ware.net'];}else{alert('This%20is%20my%20bookmarklet');void(0);}">My Bookmarklet</a>

  5. If you put those blocks somewhere on your page it will already work… sort of: If you drag the link to your bookmarks-toolbar, then click it, you’ll briefly see an empty page and the icon will appear. Now, all we have to automate is that one click. Luckily, all we have to do for that is listen for a dragend event, which occurs whenever the user drags the link anywhere. It also occurs during any other drag operation but since that brief refresh is barely noticeable it doesn’t really matter if we try to assign the icon too often. All we have to do for that is add an ondragend attribute to the a element:

    ondragend="this.click();"


    Which finally turns our “A” element into this:

    <a ondragend="this.click();" href="javascript:if(top['bookmarkletv2sample@tapper-ware.net']){top['bookmarkletv2sample@tapper-ware.net'];}else{alert('This%20is%20my%20bookmarklet');void(0);}">My Bookmarklet</a>

And that’s all there is to it. Include the link and the script element on your page, tell the users that they should add it by dragging the link and that it won’t work on the current page, adjust favicon-url and exchange the example with your own JS code. Done.

Listening in on WebGL : WebGLProfiler

I’ve been trying to learn a thing or two about WebGL, but I’ve encountered a very simple problem: Most examples you find on the web have a very complex source code that is great for demonstrating how the particular framework the developer has written/used works, but usually not so great for learning WebGL itself.

For example take a look at this sample that often turns up when you ask about realtime shadows. It’s a beautiful little thing, but it uses Google’s O3D framework, so if you want to learn about the GL code involved, without learning about O3D, you’re pretty much out of luck.

So I decided to hack up something that can give me a runnable version of all the GL calls involved in this, complete with symbolic constants at least for the most common GL calls (have a look at the top of the source to see what arguments are currently resolved). It’s very hackish and may not always work perfectly (for example, it replaces all intervals and timeout with a value of 10000 so that you don’t get flooded with frame updates, meaning that it could take a while for what you’re interested in to appear), but it creates a WebGL program that’s strictly linear and pretty easy to read. Since it was built for my own use it also has some pretty ugly hacks that could be solved more elegantly with enough time (for example it checks for the document’s load state using an interval waiting for the title tag to appear), but it works. The resulting HTML page will have snapshots of canvases or videos embedded, while the images are links to the originals.

The big drawback so far is that you’ll only get the raw values of matrices, since these are not handled by WebGL, but by a few different libraries. It’s a big drawback, since it means that you will have to look them up in the original source code, but if they are probably named (global variable names are inserted by the profiler as comments and if there is none, you can always look at the names used inside the shaders), you can usually guess them pretty well.

I’ve wrapped the whole thing into a bookmarklet for easy use. When invoked, it will load the current document into the right frame, monkeypatch it and on first use of the webgl context start reporting to the left frame. The left frame will then contain a complete HTML document that you can just copy/paste and it will produce the same output as the original page. Be warned, it will VERY likely only work in current Firefox nightlies.

Perhaps the best way to illustrate it is showing the output of a few samples (look at the source):

You can find the bookmarklet at http://www.tapper-ware.net/data/devel/web/apps/JS.WebGLProfiler/.

License is GPL3 (or later). See the sourcefor details.

Teaching IE8 a bit of Canvas with DataURIs: ducie8.js

Demo Link (ducie8.js uses a native Canvas implementation if found, so you can only really test it in IE8)

I’m still looking for ways to apply this to previous IE versions (which is pretty hard considering that there are no DataURIs, just a bunch of bugs which influence each other is such ways that they might be able to provide similar functionality), but even only with IE8 compatibility this could prove valuable, seeing as IE8 is the highest version that you can ask IE users to update to without a change of operating system or browser.

I was exploring DataURIs in IE a while back with Joseph Huckaby when he published his article on palette animation, since altering the palette in a BMP data url is faster than doing it manually for every pixel through JS and works in IE8 as well. We gave up when we realized that WebKit would simply leak all memory associated with our DataURIs and never release it (2GB can pile up quite quickly if you use about 20MB per second). I still have the source around if anybody is interested, but I’m afraid I don’t have any public domain images to put together a demo.

Anyway, this didn’t really lead anywhere, but it gave me the idea to try emulating Canvas with BMP DataURIs. Now, DataURIs in IE8 are severely limited. Most notably they can’t go beyond 32kb (in fact, no URL can in IE8). The solution is pretty obvious: Use multiple small ones (we already had to do that for the palette animation). Other than that you need a base64 encoder since IE8 doesn’t have an btoa() function and that’s pretty much it.

Of course, when it comes to actually implementing something like that things are never simple and there are a few obstacles that ducie8.js will very likely never overcome. Like the facts that trying to emulate a 32bit Canvas with transparency would require using PNG DataURIs, which in turn would mean that we can’t recalculate the base64 value for an individual pixel (since base64 strings can only be split every 24bits) and would have to recalculate the checksum for the PNG on every change (maybe there are some bugs that allow us to get away with a corrupt checksum, we’ll see). Or that drawImage won’t be able to access normal HTML Image elements (since we can’t access their pixel data), but will require a special Image-like element which will internally use XMLHttpRequest and responseBody (and will only be able to read uncompressed formats unless somebody manages to do an incredibly fast PNG decoder in IE8). Or that we never know when the JS has finished drawing a single frame (which is why ducie8.js right now uses a timeout and updates the BMPs with 1 frame delay). Or that it will never be fast enough for anything but Minesweeper (it’s still surprisingly fast considering that the code is still pretty unoptimized). Or that you have to use the setProp method for setting properties since we often need to run a setter (ducie8.js adds setProp to a real Canvas implementation if found, so that you can at least use the same code for both).

What it can do is provide Canvas in cases where using the server is simply not possible or very undesirable, but the result is mostly static once rendered. Graphs. Buttons. The current version of ducie8.js is very light on features. width, height, fillStyle and fillRect. That’s it. enough for a simple drawing program to test the speed and the architecture, but little else. However the vast majority of Canvas features could be implemented in a pretty straight-forward way and I’ll be happy to set up a project if there’s any interest.

P.S. I should probably stress that the current version uses a triple-cache (nummeric pixel data > base64 pixel data > data url) that’s actually very slow for solid rectangles and really meant for per-pixel access with transparency. So don’t think the current speed is the limit: Internally these are all Arrays and can be made a lot faster by using Array methods instead.

Benchmarking and code optimization in the browser: Emulators

Update: IE9pp4 gets 9710ms , still slowest

The GameBoy emulator posted over at CodeBase is very impressive. Unfortunately, it’s borderline unusable on anything but Chrome. I know Chrome’s V8 engine is always reported to be the fastest JS Engine, but the difference is ridiculous.

Fast Javascript means pleasing the Tracing algorithm built into browsers. If that algorithm can’t understand what you’re trying to do it will fall back to the interpreter and you don’t want to do that (even with Nitro and J├Ągermonkey). So it was pretty obvious that the problem was just that: Firefox simply didn’t understand the code.

I dove into the JSGB source and found one problem straight away… a problem that is handled remarkably well by Chrome, but even there it makes things about 15% slower. An emulator is basically a big switch statement: If opcode==a then do A, else if opcode==b then do B. In Javascript, you may be tempted to do something like this:

var table={
  a:function(){A;},   
  b:function(){B;},
}
table[opcode]();

That’s exactly what JSGB does. It’s all fine and dandy if you have few calls, but you have to remember that running a function means much more than just running the code: A new context has to be created, the name lookup chain altered, a new analysis profile for tracing created, arguments passed. An emulator does this constantly, so it’s wasting tons of cycles when all you really need is a switch statement:

switch(opcode){
  case a:A;break;
  case b:B;break;
}

The next problem are closures. Going up through the name lookup chain takes its toll, but besides that closures make interpreting Javascript code more complex, making it more likely for the Tracing algorithm to bail out. Apparently on Firefox that’s what’s happening here. An example:

var value=0;
function increment(){
  value++;
}

This may seem simple enough, but already it can cause problems. If you want to make sure, pass everything as an argument. And I mean everything: I ended up even passing the window object as argument. It may seem slower at first, but the Tracer will work and usually take care of it.

var value=0;
function increment(value){
  return ++value;
}

To test this, I wrote my own little emulator (on the page you’ll also find results for Firefox, Chrome and Opera)… it doesn’t emulate a real machine, just some commands that I needed to implement a sample program: 16bit values, 64k memory, 4 registers plus overflow. Enough to bounce a ball around the screen and see how well the browser handles it using either a switch statement or function lookups.

The results are interesting: Firefox is actually fastest when it comes to pure processing (with the switch statement), followed by Chrome and far in the distance Opera. When you’re adding screen updates into the mix (timeouts) the picture changes a bit: Opera now takes first place since it’s able to use the whole timeout period to calculate ahead, but that’s because my sample program is very short. It wouldn’t be enough for a GameBoy emulator.