I love Passbolt

I have to say I’m pretty impressed with Passbolt (a PHP password manager), mainly because it’s so braindead simple. Rather than implement some complex protection scheme on the server, it just transfers the responsibility to the client. The client is the only one who knows the Private Key and if somebody is in a group, then the secret is simply stored encrypted once for every member of the group, with each copy being encrypted with the group member’s public key.

That way whoever copies the data from the server has no way of getting to the private key (maybe by manipulating the web interface, but I’m actually not so sure about that since the web interface actually does the majority of its work through an extension… I’m not sure if the web interface has direct access to the private key).

This offers some interesting possibilities. Namely that you can export the secrets table from the Passbolt MySQL database and it will still be secure as well as readable by each member of a group, because there will be a separate entry for each user.

Decryption is very straight forward since Passbolt itself uses OpenPGP. Just grab a copy, put it in a script and all you have to do is

let privateKeyArmored=document.getElementById("privateKey").value;
let passphrase=document.getElementById("password").value;
let message=document.getElementById("encryptedMessage").value;
let privateKey = await openpgp.decryptKey({privateKey: await openpgp.readKey({ armoredKey: privateKeyArmored }) ,passphrase });
let {data:decrypted} = await openpgp.decrypt({message: await openpgp.readMessage({ armoredMessage:message}),decryptionKeys: privateKey});
document.getElementById("decryptedMessage").value=(decrypted);

And voilà your secret is decrypted. That way you can easily store a safe offline copy that you can decrypt on demand.

Javascript OOP

I thought it would be nice to have yet another person trying to explain how object-oriented programming works in JS, why it looks so different and what the pitfalls and benefits are.

Let’s start of with the basics. A class in any other programming language usually looks something like this

class Foo{
 constructor Foo(){},
 private attribute Type a,
 public attribute Type b,
 private method ReturnType c(){},
 pubic method ReturnType d(){}
}

Coming from a normal object oriented language and having been taught the usual explanation you expect this to provide a kind of contract that the compiler will enforce. If a required type is specified somewhere it will only accept instances of classes descended from this type, you expect it to load the right method based on the signature of the arguments and so on and so forth. On the other hand you don’t really expect the class itself to be accessible by your code. The instances and methods sure, but changing the class at run-time? Impossible.

Well, as usual, Javascript is very different. It’s a lot more powerful since any class can actually be modified on the fly, because there are no real classes. Instead, JS has a concept of constructors and inheritance. Nothing more. Everything else you want you can implement on top of that. You can chose to implement anything you want, but that’s all the language itself provides.
So let’s have a look at constructors work. In other languages, constructors are something optional. In JS, they are the only way to create an instance (aside from a deprecated __proto__ attribute or using [sg]etPrototypeOf in current browsers, but I’d stay away since this could have unintended consequences).

A constructor is just a common function. ANY function can become a constructor, all you have to do is call it with “new”

var x=new (function(){})();

will create an instance of our anonymous constructor.

the only two things that “new” really changes is creating a new object, running the supplied function in the context of this object and if nothing is returned, automatically returning that object.

So why don’t we do

var x=(function(){ return {}; })();
//or better yet
var x={};
//instead of
var x=new (function(){})();

The basic answer is because the special object that is created if you call something with “new” has some very special traits. Specifically, it has its prototype set to the function you used to create it.

So what the heck is a prototype? A prototype is Javascript’s model for inheritance. It’s something like a class, if you access a trait of an object you can access anything that’s provided by the prototype, but it’s a lot more dynamic. That’s because the prototype of an object is itself nothing but an object which you can access and modify at any time.

The logic of Javascript is simple. If you have an object and something tries to access some trait of it, look if the object has it, if not check its constructor’s prototype and access that instead. But since the prototype
itself is nothing but an object, it has a prototype of its own, so you can extend the chain endlessly.

So how do we access the prototype object. Pretty simple. Any function has a prototype attribute, which is a standard object.

So where in any normal programming language we’d declare a method or attribute, in JS we just assign them. We can even change them AFTER we created an instance.

//Define our constructor
var MyConstructor=function(){};
//Create an attribute on it
MyConstructor.prototype.magicNumber=666;

//Create an instance
var myObject=new MyConstructor();
//Output the magicNumber, which at this point is found on the prototype, namely 666
console.log(myObject.magicNumber);
//Change the magicNumber on the prototype
MyConstructor.prototype.magicNumber=42;
//Since the instance still has no own magicNumber attribute, inspecting it will give the changed magicNumber of the prototype: 42
console.log(myObject.magicNumber);

I know what you’re saying: So now we can change a attribute by accessing it in an even more annoying way. Great.
But there’s a method to this madness. And that’s the fact that only reading accesses the prototype chain. Writing on the other hand does not and always modifies the object you’re targeting.
So you can read from the prototype object, but write to the target object, giving you a template mechanism which is not unlike that of a class.

var MyConstructor=function(){};
MyConstructor.prototype.magicNumber=666;
var myObject1=new MyConstructor();
var myObject2=new MyConstructor();
//From Prototype:666
console.log(myObject1.magicNumber);
myObject1.magicNumber=42;
//From myObject1:42
console.log(myObject1.magicNumber);
//Since myObject2 has no magicNumber, from prototype:666
console.log(myObject2.magicNumber);
//Prototype:666
console.log(MyConstructor.prototype.magicNumber);

See, we changed the magicNumber, but it didn’t affect the prototype or any other instances. We haven’t really defined a class and the mechanism is entirely different, but we have an object now that behaves as
if we had classes (modern JS even has a class syntax… I’d stay away, because while it now looks even more like a class in a statically typed programming language, it’s still just an alias to create a prototype,
leading to much unnecessary confusion).

You can check if a attribute resides on the object itself or further up the prototype chain by using hasOwnattribute. toString for example is inherited, so hasOwnattribute returns false, even though we can see it.

var o={"a":"attributeA"};
console.log(o.a);
console.log(o.hasOwnattribute("a"));
console.log(o.toString);
console.log(o.hasOwnattribute("toString"));

So now we know that we can create an object that has a prototype link to another one and that we can use this for attributes. It actually works the same way for functions;

var MyConstructor=function(){};
MyConstructor.prototype.hello=function(){
 console.log("World");
};
var myObject=new MyConstructor();
myObject.hello();

So why do I say functions, not methods? Because methods are a concept that again, doesn’t really exist in Javascript. What we’d usually refer to as method in JS are just function attributes of an object. We can copy them we can overwrite them, we can even move them between objects. That’s because while an execution scope exists in JS, it’s not determined during the creation of the function, but once again at run time.
We can access the current scope of a function using the “this” keyword, the same way we did in the constructor:

var MyConstructor=function(){};
MyConstructor.prototype.magicNumber=666;
MyConstructor.prototype.sayMagicNumber=function(){
 console.log(this.magicNumber);
};
var myObject=new MyConstructor();
myObject.magicNumber=42;
myObject.sayMagicNumber();

Looks an awful lot like a normal method doesn’t it? But we can see it doesn’t work this way if we store a reference to that function and call that instead.

var MyConstructor=function(){};
MyConstructor.prototype.magicNumber=666;
MyConstructor.prototype.sayMagicNumber=function(){
 console.log(this.magicNumber);
};
var myObject=new MyConstructor();
var myFunction=myObject.sayMagicNumber;
myFunction();

Suddenly all it says is “undefined”. Because without being called directly on an object, the function doesn’t know which object it belongs to. The magic happens in the actual “.” or [“”] lookup. Here “this” is set to
the correct object. This is why functions assigned to the prototype don’t return the prototype object as “this”, but the one on which they were called, namely the instance object.

So let’s review for a moment:
1. There are no classes in Javascript
2. However there is a prototype chain which allows access to an objects attribute to bubble up to another object
3. There are no methods
4. But functions are called in the scope of the object to which they are assigned.
5. Everything is dynamic

Understanding these fundamental principles is really all it takes to understand object oriented programming in Javascript. So let’s look at how we can use these principles to emulate even more stuff we are used to
from class-based languages. Inheritance for example. We know that accessing something on an instance looks, if that attribute doesn’t exist, at the prototype. But since the prototype is nothing but an object itself
we can use this to link multiple prototypes to a single instance.

var MyOriginalConstructor=function(){};
MyOriginalConstructor.prototype.a=1;
var MyDerivedConstructor=function(){};
MyDerivedConstructor.prototype=new MyOriginalConstructor();
MyDerivedConstructor.prototype.b=2;

var myInstance=new MyDerivedConstructor();
console.log(myInstance.a+” “+myInstance.b);

See what we did there. We made prototype of MyDerivedConstructor an instance of MyOriginalConstructor. So now every time we access a attribute of the instance it will first look at MyDerivedConstructor’s prototype
and if it can’t find anything it will look further and see if it can find anything on MyOriginalConstructor’s prototype. There is a slight pitfall here. Creating an instance of MyOriginalConstructor may have unintended
side-effects. So what we usually do is create a temporary constructor:

var MyOriginalConstructor=function(){
 console.log("We don't want to run this just for inheritance");
};
MyOriginalConstructor.prototype.a=1;

var MyDerivedConstructor=function(){};

var MyTemporaryConstructor=function(){};
MyTemporaryConstructor.prototype=MyOriginalConstructor.prototype;
MyDerivedConstructor.prototype=new MyTemporaryConstructor();

MyDerivedConstructor.prototype.b=2;

var myInstance=new MyDerivedConstructor();
console.log(myInstance.a+" "+myInstance.b);

However we may want to run the original constructor when the derived constructor is called. We can actually already do this and make sure it runs in the correct scope since we know that all that is used to
determine what “this” refers to is the object on which the function is called. So we can just make the original constructor a attribute of our derived prototype and it will work:

var MyOriginalConstructor=function(){
 this.message="set by MyOriginalConstructor";
};
MyOriginalConstructor.prototype.message="default";

var MyDerivedConstructor=function(){
 this.MyOriginalConstructor();
 console.log(this.message);
};

var MyTemporaryConstructor=function(){};
MyTemporaryConstructor.prototype=MyOriginalConstructor.prototype;
MyDerivedConstructor.prototype=new MyTemporaryConstructor();
MyDerivedConstructor.prototype.MyOriginalConstructor=MyOriginalConstructor;
var myInstance=new MyDerivedConstructor();

You see how just making it a attribute changed everything. There’s also another way that’s often more convinient. We can specify what to set “this” to by using the functions “call” and “apply” in the Function prototype. They are basically the same, just that “call” requires you to specify arguments one by one while apply uses an array:

var myFunction=function(msg1,msg2){console.log(this+" "+msg1+" "+msg2);};
myFunction.call("Hello","Call","Bar");
myFunction.apply("Hello",["Apply","Bar"]);

These are identical as you can see. There’s a third function that’s relatively recent (I won’t explain how to emulate it for now, since this touches an entirely unrelated topic: closures). Bind. Bind gives you
a way to get a proxy function which will always run the original one in the given scope.

var myFunction=function(msg1,msg2){console.log(this+" "+msg1+" "+msg2);};
var myProxy=myFunction.bind("Bound");

myProxy("Arg1","Arg2");
myProxy.call("Hello","Arg1","Arg2");

You see how no matter what we do “this” always points to the string “Bound”. This is something you will have to deal with due to the callback-based nature of Javascript. Often a function, for example addEventListener
only allows you to specify a callback. A callback is just a function reference, so it doesn’t know anymore on which scope to run. Using bind you can make sure it still does.

So, what else can we do? What would we expect to be able to do? An often requested feature is method overloading. Again, Javascript can’t do that, since it doesn’t even have this concept of classes. Everything you create is still of type object, so how would this even work? Again, JS has a concept that doesn’t directly mimic this, but it has something that’s reasonably similar. You can check the prototype chain and see if a function’s prototype is in an object prototype chain. For this, there’s a special operator. The instanceof operator. You may already have encountered this when you were researching how to tell a plain object from any array. Since an array is not a type of its own, here too you have to check the prototype chain.

console.log([] instanceof Array);

So while we cannot overload a function, we can create a proxy function that will run different code based on type and prototype of the arguments:

var MyConstructor=function(){};

var myProxy=function(unknownTypeArg1){
 if(typeof(unknownTypeArg1)=="string")
 console.log("string");
 else if(unknownTypeArg1 && typeof(unknownTypeArg1)=="object" && unknownTypeArg1 instanceof MyConstructor)
 console.log("MyConstructor");
 else
 console.log("other");
}

myProxy("Hello");
myProxy(new MyConstructor());
myProxy(3);

There’s another useful object in JS that can make this even more versatile and that’s the arguments object. Every time you call a function an arguments object is created, containing all the parameters in an Array-like format. Checking this way is a lot more convenient than naming all arguments.

var myProxy=function(){
 if(arguments.length==1 && typeof(arguments[0])=="string")
 console.log("string");
 else if(arguments.length==1 && typeof(arguments[0])=="object" && arguments[0] instanceof MyConstructor)
 console.log("MyConstructor");
 else
 console.log("other");
}

myProxy("Hello");
myProxy(new MyConstructor());
myProxy(3);

And we can even make it dynamic. You should be able to understand most of this, but if you don’t, don’t fret, it is just an example how you can pass the arguments object and analyze it.

Function.checkOverload=function(arrayOfOverloads,args){
 var foundOverload=false;
 for(var i=0;i<arrayOfOverloads.length && !foundOverload;i++){
 if(arrayOfOverloads[i].args.length!=args.length)
 continue;
 var typeMismatch=false;
 for(var j=0;j<arrayOfOverloads[i].args.length && !typeMismatch;j++)
 if(typeof(arrayOfOverloads[i].args[j])=="function"){
 if(arrayOfOverloads[i].args[j]!==null && !(args[j] instanceof arrayOfOverloads[i].args[j]))
 typeMismatch=true;
 }else{
 if(typeof(args[j])!==arrayOfOverloads[i].args[j])
 typeMismatch=true;
 }
 
 if(!typeMismatch){
 return arrayOfOverloads[i].callback;
 }
 }
 throw new Error("No overload found");
};

Using this function is pretty simple. Just pass in a list of callbacks, each with a signature. checkOverload will return the matching function, so you can execute it with apply (or it will throw an error if there is no match).

var overloadedProxy=function(){
 return Function.checkOverload([
 {
 args:["string"],
 callback:function(s){
 console.log("String "+s);
 }
 },{
 args:[Array],
 callback:function(a){
 console.log("Array "+a);
 }
 },{
 args:["string",Array],
 callback:function(s,a){
 console.log("String "+s+" and Array "+a);
 }
 }
 ],arguments).apply(this,arguments);
};

overloadedProxy("Hello");
overloadedProxy(["World"]);
overloadedProxy("Hello",["World"]);
overloadedProxy(["Hello"],["World"]);

Last but not least, we have the old conundrum of private vs public. The usual way is purely by definition. Private attributes are usually prefixed with “_” by most JS developers. While this doesn’t really change their
accessibility, it gives anybody accessing your code a strong indicator that this field is supposed be private. Most IDEs will even go so far as hide these attributes. There’s also a way to make attributes truly hidden, but
this exploits closures and as such is beyond the scope of this document.

So there you have it. JS is again very different from what most developers are used to, but I hope this article could give you a glimpse at the flexibility that Javascript offers versus the traditional, much less flexible, class model.

Node-SQLite-NoDep : SQLite for node.js without NPM

Seeing as how Mozilla is slowly phasing out anything that’s not part of Firefox (sorry Mozilla, I think that’s the wrong call… we don’t need YACC… yet another Chrome clone), including XUL, XULRunner, JSShell and so on I’m slowly trying to replace these technologies on our servers. JSShell in particular has been invaluable in automating simple tasks in a sane language.

The obvious replacement is node.js… but as great as node.js is, it has a few shortcomings. In particular, it relies heavily on NPM to provide even basic functionality and as any admin knows, something that can break will eventually break if its too complex. An admin wants a solution that’s as complex as necessary, but still as simple as possible. So, installing node.js along with npm on a machine is a liability. Luckily node.js itself is portable, but since its library interface is unstable, depending on anything from npm is a big no-no.

One thing I frequently need is a little database. Simple text files work too, but eventually you’ll end up duplicating what any database already does. SQLite is an obvious choice for applications with very few users, for example reftest results. But connecting SQLite to node.js without anything from NPM is a pretty ugly task. Luckily, there’s also a commandline version of SQLite and while it may not be as fast as a binary interface, it can get the job done… with a little help.

Node-SQLite-NoDep does exactly that. It launches sqlite3.exe and sends any data back and forth between your node.js application and the child_process , converting the INSERT statements produced by sqlite3.exe into buffers and providing basic bind parameter support. The documentation is not entirely complete yet, but you can find a quick introduction here along with the jsdoc documentation available here.

Basically, all you have to do is grab SQLite.js, drop into into a folder, add a bin folder, drop node.exe and sqlite3.exe and you’re good to go.

const SQLite = require('./SQLite.js');

var testdb=new SQLite('.\\mytest.sqlite',[
    {name:"testTable",columns:"X int,LABEL varchar[64],data blob"},
],function(){
    testdb.sql("INSERT INTO ?testTable VALUES(#X,$LABEL,&DATA)",
    {
        testTable:"testTable",
        X:123,
        LABEL:"Hello World",
        DATA:new Buffer("DEADBEEF","hex")
    },
    function(data){
        testdb.sql("SELECT * from ?testTable",
        {
            testTable:"testTable"
        },
        function(data){
            console.log(JSON.stringify(data,null,"\t"));
            process.exit();
        });
    });
});

is really all you need to see it in action. Just put it into test.js file and launch it with

bin\node.exe test.js

to see it in action.