Optimize Player.

[bigeasy]

Player has three potential bottlenecks; checksum generation, deserialization and splicing. Checksum generation and testing is something that we do that others might not do, so we are going to pay a price for those checksums in the shootouts. Checksums require slices. Maybe it is faster if you use a JavaScript checksum modified to work with buffers and ranges. If it comes down to all benchmarks all the time, then we're going to want to turn them off in Locket, or else wait until they save someone's bacon.

Deserialization might be improved by using a binary file format. Maybe it is enough to can for the end of the header array explicitly. Maybe it is important to go binary. You could create a translator that would read a binary file format and emit your pretty text format. Binary means you could use a Packet compiler and have a best-foot forward reader. It would be easier to checksum without having to read into buffers.

You might want to consider making record serialization pluggable.

I'm concerned that the Magazine lookup on each splice is costing too much. The Cartridge could be stored in the page, so adjusting heft could be done without the lookup.

Reading the buffer from file seems to happen quickly enough and there is little that I can do to make Node.js load a file faster. All performance must be squeezed from the synchronous buffer reading loop.

[bigeasy]

A binary format means no scanning and then reading, just reading.

[bigeasy]

Two things that will make Locket slow compared to LevelDB, these are two things that Locket does that LevelDB does not. First, the checksums. I'm not sure that LevelDB does such a thing. Second it is the UTF-8 file format, which is easy to read. Should there be some corruption, it would be easy to open the leaves in a file. A binary file format is going to be even worse in Locket, since I must also parse the separation of key and value, which means a key length needs to be stored. This would all probably go much faster if it where all binary.

To stand up in a shootout, we're going to need to turn off checksums and use a binary parser. These can be switches, a switch to turn the checksums on, and a switch to use the UTF-8 format, so that, by default we have no checksum and use a binary format.

[bigeasy]

No improvement from moving slice into Page and caching the cartridge in the Page. That leaves checksums and a binary file format.