Thursday, 19 May 2011

protobuf-net v2, beta

It has been a long time in the coming, I know. I make no excuses, but various other things have meant that it didn’t get much input for a while…

But! I’m happy to say that I’ve just pushed a beta download up onto the project site

So… this v2… what is it?

Basically, it is still the core protobuf stream, but against all the safe advice of my employer I rewrote the entire core. For many reasons:

  • the original design grew organically, and ideas that seemed good turned to make for some problematic code later
  • the vast overuse of generics (and in particular, generics at runtime via reflection) was really hurting some platforms
  • there were many things the design could not readily support – structs, runtime models, pre-built serialization dlls, usage on mobile devices, etc
  • the code path at runtime was just not as minimal as I would like
  • and a myriad of other things

So… I rewrote it. Fortunately I had a barrage of integration tests, which grew yet further during this exercise. The way I figured, I had 2 choices here:

  • go with a richer code-generator to use as a build task (not runtime)
  • go crazy with meta-programming

After some thought, I opted for the latter. In particular, this would (in my reasoning) give me lots of flexibility for keeping things runtime based (where you are able to, at least), and would be a good opportunity to have some fun and learn IL emit to dangerous levels (which, incidentally, turns out to be very valuable – hence mine and Sam’s work on dapper-dot-net).

So what is new?

I don’t claim this list is exhaustive:

  • the ability to define models at runtime (also useful for working with types you don’t control) (example)
  • the ability to have separate models for the same types
  • pre-generation to dll
  • runtime performance improvements
  • support for structs
  • support for immutable objects via surrogates (example)
  • the ability to run on mobile platforms (unity, wp7 – and perhaps moot for a while: MonoDroid, MonoTouch – but presumably Xamarin)
  • the ability to serialize object graphs preserving references (example)
  • the ability to work with types not known in advance (same)
  • probably about 40 other things that slip my mind

I’ll try to go into each of these in detail when I can

And what isn’t there yet?

This is beta; at some point I had to make a milestone cut – but some things aren’t quite complete yet; these are not usually needed (hint: I’m happy to use protobuf-net v2 on my employer’s code-base, so I have confidence in it, and confidence that the core paths have been smoke-tested)

But things that definitely aren’t there in the beta

  • WCF hooks
  • VS tooling (it is just the core runtime dlls)
  • round-trip safe / “extension” fields
  • extraction of .proto (schema) files from a model
  • my build / deploy script needs an overhaul
  • tons of documentation / examples
  • tooling to make the “mobile” story easier (but; it should work – it is being used in a few, for example)

But; have at it

If you want to play with it, go ahead. I only ask that if something behaves unexpectedly, drop me a line before declaring loudly “it sux, dude!”. It might just need some guidance, or maybe even a code fix (I’m far from perfect).

Likewise, any feature suggestions, etc; let me know.