Tuesday, 2 February 2010

New spike - looking hopeful

After a few false starts, I've finally found some time to spike the much-needed re-work for protobuf-net; this pretty much means "re-write it from scratch, learning from design mistakes and copying the occasional block of code".

Key changes / goals:
  • Generics: for the most part, gone from the guts of the implementation; it now uses a non-generic "decorator" approach - you wouldn't believe how much pain they have caused me (here)
  • This should hopefully fix compact-framework (here, here), and might (no promises at this stage) even allow it to work on .NET 1.1 and micro-framework
  • Build around a runtime model rather than attribute reflection (but with attribute reflection to be added as a model provider)
  • Much easier to unit-test individual decorators
  • Double implementation; vanilla decorator for "light" frameworks, and compiled implementation for the full-fat versions
  • I lack the hardware to test, but I'm hopeful that the vanilla decorators will work on iPhone - I don't suppose you have an iPhone and iMac to spare?
  • Potential to emit the compiled version to disk (think: sgen)
  • Added buffer-pooling (makes it possible to use a larger buffer with minimal allocation costs)
  • Re-written the string handling to use pointer (unsafe) code where appropriate, and better incremental encoder handling
There is still masses to do - but I'm feeling pretty happy with the initial results... I've got the core engine in place, and enough decorators to write some basic messages. Timings below, with meanings:
  • New (decorator) - vanilla decorator implementation - no optimisations (yet; although I'm sure we can improve this once I can tell what works/where)
  • New (compiled) - compile/flatten the decorators into a single operation
  • New (delegate) - same, but to a delegate rather than class (just for fun)
  • Old: the existing protobuf-net "generic" Serializer

   New (decorator): 3399ms
New (compiled): 788ms
New (delegate): 619ms
Old: 758ms
New (decorator): 2569ms
New (compiled): 642ms
New (delegate): 604ms
Old: 754ms
New (decorator): 2549ms
New (compiled): 631ms
New (delegate): 623ms
Old: 740ms
New (decorator): 2589ms
New (compiled): 640ms
New (delegate): 608ms
Old: 750ms
New (decorator): 2587ms
New (compiled): 638ms
New (delegate): 614ms
Old: 750ms
New (decorator): 2433ms
New (compiled): 519ms
New (delegate): 504ms
Old: 664ms
New (decorator): 2318ms
New (compiled): 512ms
New (delegate): 506ms
Old: 596ms
Oh, and I need to fix all the other things, too... sigh.