protobuf-net v2 on the iPhone

It might not look very impressive to you, but this is pure joy to me. It is the first evidence I’ve seen that the revised rewritten protobuf-net can work in MonoTouch for use on the iPhone – and presumably MonoDroid in the future.

A big thanks here to Christian Weyer and Andrew Rimmer who both offered some help here (since I don’t actually own a mac, an iPhone or MonoTouch!). The grab is from Andrew (he assures me it works fine on the actual device too).

This piece of work has been a slog, and I kept putting it off (because frankly, re-writing the entire stack is a huge job). The goals have not been trivial:

• Continue to support all the existing runtime-only usage…
• ..while also making it support fully static / AOT compatible
• Allow it to write serialization dlls in advance (like sgen)
• Fix the stability issues on Compact Framework (caused by overuse of generics)
• Remove some performance bottlenecks on Compact Framework
• Support POCO (or more accurately, work without requiring special [attributes]) so you can use it with types outside your influence, or for inheritance that isn’t known at compile-time
• Make it work on iPhone etc
• Make it even faster
• Make it more extensible (for example, what if I want to write a proxy that writes a DataTable to a protobuf stream?)
• Make it support mutable structs (yes I know that they are usually evil, but people simply keep asking for this, in particular for XNA)
• Remove a heap of code-debt caused by an evolving project that grew without a 100% clear direction
• Throw in some other feature-requests and bug-fixes for good measure
• And as a side-benefit, it even compiles and runs on .NET 1.1 or via mcs on Mono (with minimal #if usage)

I should stress that it is not yet complete – but the core of the engine is there and working, demonstrating most of the key points above (albeit in a very raw state). There’s still a way to go, but it is getting there.

Everything “is” an object?

This nugget of the .NET unified type system. And the cause of oh so much misunderstanding.

My way of looking at it is that it isn’t helpful to think of value-types (structs in C# parlance) “as objects”. You can treat them as though they are, but you need to fundamentally change them (box them) in order to do it. Which has lots of uses for sure; but just make sure you understand what you are doing and why. If a value-type truly was an object you wouldn’t have to do this.

It is a bit like saying all people are meat; technically true, but if you start manipulating me like you would beef, don’t expect me to be quite the same afterwards.

Instead, I try to emphasize that everything can be treated as an object.

Oh, and mutable structs are evil, but you already knew that.

New spike - looking hopeful

After a few false starts, I've finally found some time to spike the much-needed re-work for protobuf-net; this pretty much means "re-write it from scratch, learning from design mistakes and copying the occasional block of code".

Key changes / goals:

• Generics: for the most part, gone from the guts of the implementation; it now uses a non-generic "decorator" approach - you wouldn't believe how much pain they have caused me (here)
• This should hopefully fix compact-framework (here, here), and might (no promises at this stage) even allow it to work on .NET 1.1 and micro-framework
• Build around a runtime model rather than attribute reflection (but with attribute reflection to be added as a model provider)
• Much easier to unit-test individual decorators
• Double implementation; vanilla decorator for "light" frameworks, and compiled implementation for the full-fat versions
• I lack the hardware to test, but I'm hopeful that the vanilla decorators will work on iPhone - I don't suppose you have an iPhone and iMac to spare?
• Potential to emit the compiled version to disk (think: sgen)
• Added buffer-pooling (makes it possible to use a larger buffer with minimal allocation costs)
• Re-written the string handling to use pointer (unsafe) code where appropriate, and better incremental encoder handling

There is still masses to do - but I'm feeling pretty happy with the initial results... I've got the core engine in place, and enough decorators to write some basic messages. Timings below, with meanings:

• New (decorator) - vanilla decorator implementation - no optimisations (yet; although I'm sure we can improve this once I can tell what works/where)
• New (compiled) - compile/flatten the decorators into a single operation
• New (delegate) - same, but to a delegate rather than class (just for fun)
• Old: the existing protobuf-net "generic" Serializer

   New (decorator): 3399ms
  New (compiled): 788ms
  New (delegate): 619ms
  Old: 758ms
  New (decorator): 2569ms
  New (compiled): 642ms
  New (delegate): 604ms
  Old: 754ms
  New (decorator): 2549ms
  New (compiled): 631ms
  New (delegate): 623ms
  Old: 740ms
  New (decorator): 2589ms
  New (compiled): 640ms
  New (delegate): 608ms
  Old: 750ms
  New (decorator): 2587ms
  New (compiled): 638ms
  New (delegate): 614ms
  Old: 750ms
  New (decorator): 2433ms
  New (compiled): 519ms
  New (delegate): 504ms
  Old: 664ms
  New (decorator): 2318ms
  New (compiled): 512ms
  New (delegate): 506ms
  Old: 596ms

Oh, and I need to fix all the other things, too... sigh.

Distributed caching with protobuf-net

I’ve been playing with distributed caches lately, and of course this is a really good use-case for something like protobuf-net – if there is anywhere you want to minimise your memory costs, network IO costs and CPU costs, it is your cache (it gets kinda busy).

As a pure example of “which system can I get installed and running the quickest”, I looked at memcached, using the windows service from here and the enyim.com client. Not only were they a breeze to get working, but this setup seems pretty popular, and also covers memcached providers, which uses enyim under the hood.

Fortunately, the enyim client has an extension-point that allows you to nominate a serializer through your config file (actually it calls it a transcoder, but I’m not sure it fits the definition). This allows us to swap BinaryFormatter for protobuf-net, which (for suitable types) has huge savings – in my crude tests I was seeing only 20% of the original network traffic, and roughly 75% of the original CPU costs.

Enough bluster…

OK, so in r282 I’ve added just such a transcoder (I still don’t like the name…) – and the good thing it is so effortless to configure it:

Before:

<enyim.com>
   <memcached>
       <servers>
           <add address="127.0.0.1" port="11211" />

After:

<enyim.com>
   <memcached transcoder="ProtoBuf.Caching.Enyim.NetTranscoder, protobuf-net.Extensions">
       <servers>
           <add address="127.0.0.1" port="11211" />

Wasn't that was painless? and some huge benefits (although obviously you’d want to run some integration/regression tests if applying this to an existing system). Oh, and of course you need to include protobuf-net and protobuf-net.Extensions.dll in your project, in addition to the enyim dll.

My only niggle is that because there is no way of knowing the type up-front, I had to include the outermost type information on the wire (hence the “Net” in the transcoder’s name, since it won’t be truly portable). I could get around this by having the caller indicate the type, but the enyim dll doesn’t play that game.

What next?

If you find it good/bad/pointless/buggy please let me know.

If you’ve got another cache that might benefit from some protobuf-net love, please let me know. Velocity, perhaps? Azure?

Reflections on Optimisation – Part 3, Fields and Properties

Short version:

Unless you are doing something interesting in your accessors, then for “full” .NET on classes, it makes no difference. Use automatically-implemented properties, and “commit”.

Much ado about nothing:

Previously, I discussed ways of modelling a general-purpose property accessor, and looked at existing APIs. So what next? I’m rather bored of conversations like this on forums:

OP: My flooble-gunger application uses public fields to hold the flipple-count, but I’m getting problem…

Me: back up there…. why does it use public fields?

OP: For optimisation, obviously!

But is there anything “obvious” here? I don’t think this is news to most people, but public fields are not generally a good idea – it breaks encapsulation, is (despite belief otherwise) a breaking change (and sometimes build-breaking) to swap for properties, doesn’t work with data-binding, can’t participate in polymorphism or interface impementation, etc.

With regards to any performance difference, you would expect that in most “regular” code, member-access does not represent the overhead. We would hope that the only times when member-access might be truly interesting is (like for me) if you are writing serialization / materialization code. Still, let’s plough on with this scenario, and check things out.

In particular, for simple properties (for example, C# 3.0 “automatically implemented properties”) our expectation is that they will commonly be “inlined” by the JIT. The big caveat here (where I might forgive you) is Compact Framework (for example XNA on XBox 360), which has a much weaker JIT.

So as before; let’s test it. Another very simple test rig, I want to compare (each in a loop):

• Direct property access on an object
• Direct field access on an object
• A wrapper (as before) that talks to a public property
• A wrapper (as before) that talks to a private property
• (automatically implemented properties vs manual properties)
• A wrapper (as before) that talks to a public field
• A wrapper (as before) that talks to a private field

To achieve the private tests, I’ve created the wrapper implementation as a nested type of the object I’m testing. Oddly enough, there were some oddities that popped up, but I’ll discuss those next time; this post has been long enough, so here’s the very uninteresting result (sorry for the anti-climax):

Implementation	Get performance	Set performance
Prop, no wrapper Field, no wrapper Public prop wrapper Private prop wrapper Auto prop wrapper Public field wrapper Private field wrapper

(numbers scaled, based on 650M iterations, lower is better)

Both direct and via abstraction, the cost is identical for fields and properties. Quite simply – don’t stress over it.

Reflections on Optimisation – Part 2, Existing Frameworks

Yesterday I introduced some topics on reflection performance, and investigated briefly the options to model abstraction, concluding that an abstract base-class / concrete subclass was a reasonable approach.

At this point, you could ask “so?” – the point is that I need a pretty optimised way of getting / setting data. So how about existing frameworks? Raw reflection? (PropertyInfo) ComponentModel? (PropertyDescriptor) HyperDescriptor? Let’s compare these options. I’ve setup a deliberately basic test rig, that simply tests each of these in a tight loop (again, deliberately avoiding boxing etc). As before, I’ll post the full code at the end of the series, but here’s the crude comparison:

Implementation	Get performance	Set performance
No wrapper (baseline) Base class HyperDescriptor PropertyDescriptor* PropertyInfo*

(numbers scaled, based on 750M iterations, except “*”; lower is better)

You might think “hey, PropertyInfo / PropertyDescriptor” aren’t too bad! Except for the *, which means that in order to get them into the same scale I had to introduce the line:

            count /= 100;

Where "count" is the number of iterations to perform. And even with this they are at the bottom of the table (by quite a margin). Yup – they really are over 100 times slower, which is why we need to optimise in the first place!

You might also conclude “HyperDescriptor doesn’t do too badly – just use that and move on”. Which would be valid in most cases (I’m pretty happy with that code, especially since that was my first foray into IL), except:

• It wouldn’t let me investigate field vs property, polymorphism, and the other common “is this faster” questions I mentioned in part 1
• It relies on constructs that don’t exist in CF / Silverlight, and which it would not be useful to duplicate (yet I need this for protobuf-net)
• Sometimes the extra crank of the optimisation handle matters

Summary

In this discussion, I hope I’ve demonstrated that there really is a problem just using PropertyInfo etc, that we can overcome (evidence: HyperDescriptor), but we would hope to do so in a simplified, minimal way that can be modelled (perhaps with slightly different implementations) on all the common frameworks.

Reflections on Optimisation – Part 1, Abstraction

Introduction

I want to take a few blog posts to discuss some ramblings on performance, reflection, and library code. The common “knowledge” is that reflection is slow and to be avoided, but this is missing the caveat “when used inappropriately” – indeed, as part of library / framework code it can be incredibly useful and very fast. Just as importantly, suitable use of reflection can also avoid the risk of introducing bugs by having to write repetitive code manually.

Between my musings on the Expression API, work on HyperDescriptor, and many questions on sites like stackoverflow, I’ve looks at this issue often. More, I need to do some work to address the “generics” issue in protobuf-net – so I thought I’d try to combine the this with some hopefully useful coverage of the issues.

The setup

What I am really after is the fastest way of getting values into and out-of objects, optimised as far as is practical. Essentially:

    var oldX = obj.X; // get
obj.X = newX; // set

but for use in library code where “X” (and the type of X) isn’t known at compile-time, such as materialization, data-import/export, serialization, etc. Thanks to my learning points from protobuf-net, I’m deliberately not getting into any “generics” answers from this. Been there; still trying to dig my way out.

I plan to look at a range of issues in this area, including (not necessarily in this order):

• types of abstraction
• fields vs properties vs polymorphism
• mechanisms for implementing the actual get/set code
• boxing, casting and Nullable-of-T
• lookup performance
• caching of dynamic implementations
• comparison to MemberInfo / PropertyDescriptor / CreateDelegate

Note that this is an area that can depend on a lot of factors; for my tests I’ll be focusing on Intel x86 performance in .NET 3.5 SP1. I’ll try to update when 4.0 is RTM, but I’d rather not get too distracted by the 4.0 CTPs for now. Even with this fairly tight scope, I expect this to take too many words… sorry for that.

First problem – abstraction

OK; so essentially we expect to end up with some API similar to reflection, allowing us to swap into existing code without re-writing it:

    var prop = SomeAPI["X"]; // perhaps passing in
                         // obj or a Type
object oldX = prop.GetValue(obj); // get
prop.SetValue(obj, newX); // set

This should look very similar to anyone who has used reflection or TypeDescriptor. And I plan to compare/contrast to the default implementations. So: is there any noticeable difference in how we implement “prop”? This might give us a better solution, and it also impacts how I write the tests for the other issues ;-p I can think of 3 options off the top of my head:

• An interface with concrete implementation per type/property
• An abstract base class with concrete implementations per type/property
• A sealed implementation with delegates (of fixed delegate types, one matching the get, one the set) passed into the constructor

There is no point introducing any dynamic code at this point, so I’ll test each these implemented manually (deliberately avoiding boxing), compared to a direct wrapper (no abstraction), and direct access (no wrapper, as per the original introduction):

I’ll post the full code later (and add link here), but this gives the initial results:

Implementation	Get performance	Set performance
No wrapper (baseline) No abstraction Base class Interface Delegates

(numbers scaled, based on 750M iterations; lower is better)

The conclusion of this is that, for the purposes of representing a basic property wrapper mechanism a base class is just as good as a wrapper with no abstraction – so we should assume a base-class implementation. An interface is a little slower – not enough to be hugely significant, but a property wrapper is a pretty specific thing, and I can’t see the benefit in making it more abstract than a base class.

For the purpose of adding context, the (very basic) code I’ve used to test this base-class approach is simply:

Summary

It might not seem like we’ve come far yet; but I’ve described the problem, and narrowed the discussion (for the purpose of micro-optimisation) to implementations using a base-class approach. Which is a little annoying on one level, as the delegate approach (via DynamicMethod) is in many ways more convenient.