I’ve been playing with distributed caches lately, and of course this is a really good use-case for something like protobuf-net – if there is anywhere you want to minimise your memory costs, network IO costs and CPU costs, it is your cache (it gets kinda busy).
As a pure example of “which system can I get installed and running the quickest”, I looked at memcached, using the windows service from here and the enyim.com client. Not only were they a breeze to get working, but this setup seems pretty popular, and also covers memcached providers, which uses enyim under the hood.
Fortunately, the enyim client has an extension-point that allows you to nominate a serializer through your config file (actually it calls it a transcoder, but I’m not sure it fits the definition). This allows us to swap BinaryFormatter for protobuf-net, which (for suitable types) has huge savings – in my crude tests I was seeing only 20% of the original network traffic, and roughly 75% of the original CPU costs.
Enough bluster…
OK, so in r282 I’ve added just such a transcoder (I still don’t like the name…) – and the good thing it is so effortless to configure it:
Before:
<enyim.com>
<memcached>
<servers>
<add address="127.0.0.1" port="11211" />
After:
<enyim.com>
<memcached transcoder="ProtoBuf.Caching.Enyim.NetTranscoder, protobuf-net.Extensions">
<servers>
<add address="127.0.0.1" port="11211" />
Wasn't that was painless? and some huge benefits (although obviously you’d want to run some integration/regression tests if applying this to an existing system). Oh, and of course you need to include protobuf-net and protobuf-net.Extensions.dll in your project, in addition to the enyim dll.
My only niggle is that because there is no way of knowing the type up-front, I had to include the outermost type information on the wire (hence the “Net” in the transcoder’s name, since it won’t be truly portable). I could get around this by having the caller indicate the type, but the enyim dll doesn’t play that game.
What next?
If you find it good/bad/pointless/buggy please let me know.
If you’ve got another cache that might benefit from some protobuf-net love, please let me know. Velocity, perhaps? Azure?
