protobuf-net v2 ticks along slowly; I’m embarrassed to say that due to a combination of factors progress has been slower than I would have liked – and for purely human reasons (availability etc).
But; I thought I’d better update with the tweaks I’m looking at currently; they have all been ticking along in the back of head for ages, but frankly people kept nagging me to provide them, so who am I to argue?
Caveat
This is all work-in-progress; don’t try to pull the trunk and shove it into production! Soon… I should also stress that all of the thoughts described below are outside the interoperable protobuf spec; it’ll still be a valid protobuf stream, but this only applies to protobuf-net talking to protobuf-net.
So; here’s some common questions I get…
Type meta
The serialization is fine, but I don’t know (and cannot know) all of my types up front. How can I do this?
Well, protobuf is a contract based format; if you don’t know the types, it will struggle – as will any contract based serializer…
Yes, I get that; now: how do I do it?
Now, I’ve held off putting any meta in the stream for various reasons:
- it steps far outside the core protobuf spec
- it flashes the warning signs of BinaryFormatter, my nemesis
But, so many people seem to want this that I think I have to buckle; but on my terms! So in v2, I’m adding the ability to indicate that (on a per-member basis) objects should resolve their type information from the stream. By default, by embedding the assembly-qualified-name, but providing an abstraction layer over that allowing you to provide your own string<===>Type map (and thus avoiding the knots in by stomach caused by too much type dependency).
Full graphs
The serialization is fine, but my data is a graph, not a tree. How can I do this?
Well, protobuf is a tree format; it doesn’t work like that…
Yes, I get that; now: how do I do it?
(by the way, are you spotting a pattern in the questions I get?)
Type meta is something I didn’t want to add, but graph support is something I have really wanted to sneak in; breaking all the rules with type meta seems a reasonable excuse. So in v2, I’m adding the ability (on a per-member basis) to use reference tracking to (de)serialize a complete object graph (except events; don’t get me started on that…). A minor complication here is that for technical reasons it is a nightmare to support this concurrently with inheritance (which, btw, the core protobuf doesn’t support – I feel quite the anarchist here…); but I can support it in conjunction with type meta; so you still get to keep inheritance, but implemented differently.
Repeated strings
It is fairly common to have repeated string data in a large graph. As an offshoot of full-graph support, we also get a mechanism to support string re-use for free; woot! So again, in v2 I’ll be enabling this on a per-member basis.
Footnote
All of this goes so far outside of the protobuf spec that I have little right to even call it protobuf-related any more; maybe I should force the caller to explicitly set an EnableImplementationSpecificOptions flag? So there is an portable core with some opt-in hacks for these random options.
And again; this whole area is under development; I’m working on it, honest!
That’s great! My full-graph of dynamic objects using inheritance and string re-use works fine! Now how do I load it into my c++/java/php/python/whatever client?
Don’t make me hurt you…
