Revisited : Fun with field-like events

Previously, I spoke about some problems with field-like events, in particular exacerbated due to two points:

• the synchronization strategy is different between the ECMA and MS specifications (so you can't robustly mimic the compiler's lock behaviour)
• inside the type, += talks directly to the field and does not use the accessor (so you can't ask the compiler nicely if you can please use the thread-safe code that it wrote for you)

leaving you very few options (except manual backing delegate fields and manual synchronization) for writing properly thread-safe event code in some corner-cases (note: most events don't need this level of attention!).

Well interestingly enough, things have changed in 4.0; I must have missed the big announcement, but Chris Burrows has three posts covering this. The last is largely additional info, but the short version of the first two is:

• The generated code is no longer a "lock(this)" (or "lock(type)" for static) - it uses lock-free exchange to do away with this ugly requirement (amusingly, the ECMA spec always maintained that the "how" was an implementation detail; only the MS spec insisted on a specific pattern)
• Inside the type, += and -= now use the add and remove accessors respectively, not the fields

Basically, my previous problem-case would now have worked just fine. I feel vindicated, but also very happy that this has been fixed.

On a related note; if you aren't already aware, note that even "lock(someObj)" gets an overhaul in the 4.0 compiler - see Eric Lippert's post: Locks and Exceptions do not mix

When is an int[] not an int[]?

I’ve spent my entire train journey trying to get to the bottom of this, so I thought I'd blog it for posterity. In my crazed Reflection.Emit frenzy, my unit tests were erroring with PEVerify complaining about illegal ldlen codes:

[offset 0x....] Expected single-dimension zero-based array.

If you're doing meta-programming, tools like PEVerify and Reflector are your closest allies, but this took some head-scratching. I even distilled the code down to two seemingly identical bits of code that read and discard the length of an array variable initialized to null:

The first pane declares “loc 0” and “loc 2” as a local int[] variables; forget about “loc 1” – it is unrelated. The second pane initializes each array variable as a null reference, obtains the length (which is a “native int” which I immediately convert to Int32), and then discards the value.

So why the error? And why one error and not two? PEVerify is, after all, a chatty beast… Either I’ve gone crazy in my code, or somebody is lying to me! Actually, both it turns out.

Pop quiz: what is the difference between these two Type instances representing a 1-dimension array of int:

Type explicitRank = typeof(int).MakeArrayType(1),
implicitRank = typeof(int).MakeArrayType();

The second is our friend, int[]. The first is something different, though; it is a 1-dimensional array of int sure enough, but it isn’t explicitly zero-based! (correction due: see comments) D’oh! It goes by the moniker int[*].

Simply; you can’t use ldlen on an int[*] – only an int[]. What I don’t yet understand is why the upstream code (when it assigned the array “for real”) didn’t complain about the very attempt to assign an int[] value (from a standard “get” accessor) to an int[*] local variable. Presumably the PEVerify authors didn’t think anyone would be stupid enough to try ;-p

The moral here; sometimes it pays to be less explicit (and I don’t just mean the language I used when I found the problem). I’ve also left feedback with Red Gate to tweak how it displays, but to be honest the number of people this cosmetic glitch will affect is minimal.

Binary data and strings

You have no idea how often I get an e-mail that talks about binary data (for me, usually protobuf data) and string encoding (usually via utf-8).

Here’s the thing folks: most binary data cannot be translated to a string with a utf-* translation. It just isn’t possible. If you are talking about an arbitrary byte sequence that you need to store or transmit as a string, then your best bet is base-64.

Fortunately, in .NET Convert.ToBase64String and Convert.FromBase64String deal with all the details, making it trivial to use.

I keep talking myself into (and then back out-of) adding string-based overloads (etc) to protobuf-net; maybe I’ll add them just to stop the internal debate…

The last will be first and the first will be last

I’ve probably mentioned that I’m currently re-writing some code using IL emit. Very interesting work, and I’ve learned a lot about the CLI in the process.

The work I’m doing at the moment works using the decorator pattern, giving each node the chance to manipulate the value, which is usually expected to be at the top of the stack.

The problem is… imagine we want to call:

someWriter.WriteInt32(theValueOnTheStack);

To call this, we need “someWriter” before the value on the stack (i.e. push “someWriter”, push the value, callvirt). There are two options at this point:

• Change the calling code so that we already pushed “someWriter” earlier in the code (very problematic – it makes the stack very complex, especially considering branching etc)
• Declare a local variable, store the value on the stack into the variable, push “someWriter”, push the value from the local variable, callvirt

Only the second is an attractive option, but in itself this causes me problems – not least in some code where this would lead to a lot of locals being declared (even though I have some fancy local pooling to re-use like-typed variables as far as possible; the problem is when lots of different types are involved).

So how about we change the API? As it happens, “someWriter” is easily available (as arg1 or arg2). What about if we made it a static method, and passed the instance in last?

SomeWriterType.WriteInt32(theValueOnTheStack, someWriter);

OK – this is a pretty freaky calling convention, but it fits our typical usage perfectly. Given that the value is already on the stack, now I just need to push “someWriter”, call (not callvirt). Obviously the ex-instance-method code needs changing – essentially the different between:

Note that it is important that I am not using “virtual” methods in all this, as that would demand the original usage.

But does it cause a performance problem? (note: yes I know this is all micro-optimisation, but I have a genuine reason for trying to minimise the stack size, which is the real purpose behind this).

So I fired up a typical test-rig that calls the methods an insane number of times:

The results aren’t necessarily surprising – simply confirming that this doesn’t have any negative impact on performance. It actually makes things slightly faster, but all I really needed to know is whether it would end up being 100 times slower or not. It isn’t. Not having the locals is my main aim.

So it looks like I’m going to end up with backwards methods. Which is fine – only my code will see it (it isn’t part of the user-facing API, although it does have to be public to allow for calling from other assemblies).

In this case, I can live with an odd calling convention ;-p