Hectic Weeks, Charisma; and Trolling

(not much code here - if you're after a "fix", come back soon ;-p)

A busy week...

First, I somehow got elected as a community moderator on stackoverflow. In reality, this mainly means that I get more visibility of where people flag spam / etc. You can probably guess at some of the obvious trolls and other personalities that appear (hopefully briefly before being deleted). Fortunately, it isn't keeping me too busy! The main down-side is that I can no longer cast a regular "close this" vote; but that's OK.

I'm not sure whether I should feel guilty when I click the button to delete/destroy an account... even if there is clearly no doubt to "give the benefit of"; hmm - I must get over that ;-p

(this only applies to clear and indisputable spam/troll/puppet accounts, usually with no e-mail, etc)

Then on the weekend, I was at DDD South West; I'm afraid I bombed out of the balloon debate; I must concentrate less on specific details, and more on delivery/charisma (persuasion trumping fact in a debate). It was won for VB by Microsoft's Eric Nelson, who stepped in after the scheduled VB warrior had to drop out. I'll blame my defeat on his charms ;-p A fun diversion in a busy day.

I also presented a session on the Expression API in .NET 3.5 (and 4.0) - which was daunting (not least because my laptop was throwing a BSOD hourly until Friday). I don't think I did as good a job on this presentation as I have in the past, but it seemed to be received OK. The eval forms show that it is a bit of a divisive one, though - presumably because it is quite complex; either you "grok" it or you don't. But from some after-presentation conversations I know that it clicked with the people who come to talk to me , which is cool. I must make sure I describe it as a "deep dive" (or similar) if I do this one again...

Now I just need a few weeks quiet to catch up ;-p

Multi Monitor Bliss

Sometimes the simplest things give the greatest joy... in VS2010 you can now have multiple code windows (great for multiple monitors):

Unity in VS2010

I'm currently playing with the public beta of Visual Studio 2010; as a beta, the obvious choice is to run it on a VM (although I also have it "on the metal" on a spare machine). I also mentioned previously about using Unity mode in VMware.

At the current time, VS2010 is looking good on a VM, but there are a few glitches to beware (thinking of VMware in particular):

• It doesn't like the DirectX acceleration option (presumably some glitch between WPF and the emulator) - this leaves it a little shaky in "console" mode, but plain freaky in "unity" mode
• The way the window system works means that "unity" sees everything as separate windows, and attaches badges to both menus and popups (intellisense etc):

Fortunately, you can fix both issues with some minor tweaks to the vmx file:

unity.showBadges = "FALSE"
mks.enable3d = "FALSE"

With those tweaks in place, VS2010 is a happier place to be - and we can get on with the important work of playing with "dynamic", variance, and all the other .NET 4.0/C# 4.0 features ;-p

Generics, and When Not To

Generics are undoubtedly of huge benefit for writing maintainable, static typed code with the minimum of fuss... but can it be overused? I firmly believe that generics are great for regular line-of-business code, with typed collections/lists and everything else that goes with them. But what about utility code, in particular when you're doing something complex with reflection... maybe give it a miss.

A cautionary tale: if you find yourself doing something with generics that looks complex... stop; you might be using the wrong approach. Of course, you might be right after all - but at least do a sanity check.

Consider protobuf-net; this is a member serializer (akin to XmlSerializer, DataContractSerializer, et al - but writing binary). It is quite a complex setup, but essentially it is using a strategy pattern (via reflection) to serialize each of the different properties. To avoid lots of boxing/casting, and to allow me to use the typed form of delegate invoke (much faster than dynamic invoke), it started out with a highly generic model - i.e. strategy classes that use generics to describe the input, output and anything else required.

But the problem with generics in this context is that they beget more generics... you start with something simple like "Foo<T>", and before you know it you've got types with multiple generic type arguments and constraints, inheriting from other generic types, all the way down (just like the turtles). Understanding the type signature alone is headache inducing - never mind trying to use reflection (MakeGenericType etc).

More importantly, however - the generic approach doesn't work very well if you want to use a decorator-based strategy; for example, to serialize the property "public List<Foo> {get;}", we might have a decorator that does property access, a decorator that handles list iteration, and a decorator that serializes Foo (and whatever that involves). You simply can't write that just using generics, as you can't tell in advance how deep the hole goes, and how many type arguments you'll need (let alone what they might be).

For utility code (such as a serialization engine) there are some other considerations:

• Some frameworks (Compact Framework) have limitations in the number of generic types you can create at runtime
• Some frameworks (Micro Framework and .NET 1.1) simply don't have generics at all

So faced with ever-increasing code complexity (always a bad sign), and it not working reliably (or at all) on some frameworks, I've decided to try to drop generics completely from the core implementation, and just have an "object" based decorator-chain. It then doesn't matter how complex the chain gets - I know I can still pass an object downstream (even if I've changed the type), and it won't offend CF/MF.

So what about performance? I certainly don't want to cripple the code...

• Passing objects between the different decorators in a chain might look ungainly, but actually the odd bit of boxing and casting isn't necessarily a major problem (especially if it means the code works)
• Where available (i.e. framework dependent), we can still use Delegate.CreateDelegate and the generic wrapper trick, but now as an implementation detail of an otherwise non-generic decorator (i.e. the public API would still be non-generic)
• Where available (i.e. framework dependent), we have the option of making our decorators support Reflection.Emit - so rather than having methods that do the operation, we can have methods that write how to do it. There are several performance advantages to this:
  • no boxing/unboxing/casting
  • no stack hops
  • no delegate invoke (dynamic or typed)

The refactor is still incomplete, but is slowly taking shape (although it is taking longer than I hoped). It helps that I have a good set of unit tests so that I know just how far I have to go... but I've done enough to satisfy me that a: it should work much more reliably on CF, b: it should still fly on the regular framework, and c: if I really want I could probably make it work on .NET 1.1 and MF (but I'm not promising to do this).

Live your life in Unity

Virtual machines; if you are doing development, and you aren't regularly using virtual machines, then you are probably missing a trick. Or several.

Interestingly, one of the new features announced in Windows 7 is the ability to run XP applications in a virtual machine, but per-application, and on the same desktop (rather than in a VM console). This is Windows Virtual PC - but it isn't as radical as it sounds, after all, VMware Fusion has been doing this on the Mac for years.

Of course... Windows Virtual PC is only useful if you have the right version of Windows 7 (which isn't yet released). And you might not want an XP guest; developers are typically running virtual server platforms, for example.

As it happens, I'm a VMware user (primarily of the free VMware Player, as my needs are simple). This lets you run guest OS platforms in a console on the desktop - for example, here we have an XP host (showing a copy of C# Express in the host) and VMware Player. The guest OS is Windows 7 (RC), also running C# Express:

While this works, it doesn't make good use of screen real-estate:

• the guest applications are limited to the guest console (see how the guest application is clipped) - you can't move them outside the frame, etc
• you have two desktops / start menus / task bars / etc to contend with

And of course, if you have a small screen (laptops etc) this only gets worse.

Now - maybe I missed the big announcement, but towards the end of 2008, "Unity" support was added - even into the free player. A range of common guest platforms are supported as long as you have VMware Tools installed on the guest machine.

So if Windows VirtualPC might let us run XP apps on a Windows 7 desktop, can we run Windows 7 apps on an XP desktop? You can with Unity. Switching modes is simple:

This has a very dramatic effect; our console window minimises itself, and the guest applications are transferred to the host desktop. Here we have a selection of host / guest applications to show how they can be mixed:

Two of those are running on the guest- and two on the host; but all function as you would expect. We can now move all the guest applications where ever we want, resize them, stack them, drag them onto different monitors, etc.

You get some visual clues: the VMware logo and some border shading (these options can be configured in the Workstation product); you also get access to the guest system's start menu (or equivalent) when you go near the host menu - letting you start new guest applications:

Overall - a very satisfying way of developing on virtual platforms. It feels very natural and intuitive. Now we just need Remote Desktop (and competition) to catch up and have remote applications rather than the whole desktop.

EDIT: this exists, via RemoteApp

Note: the screen-shots above are using VMware Player; for serious development the Workstation product may be worth considering - as (even before Unity) this has a wide range of more advanced features to let you develop efficiently (and fix things quickly when you break the server), including Unity of course.

I don't know about you, but I'm smitten.

Handy editor tweaks with SlickEdit Gadgets

I'm fairly conservative when it comes to additional IDE add-ins (Visual Studio has enough going on without throwing in the kitchen sink), but I came across a very interesting little library the other day; SlickEdit Gadgets - and just have to share. Best done with an image:

What we can see here is:

• an "indentation guide" (the vertical dashed line) - very handy for "at a glance" indentation checks - which (if your code follows any kind of convention) means that you can also get an immediate clue to the nesting of the current line. Much easier than trying to brace highlighting, or the confusing mess of lines that some tools insert - just move to the line you are interested in, and "presto".
• a "line ruler" (the horizontal highlight) - again, tied to the current line, but giving a very visual cue where we are. Much easier than trying to play "hunt the caret" (which is not a euphemism). In particular, I see this as being useful for anyone doing pair/team programming, or giving presentations - no more mouse-waggling / laser-pointer-tag trying to indicate a specific line.
• an "editor graphic" (the coffee cup) - an image of your choice right there in the editor window... maybe not hugely desirable on a day-to-day basis, but maybe a sweet gimmick when giving external presentations - perhaps put the user-group's / code-camp's logo up there as a reminder. Or not ;-p

As you might expect, there are suitable options for styling (colour, animation, etc). There are a few other tools too (not shown) - including an in-IDE file explorer, automatic "copy", a tool for sniffing the IDE commands, etc.

But most importantly, these tools are freely available.

I also intend to take a look at their actual Visual Studio products at some point - the Editing Toolbox / Versioning Toolbox; in particular, the SVN integration looks like it might actually be usable (unlike a few I've tried - hence I currently use the excellent TortoiseSVN in explorer). The prices don't look too terrible (but it is a little odd that there isn't a bundle for the pair).

Re-implementing Expression for Compact Framework

Background

In this previous post I gave an indication of how the RPC stack of protobuf-net is implemented - in particular (in .NET 3.5) with Expression - allowing:

  ProtoClient<IMyService> client = ...
  double amount = client.Invoke(svc => svc.DoSomeMaths(
    12, someObj.Qty));

where DoSomeMaths is a method defined on the IMyService interface. This works well, and avoids the need to do any kind of code generation, yet still have full static (compile time) checking that we're using methods that are defined on our service.

So what is the problem?

As always, there is a fly in the ointment; it transpires that the Expression API doesn't exist on Compact Framework 3.5. Perhaps Microsoft thought that it served little purpose without a matching Reflection.Emit implementation (allowing you to compile expressions to delegates) - who knows. But it is vexing that it works everywhere (including Silverlight 2.0) except here. So what can we do?

Well, pretty-much all of C# 3.0 is actually compiler magic hitting known types in the base class. For example, you can use LINQ (with delegates) with C# 3.0 targetting .NET 2.0 by declaring ExtensionAttribute and adding a pile of extension methods on IEnumerable etc (or more simply: download LINQBridge). So can we do the same here? I've previously always been dismissive of this, but it turns out that yes, we can.

What, all of Expression?

First - a caveat; I don't need all of Expression; for example, I don't need any compilation features, I don't need to be able to add things etc - all I need is:

• a lambda that accepts parameters
  • that can invoke methods
  • that can use constant argument values
  • that can use properties and fields as values
  • that can use captured values (which are implemented as fields)

Anything more complex, and the caller can simply pre-calculate their argument value into a variable, and pass that in via a capture - i.e. this won't work:

  double amount = client.Invoke(svc => svc.DoSomeMaths(
    12, someObj.Qty * 2));

but this will:

  int qty = someObj.Qty * 2;
  double amount = client.Invoke(svc => svc.DoSomeMaths(
    12, qty));

What is involved?

Actually, not a vast amount - but we are hampered a bit; the C# spec doesn't actually indicate what Expression features it uses, nor when. Very unusual for C#, which is usually very detailed in this sort of thing... but actually, we can get an appreciation of what we need from looking at what the compiler does behind the scenes (as discussed previously). Given the requirements above, it turns out that we need the static methods:

• Expression.Constant
• Expression.Parameter
• Expression.Field
• Expression.Property
• Expression.MethodCall
• Expression.Lambda<T>

and a few classes to hold the state for each of these, so that we can inspect it afterwards. Not a huge task at all.

plz send teh codez

OK, enough already... the code for this (and it isn't very much) is now committed into protobuf-net (but only applies to the CF35 build) - and is available here - at just over 250 lines (including comments), this seems a fairly pragmatic way of unifying service access over the different frameworks. Enjoy.