Monday, 16 November 2009

SelectMany; combining IDisposable and LINQ

Something that often trips me up in LINQ is that you frequently have to break out of the query syntax simply to add a few “using” blocks. It is important to keep the “using” to ensure that resources are released ASAP, but the regular LINQ query syntax doesn’t let you do this. For example (and I realise there are better ways of doing this – it is for illustration only):

image

Trying to write this (or something like this) in LINQ query syntax (or fluent syntax) just gets ugly.

If only there were a way to convince it to run the “Dispose()” for us as part of the query! A “let” looks promising, but this won’t do anything special in the error case. But there is a pattern (other than just “using”) that lets us conveniently run code at the end of something… “foreach”. And both query-syntax and fluent-syntax already play nicely with extra “foreach” blocks – it falls (loosely) into “SelectMany”.

Thoughts; we could write an extension method that turns “IDisposable” into “IEnumerable”:

image

image

This is a step in the right direction – but those hanging “Using” blocks bother me - and the disposable object isn't quite inside the iterator - there are probably some corner-cases that would lead to items not getting disposed. But LINQ syntax is purely a query comprehension syntax; we aren’t limited to the current conventions – we can write an alternative version of “SelectMany”:

image

(hey, I never said SelectMany was pretty!)

This allows a much more interesting use:

image

This now does everything we want, without any ugly – our disposable items get disposed, and we can write complex chained queries.

I still haven’t decided whether this is too “out there” – i.e. whether it adds more benefit than it does confusion. All thoughts appreciated!

Sunday, 15 November 2009

State Assassins with experience of Reporting Services

No, this isn’t one of the more unlikely job listings on careers – instead, a bit of work from last week that I found pretty interesting.

images[1]I was essentially adding some consultancy input to an unfamiliar codebase; the problem was that the site would be running very smoothly, then page-times would go crazy for a while, and then suddenly go back to normal.

It turned out (after putting state into SQL-Server for analysis) that some of the state records were unexpectedly large. Damagingly huge. So what were they? After all, the app only used state to store one tiny innocent object…

One advantage of using a database for state is that you can get the data back out, and it turns out that it isn’t too hard to reverse a BLOB back into the state objects (it helped that stackoverflow had half the answer already).

The cuplrit? You might guess (from the post title): reporting services and the web-forms report viewer control. As soon as a user viewed a report, it was using state to cache the data – which might work great for small reports, but if the report is large it gets… fun.

Perhaps a design failing in ASP.NET state; once a large object is in state, it goes up and down the wire (between the web-server and the state storage) on every state-enabled request for that user, even when no longer using that data. So if they used the report 10 minutes ago, and are now just looking at nice slim tidy pages, they are still dragging a huge state object over the network all the time, potentially crippling the site for lots of other users.

If the page had been highly AJAX-enabled it could have been much worse. And the reason the problem ended so abruptly? My reasoning is that the user in question (who is harming the site through no fault of their own) got bored of sluggish responses and did something else for a while.

Learning points:

  • Even if you aren’t using SQL Server for state normally, it makes analysis much easier as a temporary swap-out
  • Watch out for web-controls (especially reports) that might (ab)use state
  • You can decipher a regular state BLOB – it isn’t tricky (I’ll update that SO question with an example if anyone is interested)
  • Consider partitioning your app, or disabling state in some scenarios
  • We aware of how much data your reports are dragging around with them

Tuesday, 10 November 2009

Controlling WCF / protobuf-net at the Endpoint

protobuf-net has had WCF hooks for a long time, but (to be frank) they’ve been a tad…. flakey.

Firstly, if you use “mex” generated proxy classes, it has a habit of discarding key metadata (the “Order” of the [DataMember]). This can be resolved either by using a shared DTO assembly, or by using fixups in the partial class – but…

Secondly, even if you have a shared DTO assembly, if you use the standard “add service reference” (or svcutil.exe), it duplicates the service-contract interface, and it drops the [ProtoBehavior] marker. Again, you can get around this by using the WCF channel directly or subclassing ClientBase<T> – but it would be nice if this wasn’t necessary.

Additionally, it would be very nice if the protobuf-net aspect was configurable (app.config/web.config); allowing greater reuse, and allowing parallel endpoints (with/without protobuf-net, to support vanilla and protobuf-net aware clients).

The community to the rescue

Step up Scott Prugh, who an embarrassingly long time ago very kindly supplied me with all the cogs to make this happen. Apologies for the delay, but things have been hectic and when I tried to test it my WCF brain-cells were switched off (in short: I borked my tests, couldn’t get it working, and had to shelve it for a while).

So what’s the shiny? Simply: rather than decorating your operations with [ProtoBehavior], we can apply it in the configuration instead, by adding an endpoint-behavior that references a custom WCF extension (which we must also add). This sounds worse than it is'; inside “system.ServiceModel” (in your app.config/web.config, this means adding a few lines of boilerplate:

image

And then to both client and server, we simply specify the endpoint configuration:

image

(in more complex scenarios you might want to add the custom extension to your existing endpoint configuration).

What’s the catch?

In the most part, that’s it. WCF will use protobuf-net for any suitable objects (data-contracts etc). Note that this is a coarser brush than the per-operation control, though (you could always split the interface into different endpoints, of course).

Also, protobuf-net does have some subtle differences (especially regarding empty objects), so run your unit tests etc.

Note that it only works on the full-fat WCF; it won’t help Silverlight etc, since it lacks the extension features – but that isn’t new here.

Finally, the resolver in WCF is a pain, and AFAIK wants the full assembly details including version number; so one more thing to maintain when you get new versions. If anyone knows how to get around this?

plzsendtehcodez

A minimal (but working) example is in the repo (client, server). This uses a shared DTO assembly (to avoid the need for fixups) and is based on the WCF project template. I’ll try to go back and retrofit the Northwind example shortly, and perhaps include the WCF samples in a simple download.

Monday, 2 November 2009

Highly Polished Software

Slippery surface
No, not what you think...

Is it just me that dreads finding that the office cleaners have come around? Re-attaching jostled cables and re-adjusting monitor heights is one (annoying) thing - but how much polish does a desk really need? I can barely touch my laptop without it sliding over the desk - so you can imagine what my "freak keyboard" is like.

Coffee spill
Looks like I'm going to have to spill copious amounts of coffee just to make the desk usable again. Sigh.

Tuesday, 20 October 2009

Express 2010; Basic and Expert Settings

Or: “Where has Start Without Debugging gone?”

I'm a fan of C# Express; it is great tool for quick/scratch code, and if you find my machine without at least one Express window in the task-bar, then it must be Patch Tuesday.

It looks like the Express editions of 2010 have different menu configurations for basic and advanced users:

Tools/Settings/{Basic|Expert} Settings

This makes a lot of sense, but caught me off guard - especially since many of the key chords work in both modes.

I won’t attempt to list all the differences; but if you can’t find your menu items… try switching to Expert mode.

Lots of 2010 Shiny, and me feeling stupid

Lots of changes are underway:

And because you'll want somewhere to play with beta 2:

In particular, beta 2 now includes the “Express” line-up, and TFS now supports installation on client OS and has a simplified install; maybe getting a CI server (on just the Microsoft stack) will soon be practical for the individual / small team.

Feeling stupid – moving files between guest and host

I’ve downloaded the new VS2010 line-up – onto my host OS; obviously, this is beta software with the potential for mayhem (or more likely: not be 100% uninstallable), so I want to install it on the guest. In case you haven’t tried Windows Virtual PC yet (perhaps you haven’t got Windows 7 yet), then note that it loses the Microsoft Virtual PC feature to drag files between host and guest. And my guest can only really see http (no local network). So how to get the files there?

I sat there looking silly… USB was an option (Windows Virtual PC can grab USB devices), but seemed overkill. It turns out that the answer is blindingly obvious. So obvious that it didn’t even occur to me to try it. Copy… Paste. D’oh! (this does need the integration features enabled, but that isn’t a problem).

So there you go; I’ll admit to being a muppet, and hopefully it’ll save somebody a few minutes wondering.

Sunday, 18 October 2009

Anonymous Type Tricks - a clarification

The other day, I posted about using anonymous types to pass multiple pieces of state into a method - reminder:

byte[] resp = client.Post(destUri, new {
id = 100021,
dob = DateTime.Today,
caption = "Hello world"
});
About the same time Ayende Rahien posted about a related but different use in Entity Framework - example:

.Case(
e => new {
manager = e.Manager.Id
thisIsADiscriminator = “E”
}
)

There is an important distinction between the two; in the first example, the property names (in the anonymous type) are only relevant to the caller. In the second (EF), the names are relevant to the callee. This is a huge difference. I don't know enough about EF to say more, but just a note: if you are doing something like this for your own types, please use a named property object to pass expected state into a method. Think of the kittens.

Novel uses of anonymous type initializers

On a related note; there was an interesting stackoverflow question today that shows another use of the anonymous type syntax - for specifying multiple members in a type-safe way:

IncludeProperties<IUser>(u => new {
u.ID, u.LogOnName, u.HashedPassword });

The important thing to note is that the above is an Expression (not a delegate), and the code inside tears the Expression apart to find the properties that we are interested in. It never actually invokes the code as a delegate, nor does it ever actually instantiate the anonymous type. It might just be me, but I thought that was quite cute ;-p