Thursday 12 April 2018

Having a serious conversation about open source

Having a serious conversation about open source

A certain topic has been surfacing a lot lately on places like twitter; one that I've quietly tried to ignore, but which has been gnawing away at me slowly. It is seemingly the most taboo, dirty and avoided topics.

Open source and money

See, I said it was taboo and dirty

Talking openly about money is always hard, but when you combine money and open source, it very quickly devolves into a metaphorical warzone, complete with entrenched camps, propaganda, etc.

This is a complex area, and if I mis-speak I hope that you'll afford me some generosity in interpreting my words as meant constructively and positively. This is largely a bit of a brain-dump; I don't intend it to be too ranty or preachy, but I'm probably not the best judge of that.

I absolutely love open source and the open source community. I love sharing ideas, being challenged by requirements and perspectives outside of my own horizon, benefitting from the contributions and wisdom of like-minded folks etc. I love that packages I've created (usually originally because I needed to solve a problem that vexed me) have been downloaded and used to help people tens of millions of times - that's a greate feeling! I love that I have benefitted indirectly from community recognition (including things like an MVP award), and professionally (I doubt I'd have got my job at Stack Overflow if the team hadn't been using protobuf-net from a very early date).

But: the consumers of open source (and I very much include myself in this) have become... for want of a better word: entitled. We've essentially reinforced that software is free (edit: I mean in the "beer" sense, not in the "Stallman" sense). That our efforts - as an open source community: have no value beyond the occasional pat on a back. Perhaps worse, it undermines the efforts of those developers trying to earn an honest buck by offering professional products in the same area... or maybe it just forces them to offer very clear advantages and extra features, which perhaps is a good thing for them? Or is that just me trying to suppress a sense of guilt at cutting off someone else's customer base?

Yes it is true that some open source projects get great community backing from companies benefitting from the technology, but that seems to be the minority. Most open source projects... just don't.

  • maybe this is because the project isn't popular
  • maybe it is popular, but not in a way that helps "enterprise" customers (the people most likely to pay)
  • maybe the project team simply haven't made a pay option available, which could be lack of confidence, or lack or know-how, or legal issues - or it could be the expectation that the software is completely free
  • maybe people like it, but not enough to pay anything towards it
  • maybe anything other than the "free to use and open licensing" is massively disruptive to the dominant tool-chain for accessing libraries in a particular ecosytem (npm, nuget, etc)
  • maybe with multiple contributors of different-sized efforts, it would become massively confusing as to who receives what money, if any was made
  • (added) does your daytime employment contract prohibit taking payment for additional work

But whatever the reason; most open source libraries don't get financially supported. Sometimes this might not be a problem; maybe a library is sponsored internally by a company that has then made that software available for other people to benefit from. But: the moment a library hits the public, it has to deal with all the scenarios and problems and edge cases that the originating company didn't need to worry about. For example, you'd be amazed at how much trouble SQLite causes dapper due to the data types, or how much complexity things like "sentinel", "cluster" and "modules" make for redis clients. But: the originating company (Stack Overflow in the case of dapper, etc) doesn't use those features, so they don't get fixed on company time. This is a recurring theme in many such projects - and now you're in an even more complex place where the people maintaining and advancing something are doing a lot of that work on their own time, but it is now even more awkward to ask the simple question: "this thing that I'm doing to benefit real users: am I getting paid for this? can I even accept contributions other than PRs?".

Is this a problem?

Perhaps, perhaps not. I'm certainly not bitter; I love working on this stuff - I do it for hobby and pleasure reasons too (I love solving challenging problems), and it has hugely advanced my knowledge, but I have to be honest and admit that there's a peice of me that thinks an opportunity has been missed. Take protobuf-net: if I sat down and added up the hours that I've spent on that, it would be horrifying. And I know people are succeding with it, and using it for commercial gain - I get the support emails from people using it in incredibly interesting ways.

Quite a while back I tried adding small and discreet contribution options for protobuf-net (think: "buy me a beer"). It wasn't entirely unsuccessful: to date I've received a little over (edit: incorrectly stated USD) GBP 100 in direct contributions; most of that was in one very much appreciated donation - that I can't find the details of because "pledgie" closed down. But overall, almost all of the work done has been completely for free. Again, I don't resent this, but it feels that there's a huge imbalance in terms of who is doing the work, versus who is benefiting from the work. There is very little motivation for companies benefiting from open source to contribute back to it - even when they're using it in commercial ways that are helping them create profits from successful products or services.

In my view, this is just as bad for the consuming company as it is for the author: if the developer isn't motivated to improve and maintain a library that you depend on for your successful product, then: that sounds a lot like a supply-chain risk. But then, I guess you can just move onto the next competing free tool if one author burns out.

I'm not sure I have a solution here, but I do think there's a very real conversation that we shouldn't be afraid of having, about how we - as an industry - avoid open source being treated simply as free contractors.

I'm toying with ideas

For protobuf-net, I'm aware that a good number of my users are doing things like games, which tend to run on limited runtimes that don't tend to have runtime-emit support (they are "AOT" runtimes). This really hurts the performance of reflection-based libraries. I've been toying for ages with new ideas to make protobuf-net work much better on those platforms by having much richer build tooling that does all of the emit code up-front as part of the build, and one of the ideas I'm playing with is to:

  • keep the core protobuf-net runtime library "as is" (and continue to make it available for free)
  • add an additional separate package that adds the AOT features
  • but make this package dual licencesed: GPL or purchase non-GPL

But I'm very very unsure about this. Philosophically, I kinda hate the GPL. I just do. But the stickiness of the GPL might be the thing that actually gets some customers - the ones who care about compliance - to pay a little for it. It doesn't have to be much; just enough to make it feel justified to spend such a vast amount of time developing these complex features. As for the people that don't care about compliance: they weren't going to pay anything anyway, so frankly it isn't worth worrying about what they do.

Is this a terrible idea? Is this just me exploiting the fact that I know some users have AOT requirements? Am I just being greedy in my middle-age? Is this just going to make a nightmare of accounting and legal problems? Am I just being grumpy? Should I just accept that open source is free work? I genuinely don't know, and I haven't made my mind up on what to do. I'm genuinely interested in what people think though; comments should be open below (edit: comments were open, but... blog spam, so much blog span; maybe tweet me @marcgravell).