Will controversy kill the Hugos?

[Sadly, as I was writing this two authors Annie Bellet and Marko Kloos have withdrawn their work from the nominations.  This is a crying shame, as they're both promising young writers and deserved to have their work judged on its merit, not ponied out as pieces in a wargame.  I think there is plenty of blame to go around, and that certain voices howling in the wilderness that is the internet ought to be ashamed of themselves.  In any case, I stand by what I wrote below - I'm just not sure it's relevant to this year any more.]

Bluntly, I don’t think so.

There is no question that the bile being spilled [1] is creating a sour atmosphere for 2015, but at this point it’s not at all clear that the increase in memberships and ballots represents much more than an increased awareness that the Hugos are a fan award, and that anyone who has a membership in Worldcon – wherever it might be that year – is able to participate.  How many are enlisting in response to the calls for action on every side?  How many are just realising "hey, I could have a voice in this - that's pretty cool" and signing up to try it out?  It's hard to say.

Oh, without question there are some who emptied their pockets simply for the right to spit in someone’s eye [2] but how many?  Some will point to the outcome of the nomination ballot as evidence that the answer is “many!” but I’m not so sure that’s the case, and in fact the argument for why I’m not sure seems to have some bearing on why there are those who suspect conspiracy in recent years’ slates and results.

Look, the statistics on Hugo participation are available online at varying levels of resolution right on the Hugo Award website.  Unfortunately, the data available are really only from 2005 on [3] so it’s difficult to get a really deep understanding of how this works, but there is a clear pattern that emerges.  I’ve collated the most recent data available into a graph for convenience – but don’t trust me, go look at the numbers yourself and see what it looks like.

 [4]

Now, looking at the above there are two things that really stand out – First, it’s obvious that there has been a steady increase in both nominations and votes since 2007 [5], discounting the sudden jump last year.  What’s really remarkable though is that up until about 2012 the Hugo slate was nominated by something less than 1,100 people.  This is astounding, considering that nomination ballots can be submitted by supporting or better members of Worldcon not only in the year in which the current Hugos are to be awarded, but in the previous year and the following year as well.  Sadly, attending numbers for Worldcons are hard to come by, but recent years have seen attendance range from about 3,500 to 6,300 people.  On top of this are numbers of supporting members of course, which I suspect would take a great deal of digging to reveal.  What I do know, however, from discussions with those who have been involved in the past is that the number of nomination ballots received is typically around 15% of the total number of eligible members (remember, this includes supporting or better members from 3 years).

So what we have is 15% of eligible voters turning out to nominate works.  This isn’t terribly surprising – the huge volume of SFF being published these days is daunting, and it’s easy to talk yourself out of submitting the nomination ballot by wondering to hard about whether you’ve really read/watched enough to fill out that form.  If you haven’t participated before, it’s hard to look at those blank spaces and shuffle through everything you’ve read this year and choose just 5 of them, let me tell you.   No, it’s not surprising – and neither is the result of having the slate decided by under 1,100 people and voted on by (in an average year) about another 50% of that.[6]

Remember, there’s a huge volume of SFF material being put out right now – this is really the golden age.  With this in mind, and thinking only of print, is it really reasonable to assume that the nominators have read even a majority of it?  One person could probably watch all the SFF movies and see enough of the best SFF TV to have a good idea of what to nominate, but print? You do like eating and sleeping, right? [7]

OK, but the nominating ballots were filled out and sent in, so the people filling them must feel as though they at least know some Hugo worthy work – the problem is the question of overlap: to qualify, nominated work needs to get at least 5% of the vote.  That means that for 1000 ballots at least 50 people need to have voted for a given entry – with so much to read, you might expect nominations to be all over the map.  It’s conceivable that the convergence of text volume and time constraints could result in 1000 ballot forms containing 5,000 unique entries – i.e. that every single nominating ballot carries 5 entries that don’t appear on anyone else’s ballot.

Luckily, human nature being what it is, this is really, really unlikely.

There is a lot to read, but there are certain core venues and publishers that large swathes of fandom read.  Logically, most nominators have probably read many of the same things.  Tastes vary of course, but there are bound to be other people out there who have very similar if not identical ideas as to what the 5 best novels or stories were.  When there’s enough overlap, a nominated work passes the 5% threshold, and of the ones that do it’s the top 5 that make it to the slate.

Now, the issue here is the way in which ballots are counted – Hugo nominations and award voting are both handled with an instant run-off voting system.  That’s to say, voters rank their entries in order of preference – and if an individual voter’s first choice is knocked out of the running for some reason (deemed ineligible, didn’t get enough votes to qualify, etc) the next choice counts as their vote.  This achieves a couple of things: first, it makes it very difficult to deliberately manipulate the process, but second it means that when minority voters’ their second or even third choices go to amplify the position of leaders.  When large numbers of votes are involved, this is a good thing: it helps solidify the choices into a small number of clear leaders rather than an unmanageable scattering of things with marginal followings.  But when the numbers are small, problems creep in.

Remember what I said before – that with 1,000 ballots you need 50 votes to qualify for the slate?  But think: 50 people voting together isn’t that big a stretch, even if they’ve never even met.  And what of all the other people who voted completely differently except for that one entry?  If their other entries all got knocked out, their fifth place ranking can go to amplify the signal even further.

So yes, it’s obvious that this year’s slate campaign has had an effect on the number of votes for certain items [8] but the number of lockstep slate voters needn’t be that large to achieve this.  All it would take is for 50 people to put these items first and for enough of the scattered remainder to rank one or more items at all – and for their higher picks to be knocked out of the running for lack of general support.  Now, I’m sure there were more than 50 slate voters, but it won’t be clear just how many until after the stats are published after the con in August.  Meanwhile, what’s sure is that it’s vanishingly unlikely that the final number will really be a huge proportion of the eligible voters, and many of the new members are joining because until now they didn’t even realise they could vote.  That power is a heady thing, and I expect to see lots of new fans reading diligently and making considered choices.  This year’s slate?  Well, there’s some drek there to be sure, but there are also some not so bad work on there so I’m sure the final verdict will work out in the end, even if I might not completely agree with the winners. [9]

But that brings me to the other side of this – the whole kerfuffle started because of accusations of secret cabals and whisper campaigns.  Not having all the facts, I guess I’m not really in a position to say whether a cabal really existed, but I think I can say with conviction that I don’t believe it.  Not only do I find it difficult to believe it’s even possible to organize enough people to properly subvert an instant run-off system in the way that has been suggested, but the fact of the matter is that no cabal is necessary to explain the things that led to the accusation in the first place.

You see, the same factors that prevent a disasterous fragmentation of the vote work against us when the actual voting population is very small – like in 2007, for example.  Even with everyone voting completely honestly and diligently for the best of the best they have read that year the amplification effect that makes instant run-off so powerful to prevent fragmentation also amplifies any clustering at all.  So you don’t need to be a completely marginal voter to feel completely disenfranchised – all you need is to be up against swathes of other voters who have little in common except some marginal overlap. 

Result?  The points of overlap end up running the show. 

No cabal necessary.

The thing is, looking at the numbers and understanding how this works, it’s completely understandable how some groups might start to feel they have been locked out of the system.  The system is practically designed to do it, because giving them their full voice would make it impossible to get a result out of the vote at such a low level of participation.  The problem isn’t a cabal, it’s not a lack of strong SFF of all kinds, it’s statistics.

What’s needed is a larger pool of nominators and voters – and the good news is that the scandals of the last couple of years might well provide that for the long term.  And in that case, the controversy won’t kill the Hugos, but reinvigorate them.

The problem facing Worldcon right now, of course, is to survive the next couple of years.  The firestorm is fierce, and no matter what you think of past slates or the Sad Puppies campaign's effectiveness it's impossible to deny that they've seriously distorted the results this time around.  People are understandably upset on both sides, and everyone is hampered by the inability to see the whole picture,  We all end up with opinions on what's really happening and why, and none of us really has the truth of it. So yes, there are probably questionable items on the ballot this year (as, from some perspectives, there are in every year when someone, somewhere believes they've been robbed the joy of seeing a work they really liked get nominated and maybe win) and what's there might or might not actually represent the best of what was published in 2014.

Does that matter?  It never has been truly representative because the slate doesn't represent the best of what was published, but the best of what the nominators actually read.

OK, so maybe some of the works were nominated for nefarious reasons not related to the nominators actually having read them and deciding they were good enough.  That doesn't mean the show can't go on - it just means you might have to read and judge work you wouldn't ordinarily have read.  Maybe you won't like it and will be convinced it never passed muster for nomination.  Maybe you won't like it but will grudgingly admit that it was actually pretty well written in spite of that.  Or maybe you'll actually like it, who knows.  I can't tell you what your reaction to the work on the ballot will be like - but more importantly neither can you unless you read it.

It's quite possible you could read everything and conclude that nah, this year there's no one who makes the grade.  Or it could be that you will read and discover something new that you had previously dismissed.  The same goes the other way of course: maybe you'll read something you were told you should like because [reasons] and find it just isn't up to snuff.

But to make these decisions you actually do need to read the work.  And by reading the work and making a good faith judgement as to whether the writing is really Hugo quality or not even if the stories themselves aren't your usual cup of tea you defuse the bomb that has been set for us.  The best thing for the Hugos in my opinion would be for everyone to stop writing and start reading, to explain in glorious, gruesome detail why they don't like things, why those things didn't make the Hugo grade.  There would still be scandal, there would still be controversy, but at least it would be an honest conversation.

So read, judge, and vote in good faith, and we all win.

---

1. Let’s be honest: on both sides.

2. See 1.

3. With fragmentary information available further back, for example in 1998 and 1996

4. These are the raw numbers, which I didn’t think it was worth cluttering the post with:

Note that I have indicated 0 votes for the current year – that’s because no-one has voted yet.  I have also noted several entries in red – this is because the convention for reporting numbers was only formalized recently, and in those years actual information on nominations and ballots was unavailable – I have made the assumption that all the ballots received in those entries included the “Best Novel” category – not strictly true, but the correspondence between “Best Novel” ballots and total ballots received seems strong in other years.  If you have a better idea, I’m happy to hear it.

5. Actually, we could discount 2007 as an outlier, as Nippon 2007 would naturally have had lower attendance, being a primarly English language con held in a non-English speaking country.  Notice how the ratio of nominators to voters doesn't match the pattern either.

6. And there's not really any way of knowing whether this represents all nominating voters - how many nominate but for one reason or another don't end up voting?  This is something that isn't captured in the system.  The profile of those who vote may be drastically different from the profile of those who nominate.

7.Granted, I’m probably not the only one who would give both up if it were possible, just to have more time to read.

8. Let’s be blunt: lots of items.

9. Is there ever a Hugo ceremony that makes everyone happy? [10]

10. Trick question - the answer is always no because everyone always has something to complain about, trust me.