The Thumbs-Economy – Looking at the Data Behind a Review’s Popularity

Since the beginning, I’ve been fortunate that my reviews became popular on BGG quite quickly. My assumption has always been that it’s a mixture of the format (long form, in-depth), having lots of pictures that help to illustrate points, and that I don’t write fluff pieces. I don’t accept review copies and can thus write about what I really liked or didn’t like for a game, and people seem to feel that’s helpful. Recent events caused me to revisit a couple of fundamental questions:  does it make sense to write negative reviews? To review older games? How long or short should a review be?

While I can’t answer these questions universally, I realised I’ve been doing this for almost three years now. So effectively, I’m my own dataset! I’ve only published 62 first impressions & reviews, but the benefit of this data set is that my style has stayed relatively consistent over the years. If you look at my Messina 1347 first impressions, its structure and length is rather similar to more recent reviews I’ve written. Boardgamegeek unfortunately gives reviewers almost no analytics, for example there is no way to see view count. But I tried to combine the public information there with some data from my own website and see if I can spot some trends.

Disclaimer: If you don’t know Tyler Vigen’s work, definite check out his side on correlation is not causation. He has very funny illustrations of this, my favourite being the number of movies released by Nicolas Cage plotted against drownings in pools. I’m also limited with the visuals here because I quickly hacked this together with Apple Numbers.

Do My Positive Reviews Get More Likes?

I intentionally don’t put ratings in my reviews because I honestly believe it’s better if readers read my experience and then figure out for themselves which part resonate for them and which don’t. But for the sake of this analysis, I quickly put general categories (not for me, neutral, positive, outstanding) on to them and then roughly mapped them to 6-9 on the BGG scale.

The first graph shows that a positive or raving review I write definitely – on average – gets way more thumbs up, almost by a factor of two. That’s hardly surprising: people who like a game are more engaged with it and thus more likely to give a thumbs up if they find someone agreeing with their assessment.

Also the more popular a game is (=higher BGG score), the more thumbs up my review gets. That intuitively sounds right as a higher score usually means more people are interested in a game and so are inclined to read a review. I’ve noticed multiple times that if people are not looking for a particular game on BGG (for example because it is already 10 years old), there will be few people reading a review I write and thus few thumbs up.

Are My Reviews on Crowd Funding Games Doing Better or Worse Than Retail?

I buy quite a few crowd funding games, the vast majority of those second hand. Part of it is because it’s fun to be part of the buzz, part curiosity, but also because I like nice productions. Looking at the chart though, my reviews of crowd funding games started strong and now do on average about the same as those for retail games. My assumption for this is that lately there have been a higher number of crowd funding games that left me cold and as we have seen, a lower “rating” causes less thumbs up. The graph plots the factor of thumbs up my review got to the best one someone else posted (e.g. 3 means my review got three times as many thumbs up).

If we look at how my ratings changed for both categories of games, we see there were some crowd funding games that really blew me away during my first year or so (e.g. Carnegie, Foundations of Rome, Castles of Mad King Ludwig Deluxe) of writing reviews but lately they were rather in the bad-to-okay range. I don’t think it’s me having changed my taste because I still like those “older” crowd funding projects as much as I do now. However, it matches a trend I’m seeing that more recently fulfilled Kickstarter tend to have amazing productions but the games don’t seemed as well developed anymore. There also seem to be less reprints/deluxe editions of older tried-and-tested games.

Interesting is also that in the last months, those games that have been blowing me away were older designs like those from Splotter Spellen. Without going “back in time”, the retail trend would likely also go down.

Does It Make Sense to Write About Older Games?

This primarily depends on how one defines success. The best metric I could come up with was calculating the factor between the thumbs up my review got and the best thumbs up any other review on BGG got. So 100% means I’m doing as well as the best other review, a factor of 200% that my piece got twice the number of thumbs up.

First, let’s look at the number of years between release and me writing about a game: of course the granularity is pretty bad and I haven’t yet written about that many games that were older than 2 years, but still, there is a noticeable drop off even if the game was released the previous year. I don’t have hard data on this, but I would say the real drop off is actually somewhere between 3 and 6 months after release. Writing about even older games doesn’t really make sense because a) few people will browse these forums to read reviews and b) there will already be a lot of other reviews.

Do You Have To Be Among The First Ones?

Looking at Youtube, it always seems like everyone is rushing at being the first to report about a game. There are previews, playthroughs of prototypes, etc. Since I buy my games and usually don’t accept review copies, the best I can do is pre-order a game or pick it up early when it comes to retail. But written reviews are often happening after all the YouTube reviews are out already because a) writers seem to do more plays before reporting on them, and b) most other writers also don’t have access to pre-release copies, same as me.

So let’s see how my reviews stacked up against the “competition” depending on how many reviews already were there when I published mine. To be precise, the graph shows where my review was when ordering all BGG reviews by date of publishing:

Based on that data, I would say if there are around 6 reviews on BGG already, the best one can hope for is getting like half the thumbs up the best review has. What definitely helps is if one has a geek list or some other form of subscriber base so one’s own review has a good chance to be pushed above the magical number of 18-20 thumbs which usually pushes it to the BGG front page (and again results in more eyeballs).

How Long Should It Be?

I’m know for my in-depth writing and so this data set cannot be generalised to other reviewers. But for me, there seems to be a sweet spot around 3000 words which is around 15min of reading time. There can be multiple reasons for this: less word count usually means less interesting stuff to talk about (e.g. an expansion vs a completely new game). Or I’m personally less interested in the game and therefore my personal assessment of the game is worse than other games, and that will as we have already seen naturally result in lower ratings.

When it comes to engagement in the form of comments though, a slightly higher word count of 4000-5000 seems to work well. However, this might also be rather the other way around: an interesting game might just naturally have more things worth discussing, leading to me writing more words and readers leaving more comments.

How Many Images Should There Be?

If I go with how popular the review is (again measured against other reviews on BGG), the words per image don’t seem to matter that much. However, I like taking pictures and integrating them into my texts, so I don’t really have numbers on how they would have done without them. I always assume they are a big part of why I manage to get such good numbers for my reviews.

Should I Write About Less Complex Games?

Well, so far the answer seems to be no, which is hardly surprising. The way I write, it’s more interesting if there really is meat to the bone. I’ve seen some very short reviews on BGG doing really well, same for reviews that basically just said how much someone liked a game and not much about the game itself. Still, I like dissecting games, really work at it to find out why a game works for me or not and if there are any flaws in the design. Heavier games also fit my taste better, so I guess I stick with the mid-to-heavy range when it comes to writing.

Conclusion

So, any revolutionary new insights? I don’t think so. I knew that writing about older games and still getting eyeballs on them is challenging. For example, I absolutely love Antiquity and Indonesia, some of the best purchases I have made. But most people that even have heard of them either already have a copy or are not that much browsing the BGG forums or googling for them. Writing about them is something one should do for fun, not to get a bigger audience.

If you want lots of thumbs, most importantly write a) positive reviews about b) games that have been released in the last 3 months or so, and c) have a rating of at least 7.5. Stick with around 3000 words and have an image approx every 300-400 words. Build a subscriber base to help you push your reviews to the BGG front-page. And don’t worry that much about buying all the crowd funding projects.

For me, I’ve from the beginning decided that having the biggest audience isn’t my goal. I want to have fun writing and exploring games and hopefully help you make more informed decisions when buying games so you find more you love and buy less you don’t. So doing all those numbers was more a fun experiment rather than that I want to actually use the data. But still interesting, right?


P.S.: I’ve filtered out my Heat: Pedal to the Metal review out of a couple of graphs. That single 500+ thumbs data point just messed up a lot of stats.

Leave a Comment

Your email address will not be published. Required fields are marked *