We broke and matched two records yesterday for two of our most important metrics. We had a total of 45 links added in the one day, and they were all added by 7 different people, beating our past record of 35 links, and matching the 7 which we have reached twice previously.

Measuring Activities Which Matter

Back in January we took the time to identify the metrics which really mattered and started to track them to ensure we were actually moving in the right direction. You see, it is too easy to get caught up on ‘Vanity Metrics‘ (or Bullshit Metrics as I saw them called just the other day) and think that things like getting 10,000 visitors in a day means you are on the road to success! But in reality, it just means you got 10,000 people to come to your website. It actually has no bearing on your likelihood of success at all.

Our vanity metrics mostly include our registrations. We have over 4,000 accounts now. It is exciting to have so many accounts when still in the developmental beta testing phase, and really exciting that virtually all of that growth was organic, word of mouth, or through spontaneous coverage. But at the end of the day, registrations don’t add rebuttals. They don’t even necessarily mean people are using the plugin! They imply interest, but we can do better than imply…

Actionable Metrics

For us, success means mapping every rebuttal on the internet against every claim it rebuts, and delivering those connections to every internet user who visits a rebutted page. So our most important metrics are:

1. How many people have the plugin running?
2. How many people get rebuttal alerts?
3. How many links are being added?

There are multiple dimensions to each of these metrics. Daily/weekly/monthly? Is it better to have one person add 10 links, or 10 people add 1 link? Things like that, and we try to keep track of what we think matters the most in each of them.

So today we had a record number of links added, at 45. This is cool, though we need to break it down a bit, because technically, on 22 of those links were added by people other than myself and Craig (we keep us separate in the stats because, reasonably enough, we add more rebuttals than anyone else). The previous record for rebuttals in a day was 31 (19 by not us) in March 2013, while the most ever added by people other than Craig and myself was actually 23 in August last year (shortly after TAM2012!).

As for the number of people adding links – this is a slightly different look at the same metric. It is cool for one person to add lots of links, but it is much cooler if we have lots of people doing a little work, and feeling like part of a larger active community, rather than like the only person putting in any effort. So the more people we have adding rebuttals, the better it is.

I’m not sure how many people we have who have added at least one rebuttal, though I am confident it is over 100. But total amount is less important than day to day operations – how many people take some action each day? Or each week?

We have now had 7 different people adding rebuttals on one day at least 3 times over the past month and a half. I don’t have the numbers on what our average number is, but it is trending up and to the right, like all good stats should. ie: Each week is seeing an increase in the average number of people adding links. Typically hovering between 3 and 5 each day.

I expect to break 7 any day over the coming weeks, though I also realise we need to start collecting this stat on a weekly basis too.

The 90-9-1 Rule

I think it is interesting to reflect on our numbers just quickly in light of the the 90-9-1 rule, which posits that 1% create most of the content, 9% edit, modify or add some content, and 90% use the content/lurk. We had 758 people with plugin activity on the day we had 7 users adding rebuttals. Of course we don’t really have a 10% section because we don’t have any half-participation options. You either submit rebuttals, or do nothing (ignoring requests, which most of you do… :p).

So to have roughly the 1% participatory rate is amazing when you consider that adding rebuttals isn’t as easy as making a forum post, where you can say pretty much anything you want. Rebuttals require finding, and that is not an easy thing to do – hence my desire to create rbutr.

In general though, I think we actually average lower than 1%. This was just a superficial snapshot of one day, so hardly representative, and the 90-9-1 rule should really be applied to a longer time frame, and a much larger amount of content than that created in one day. I’ll keep an eye on it for the future though, because if we do remain around 1% then that is pretty amazing, considering the difficulty surrounding finding rebuttals, and the fact that you either have one, or you don’t.

I also hope it will improve as we improve our usability, plugin features, and on-boarding processes.

This post ended up being much longer than I expected. I hope you enjoyed a little insight and tour of how we think about stats, and track our growth. If you aren’t

{ 2 comments }