All Collections
FAQs and Troubleshooting
Metrics
How do you calculate overperforming scores?
How do you calculate overperforming scores?

Generally it's: (actual interactions) / (expected interactions), but the details get complicated

T
Written by Tess
Updated over a week ago

You know that if a post in CrowdTangle is 10x, that probably means it's good, and if it's -10x, that probably means it's bad. But it can get a little fuzzy when you start digging into the math, so this article is here to lay it all out.

WARNING: This gets mathy very fast. If you're game, buckle up and come along. But if not, you may want to just take our word for it.

Still here? Let's do this. First, though, an overarching philosophy:

Score numbers are meant to be quickly and easily digested. 

Scores should instantly communicate how a post is performing to anyone who is using CrowdTangle for the first time. This means that we are willing to make the background math more complicated for the sake of making the scores instantly understandable. Every algorithm comes with tradeoffs, and this one is ours.

On to the math:

First, let's start with overperforming scores of 1.0 and greater. They're the most clear, and the starting case that the rest hinges upon.

Overperforming >= 1.0x

Every post is compared to a benchmark, which is the expected value of that post. The equation is:

score = actual / expected

So if a post has 100 interactions and its benchmark was 50, that's 100/50 = 2.0x. To generate the benchmarks, we take the last 100 posts from a given account and of a given post type (link post, image post, etc.). We drop the top and bottom 25% of those 100 posts, and calculate the mean number of interactions that the middle 50% of the posts have at each age (15 minutes old, 60 minutes old, 5 hours old, etc.).

Underperforming <= -1.0x 

If you know math, and I know you do, you'll recognize that if we're doing simple division, then anything overperforming should score >= 1.0, and anything underperforming should be between 0 and 1.

But we liked the idea of making underperforming be negative, because it seemed more instantly obvious. Remember the philosophy above. So how to do that?

If something is expected to have 200 interactions and it has 100, it's at 50% of its expected value, and that seems like -2x underperforming to us. So we flip the equation and make it negative:

score = -1 * (expected / actual)

Or in this example, -1 * (200 / 100) = -2x.

Really Underperforming. Like REALLY.

What happens if a post has 0 interactions? That would mean dividing by 0, so we need to be creative.

In the above example, if the post had a benchmark of 200 and 1 actual interaction, that would be a score of -200x. Having 0 interactions should obviously score worse than that, so we simply double it and say that 0 against a benchmark of 200 is -400x. So the equation there is:

score = -2 * benchmark

Seems arbitrary? It is, a bit. But remember the philosophy --  we're willing to do it to make it clear what's happening. Having 0 interactions is really bad, so it gets a really bad score.

Overperforming between 0x and 1x

The above equations work great. But what if a post has an expected value of 1 and this post has 5? That's definitely overperforming! But it's so easy to overperform against 1, and does this one deserve to be placed alongside a post that put up 500 against a benchmark of 100?

We don't think so. So if a post is overperforming but it's below some minimum (which changes per platform), we put it between 0x and 1x.

score = actual / minimum

Imagine a post that has a benchmark of 2, and it scored a 9. That's overperforming! But it's below a minimum of 10. We don't want to treat it as underperforming, but we don't want to give it huge status. So we do 9 / 10 and come up with a score of 0.9x.

This means that it shows up as overperforming, it just sorts below all other overperforming posts that have more interactions than the minimum.

Underperforming between 0x and -1x

This is the corollary to overperforming in this range, though it might be the most confusing math yet. But you made it this far, so you can handle it.

If the post is underperforming but the benchmark is below the minimum, we want to stuff the score between 0x and -1x.

Here's how we do it:

score = -1 * ((benchmark - actual) / benchmark)

Why? Picture a benchmark of 5 and an actual of 2. That would be:
-1 * ((5 - 2) / 5) = -0.6x.

And if that post got another interaction? It would be less underperforming, so its score would edge toward 0:
-1 * ((5 - 3) / 5) = -0.4x

The end result is that it's properly noted as underperforming, but doesn't hop to the top (er, bottom) of the list.

One More Wrinkle

You've almost made it! The last thing to note is that actual and benchmark numbers have multipliers. Weights allow you to track comments and not likes, for example; or say that a like is worth 3x what a video view is worth. In all examples, weights are multiplied to get actual and benchmark numbers before the above multiplication and division takes place.

The End

Congrats! You get this: 🥇

Did this answer your question?