Understand the building blocks of community measurement to construct a measurement case as unique as your business and community are.
Back in July of 2020, I wrote an article for my blog called “Three Metrics Every Community Manager Should Track”. (I guess community metrics are a summer topic!).
I was coming off the heels of a big measurement project at Teachable that allowed me to, at long last, go beyond reporting only on engagement metrics and finally report on the impact my community program had on customer retention. To date, this is one of the most popular blog posts I’ve ever written.
But, over the last four years, my views on metrics have evolved. They haven’t changed, exactly—when I revisit that old post, I still find it valuable. But, they have grown quite a bit. I now believe that, contrary to the title of my previous post, community metrics look really different from one project to the next—and they should. That’s because:
The best communities serve business goals, and no two businesses have the exact same goals (even if they may appear similar)
Community metrics, and all metrics, must be contextualized to make sense. You may think you're looking at equivalent metrics at two different businesses that are actually telling completely different stories.
Metrics are all about answering questions—and the questions you might have about one community instance are going to be very different than the ones you have about another.
So, in short: we shouldn’t be prescribing which metrics are appropriate for a specific community project without knowing quite a bit about the dimensions of that project.
We shouldn’t be prescribing which metrics are appropriate for a specific community project without knowing quite a bit about the dimensions of that project.
That said, over the last four years since I first wrote that post, because my work first at Commsor and then as an independent strategy consultant has afforded me the opportunity to work with dozens of different community projects and companies, I have observed that most community projects are using variations from the same menu of options for measuring and reporting the value of their community.
So, rather than updating my old post, I decided to create something brand new for you here—and in keeping with tradition to some of my recent posts that y’all have enjoyed (like my Complete Guide to Online Community Onboarding and my Complete Guide to Creating an Online Community Content Calendar), I’ve decided to try to create a Complete Guide to Proving the ROI, or Return on Investment, of your Online Community.
What’ll make this guide complete? I’ll walk you through:
A menu of options, or building blocks, to prove community ROI connected to different business cases
A guide to deciding which options are best for your use case
Examples of how community industry leaders are doing it today
Plus, I put together a responsive template (i.e.—plug n' play spreadsheet!) you can use to plug in your own numbers and understand how some of these metric calculations work in practice. This is one of my best templates yet—I almost melted my brain putting it together for you—so I’ve decided to make it a part of my premium template library.
That means if you want access to it, you can either buy it à la carte or as part of my complete template library:
If that’s not for you, don’t worry—I still have a ton of free resources and templates available all over this blog, and this blog post will explain these methods so you can create your own measurement case. But, if you want to plug in the numbers and have my formulas spit out answers, you're gonna need the template.
Without further ado, let’s jump into the guide!
An approach to measuring community ROI (Return on Investment)
Before I get into any specific methodologies, I want to do some scene-setting around what I’m really getting at when I talk about community and ROI so that readers can understand my approach to this post.
In my experience, I’ve found that it’s often actually somewhat difficult to get community folks to talk about their projects from a financial perspective. When I ask community builders about measurement and impact, they seem to much more readily reach for things like:
“My community members learn and feel connected when they’re a part of my community.”
“My community helps my members achieve the mission of my brand.”
“My community helps my organization learn about and support our members.”
These things are important, but ROI they are not. Perhaps the concept of “ROI” has become so colloquialized that we’ve started to use this term interchangeably with things like “impact” or “outcome,” which have broader meanings.
But, however obvious it may seem, it’s important to state plainly that ROI = Return On Investment. In other words, does the amount of money the project generates outweigh the amount of money the project costs?
And while I find many community builders are fundamentally people-oriented, and feel somewhat allergic to boiling down their projects into such cold components as dollars-and-cents, it’s essential to learn to measure and express ROI as concretely as possible.
If you believe strongly in the “softer” outputs of your community I bulleted out above, you have to believe just as strongly in ROI—it’s what’ll keep your project solvent and allow you to continue delivering mission-oriented outcomes.
If you believe strongly in the “softer” outputs of your community, you have to believe just as strongly in ROI—it’s what’ll keep your project solvent and allow you to continue delivering mission-oriented outcomes.
So, what I’m focussed on in this post is not measurement generally—that would include measuring engagement outcomes, which I will mostly not address here—but measuring financial impact.
But, don’t worry, we’re not throwing the baby out with the bathwater: in the next section, we’ll start to tease out the potential connection between engagement metrics and true business outcomes.
Engagement Metrics, Business Metrics, and "The Relentless Why"
The good news is that if you find it easy to reach for softer, more engagement-outcome oriented statements when you’re asked about measurement, you’re likely only a few steps away from tightening up your approach and starting to understand ROI.
That’s because engagement-outcome statements often have baked-in assumptions about financial outcomes that we can tease out using an exercise I call “The Relentless Why.”
“The Relentless Why” is an exercise I use with my clients (and myself!) to try to really get to the bottom of why we’re pursuing a specific engagement outcome. It’s very simple—all you have to do is keep asking why until you finally hit upon a business outcome.
For example, let’s say I ask a client to tell me about what they think their business will get out of hosting a community, and they answer:
“I think hosting a community will get my members to come to my events.”
(And by the way—this is anonymized and paraphrased, but this is a real example).
There’s an engagement outcome in there (event attendance), but not a business outcome, so I would push on that with a why:
“Why do you want your members to come to your events?”
“So that they learn and feel connected to my brand.”
“Why do you want your members to learn and be more connected to your brand?”
“Because then they’ll be more successful with my brand’s product.”
“Why do you want your members to be successful with your product?”
“Because then they’ll stick around and stay customers.”
And BOOM. We’ve hit on a business outcome that’s measurable with dollars and cents—customer retention. This kind of exercise may feel obvious or pedantic, but I’ve found it to be a powerful way to bridge the gap between engagement metrics and actual business outcomes, and start to understand how we might approach ROI measurement in our business.
Of course, that’s not the end of the story—we have to first validate that the hypothesized business impact we’ve reached (in the example above, customer retention) is the right fit for our business and use case, and we have to figure out how to measure it accurately. We’ll talk more about all that as the post goes on. But, this is a great first step to start to excavate business impact from our assumptions.
Now, let’s start to take a closer look at what some of our options are for measuring community ROI.
A menu of options for measuring online community ROI
Now, let’s start exploring some options for measuring the ROI of your community project. I’ve decided to present this almost like a “menu” because I really believe that no two community measurement cases are exactly alike—however, they will tend to loosely connect to the categories and ideas I’ve listed below. You should think of these as building blocks to customize to suit your use case.
This menu is long—cheesecake factory long—because there are lots of options for how you can prove the ROI of your community. I’ll talk in the next section about how to approach choosing which forms of measurement are right for you, and the template will give you the chance to plug in your own numbers and compare some of these methods, but for now, let’s just take a look at some options:
Option A: Cohort Analysis on Customer Retention
A popular form of measurement for communities hoping to impact metrics related to customer retention is cohort analysis. In other words: when you take a cohort of customers who are community members, and a cohort of customers who are not community members, how do their performances compare on metrics like:
Churn percentages—i.e., what percentage of the group stops paying for the service in a given timeframe (generally a month, quarter, or year). This example and how to extrapolate it into concrete ROI is included in my template spreadsheet.
Lifetime value (LTV)—i.e., what is the average total dollar amount earned from the customers in a selected group. (This has to do with understanding how long on average customers stay customers, and might also help you understand how to build community journeys accordingly.)
Repeat Purchase Rate (RPR)—i.e., how often are customers in the given group coming back to repeat a purchase. Particularly relevant in retail scenarios without automatic recurring revenue for whom "churn" is a less relevant measure.
As you’ve probably guessed, these metrics make the most sense to use for customer communities—it can be harder to understand this specific type of cohort comparison when comparing a larger customer base to a community or subset of a community who are not, or not all, customers.
If you are running a community where some members are customers and some are not, try to only apply this type of cohort comparison to the customer subset within the community so that you're not comparing apples and oranges.
A few things to keep in mind about cohort analysis as a measurement tool for ROI:
It can be used for more than just retention metrics: Once you understand how to use cohort analysis as a measurement strategy, it will start to become a powerful tool for answering a variety of questions you may have about the impact of your community—not just retention-based metrics. That’s because it allows us to compare how community members stack up against non-community members—so, cohort analysis can help us answer variations of the question: “does participating in my community make members better at x behavior or outcome?” You could plug in a lot of things that have business impact for x there—here are some examples:
Generating revenue the business earns a percentage on—many business models (my alma mater Teachable is an example of this, but so are PayPal, Etsy, and Airbnb) rely on taking a percentage of sales generated on marketplace platforms.
Generating case studies or other advocacy content that help the business win contracts
Participating effectively in affiliate or referral programs that help the business win new contracts
It requires data infrastructure:
To do this type of analysis, you will generally need to rely on existing data infrastructure that likely lives outside of your core community platform. In other words, it’s going to be hard to understand community impact on churn if your business doesn’t already have a mechanism for understanding churn at large.
Further, this type of analysis requires you to be able to map community accounts onto customer accounts—in other words, you need to know that x person in your community holds y customer account. This often requires integration between your community and core data platforms and the collaboration of an internal data scientist.
It can be deepened with longitudinal studies: One big challenge many have pointed out about cohort comparison is that while it tends to do a good job showing correlation between community involvement and business outcomes, it doesn’t speak directly to causation. In other words, the fact that community members churn less frequently than non-community members might be an output of the fact that more engaged customers are both more likely to stick around and more likely to participate in communities. However, most community builders are going to want to indicate not just that community is related to churn, but that it’s an effective intervention against churn. Longitudinal research is a way to combat this. To do this, you need to follow the same cohort of customers over a period of time to try to track how participating in the community might change their specific business behaviors versus a control group. Conducting longitudinal studies can help you understand how much of the delta between community and non-community cohort performance can be explained by causality versus correlation.
It can also be deepened with more granular engagement studies: While at the top level, cohort analysis is often comparing the binary of community member versus non community member, community builders know that not all community members are exactly alike. To understand which community interventions are most effective at producing business outcomes, and therefor worth investing in, you may wish to also analyze things like:
Do more engaged community members perform better on the target metrics versus less engaged ones?
Do community members who engage in a certain format (i.e., events versus textual conversations) perform better on the target metrics?
Does the impact of community engagement on the target metrics “stick,” or is it only measurable while the members are actively engaged?
Ready to start playing around with some of these numbers? My premium ROI template lets you plug in some of these numbers to a spreadsheet and spits out answers via built-in formulas.
I focus on the "churn reduction" example as a customizable base block in the template, which can also be tweaked to suit a different use case. The template provides infrastructure to understand the mechanisms behind cohort comparison calculations that can be adapted for different numbers.
You can either grab that à la carte or as part of my complete template library here:
Option B: Lead Generation, New Customer Acquisition, and Expansion Revenue
In the last section, we focussed on a method of understanding ROI that is mostly applicable to communities that are primarily or entirely made up of customers. This persona focus allowed us to look at the impact of community on customer behaviors like churn.
However, many communities do not function this way and may be actually composed primarily of non-customers. Community instances like this can still yield enormous business impact for exactly this reason: they are geared toward a group of non-customers that the business hopes to either convert directly or mobilize to aid in converting other potential customers outside of the community.
This can be powerful because communities are higher-touch channels that might go beyond a standard sales motion (i.e., email marketing) and yield higher conversion rates, and because communities allow you to mobilize members to take part in the marketing effort itself, rather than only being its target. Here are a few of the ways that companies that are making this bet measure this category to prove ROI:
Community qualified leads & conversions: the most straightforward way to approach understanding community ROI in this category is to simply track the number of leads sourced from your community rather than another channel, and then track the conversion rates from those channels and extrapolate them into revenue. If your community is very small and high-touch you can track this manually; however, it’s best to start using tracking links and automated dashboards as quickly as possible to ensure nothing slips through the cracks. This example is included in the template spreadsheet.
Expansion revenue: Using a similar approach to the one described above, you may also wish to track the number of upgrades or repeat purchases that come from your community and the resulting revenue. This can be a great option for communities that serve primarily customers, and therefore can’t attribute original sales to the community, but still want to demonstrate an impact on direct revenue. This example is also included in the template spreadsheet.
Cost per acquisition: Because some companies rely not just on organic, but paid acquisition (i.e., running ads or other forms of paid marketing), many companies track the amount of money they pay on average to acquire each new customer, or even each new lead. Demonstrating that community is a cheaper, and ideally more efficient, way to acquire and close new customers is another possible way to feed into your ROI calculations around your revenue and acquisition impact.
A few things to keep in mind about tracking revenue impact to calculate community ROI:
Beware of attribution: just because a member of your community became a customer or upgraded doesn’t mean that motion can be attributed to the community. Using tracking links and following your company’s standard practices around first-and-last touch attribution can be a good way to make sure executives and other teams will understand and agree with your analysis.
Community expert Scott Baldwin, who leads community at HiveMQ, had this to say about this topic:
“You need to understand your community’s impact on open pipeline and closed won deals across upsell, new business, and renewals. Of course there are other factors to consider from across the funnel—but these will help you know.
Attribution can be a tricky game, but often you can look at first and last touch, your event sign-ups and attendance, and how folks move from top of funnel (expressing interest) to warming up, to potential folks your organization can engage with more. I also like to remind folks on the go-to-market side that people don’t join a community to be sold to. So, use these signals as potential markers of interest toward the goal of building a deeper relationship with people vs. jumping right into selling. That’s where something like community being a part of your lead scoring comes into play (e.g. X points for joining, X points for attending an event, X points for downloading a resource, etc.)”
Depending on your community setup, this strategy can be combined with the cohort comparison strategies I shared above. In other words, it’s not either or: you may find yourself tracking one thing from one category, and one thing from another. My spreadsheet will allow you to easily choose forms of measurement from different categories, total them up, and compare them against your losses to create a comprehensive picture of ROI.
Option C: Business Cost Reductions
The category I’ve saved for last here actually has the longest history of being used at large corporate organizations to measure the impact of community on business outputs: the category of business cost reductions. That’s because this category is appropriate for measuring the ROI of a tried-and-true community instance: the support community.
This category is all about measuring how using a scaled community instance saves money on activities that businesses have to do anyways, and therefore often already understand the cost of. This strategy is a standard and successful one because it doesn’t aim to suggest or prove the impact of a net-new business activity, it just aims to demonstrate how community saves money on something that’s already happening.
Here are some examples of ways you can measure your community’s cost-reducing impact:
Support cost reduction: The classic methodology for proving community ROI. Most communities do this by tracking the number of support cases that are resolved via the community, and therefore deflected from 1:1 ticketing platforms. Because many companies already track the “cost per interaction” on their support channels (and the averages might shock you), it’s relatively easy to extrapolate this into a dollar-amount-saved by the community. This metric is usually referred to as “ticket deflection,” and many enterprise community platforms have in-platform ways to track it, which may contribute to why it’s such a popular form of measurement. I’ve included this example in my spreadsheet template, which can be used as a base block to easily tweak and understand other forms of cost-saving measurement.
Content leads: Similarly to how you can track support-cost deflection, you can track the cost deflection of other business activities with a known price tag if you’re able to accomplish them more cheaply with the collaboration of community members. Content creation is a great place to start. Community members can often help reduce the:
Cost of creating a piece of lead-generating value (blog, social media post)
Cost of creating a case-study that assists in closing a sales.
Tracking how community-generated content converts versus team-generated content can also help make a case for investment.
Cost reduction of product-team efforts: If you are able to estimate the cost of activities like securing a user interview or shipping a product before and after community intervention, you can also extrapolate this technique onto your collaborations with your product team.
The long and short of it: if you can understand the price tag of something your company already does, and show that it costs less to do it via community, you’re one step closer to proving ROI.
We've covered the return...what about the investment?
Now we’ve walked through your options to prove that your community is generating value from a financial perspective. But, that’s only one half of the ROI equation—the return part. What about the investment?
In order to really prove ROI, you have to not only show that your community is generating revenue that sounds impressive in a vacuum, but that that revenue is adequately outweighing the financial investment that your company is making in generating those returns. In simpler words, are you making more money for the business than you’re spending?
To answer that question, you have to understand how much money your community program costs to operate. This may seem daunting, but in reality is usually relatively simple if it’s not already documented somewhere at your company. Most community team budgets boil down into a few key categories, with the top two generally being by far the largest:
Headcount: What does it cost to employ the people who work on the community program each year? What are the costs associated with employment, outside of salary (i.e., costs of benefits? Often finance teams can provide you with this information). Are there any contractors or part-time internal resources the community team relies on to produce value that need to be included?
Software contracts: How much does your team pay annually for the tools that support your community program? Consider not only your “home base” community platform, but event management tools, design and marketing tools, and anything else your team needs to succeed.
Event costs: If your team runs events, especially in-person ones, what are the costs to put those events on? Consider everything from space rental and decor to the cost for your staff to travel to and work the events.
Swag programs: Some community teams are responsible for swag programs. Consider the cost of design, production, and shipping.
Miscellany: Consider any additional miscellaneous team costs, such as team outings, dedicated marketing budgets, or anything else that applies to your team.
I’ve also outlined these in even more detail into a simple, responsive plug & play template that pits your profits (the returns outlined in the above sections) against your losses to help you really answer the ROI question once and for all.
You can either grab that à la carte or as part of my complete template library here:
Choosing which data is right for you
By now, we’ve spent a lot of time—10-pages-single-spaced-Google-Doc-time 😅 —exploring all your various options for proving community ROI. In other words, THIS IS A TON OF INFORMATION.
And like I said back at the beginning of this post, the whole point of creating a guide like this is to acknowledge that every community case, and every measurement case, is different. It’s unlikely—and would probably be pretty un-strategic—to try to implement every suggestion in this post, which are geared toward providing different options to use cases.
Community expert Rachael Silvano, who heads up community operations at Zapier, had this to say about how measurement varies depending on the use case:
“Zapier actually has two communities, which measure ROI in very different ways.
Our Support Community, naturally, tracks ROI via ticket deflection and free user support. That is a relatively well-documented (as far as ROI goes) process for measuring how many views a thread with an answer gets, and assuming the value of that question asked in a support queue.
The Zapier Wizards Guild, a Community of Practice based around AI and automation, measures entirely different ROI. In the Guild, we care much more about adoption of new products, and mastery in getting AI to go beyond a party trick and truly revolutionize your work. Of course, this measurement is trickier to document. Here we're looking at a few things, one is customer stories; who is using AI for great use cases that we can share across Zapier. Two is tracking members of the Zapier Wizards Guild and their behavior in Zapier; do they actually use more new products that Zapier users not in the Guild? What about an AI step in a Zap?”
In this example Rachael is showing how cohort comparison is more suited to one use case versus ticket deflection to another.
So, how do you choose which metrics from this menu to track for your use case?
Here are my tips to evaluate that question:
Choose the metrics your company leadership is most focussed on. One of the most impactful ways I’ve found to determine what to focus on in my community work is listening for what my company’s executive leadership team talks about in all-hands meetings, and then trying to create community programs geared around impacting those metrics. This is a powerful tactic because it allows you to ensure that what you’re working on is already valued, and that you won’t have to do as much storytelling to build buy-in for your chosen form of measurement. Not doing this is ultimately why so many community builders fail trying to tell community-specific stories: executives don’t necessarily already understand community engagement metrics and their significance, but they probably do already understand churn reduction.
Choose the metrics most aligned with your community use case. If you’ve read this blog post closely, you’ve probably noticed that some of these forms of measurement are more suited to certain use cases than others. Let your use case dictate what you focus on measuring, or, even stronger, let the use case you design be dictated by the results you want to be able to report on. The connections are mostly logical. For example, the churn-reduction example is most suited to customer communities, the ticket deflection example is most suited to support-focussed communities.
Choose the metrics that are most possible to systematize. Another factor to consider when choosing which metrics you’ll focus on is what data you already have at hand. If you don’t already have a mature data ecosystem, it may be difficult to do sophisticated churn cohort comparisons, but still easy enough to track leads and conversions. Plan accordingly, your mileage may vary.
Model out your opportunities and choose the one with the biggest impact. Part of why I decided to create a plug-and-play template to accompany this blog post is that I wanted you to be able to experiment with how different forms of measurement might yield different impacts for you. Even if you don’t have any concrete numbers yet, my template is a way you can explore what’s possible for your specific community and begin to create targets for yourself, as it allows you to input certain targets and get dollar amount outputs. Doing this might help you figure out where your biggest and easiest opportunities for impact lie.
Programmatizing your data
Lastly, before closing out this post, I want to acknowledge that actually tracking this data on a sustainable basis is going to require more than plugging some numbers into a spreadsheet. Here are some elements to consider to systematize your data:
Choose tech based on data needs: If you get to the end of this post and are worried that none of these forms of measurement are available to you because of your community tech stack, don’t fret. Understanding your data needs is the first step to solving them, and may actually be a primary motivator for a platform migration or for introducing new tech into your stack. This challenge is a big piece of what motivated me to migrate a community off Facebook in my first community job. Once you understand your needs and use case, use those as a major evaluation factor in selecting technology. Aim to use tech that either natively addresses the data points you need or has easy data flow out of the platform itself.
Understand your company’s current data sources: This is often going to be the biggest variable from one company to the next. In order to make meaningful connections between community and business impact, you have to deeply understand how your organization tracks business impact regardless of community.
Work with your data team to ensure community metrics are up to standards: Many community builders are one-stop-shops, but that shouldn’t prevent you from tapping the resources available at your company to make sure you’re delivering the highest-value product possible. Set up a meeting with a data scientist at your company to help you understand what resources are available to create data connections and dashboards to support your needs in a sustainable way.
Link-track like a marketer: If you are relying on customer acquisition or expansion revenue to justify your use case, do not, I repeat, do not, use raw links in your community. Always track the links you share so that you can create a picture down the line of the role community plays in product awareness, engagement, and conversion. Even if you don't yet have the data infrastructure to analyze much about these links, track them anyways.
Want to take any of this a step further?
I hope this post has been helpful for you in thinking about proving the ROI of your community instance. And if you want a head start crunching these numbers, I highly recommend getting yourself my Community ROI Calculator Template, either a la carte or as part of my full template library.
I made this tool not only so that you can input numbers and have my formulas spit out a result (which I think is pretty cool), but so that you can use this calculator to model potential areas of opportunity and create targets for yourself. You can grab that below:
If you'd like even more support putting this into practice, or understanding these concepts, I'd love to work with you more deeply—some options below.
Strategic coaching: I work with clients every day as a strategic coach for online community projects—helping my clients understand the business impact and goals of their communities is one of the most common focuses of my work. If you would like to work together 1:1, you can learn more about how I work and get in touch here.
For folks who want to deepen their community work but don't have the budget to work with me 1:1, I also offer an online program called the On-Demand Coaching Core Bundle—it distills the most common topics I work with my 1:1 clients on into a self-paced format with videos, my most in-depth templates, coaching scenarios, and a client community with bi-weekly group coaching. All of that comes for a one-time purchase of $500 USD, less than the cost of 2 1:1 coaching sessions with me. You can learn more about that here or enroll now below:
You're also always welcome to shoot me an email at noeleflowers@gmail.com if you have questions, suggestions for what I should add to this or other blog posts, or anything else you'd like to discuss. I love hearing from readers—thanks again for being one!
This holiday season, Edison Jacket brings you the ultimate Christmas Sale, an exciting opportunity to elevate your wardrobe while enjoying incredible savings. With up to 50% off on a stunning range of premium jackets and coats, this festive sale is the perfect chance to grab iconic fashion pieces inspired by your favorite TV shows, beth's blue coat on yellowstone and celebrities. Whether you're shopping for yourself or searching for the ideal gift for a loved one, Edison Jacket ensures your style game stays on point during the most magical time of the year.