At LaunchPad Lab, we try to stay away from gut feelings to make product decisions. Using experience to make assumptions about how a user receives updates or feature ideas is dangerous to the product development process. Therefore, we focus on leveraging hypothesis-driven decision making and evaluate our ideas by leveraging data and metrics.
Product development is a continuous process, and in order to evaluate what should be done at each point, we need to have a framework to evaluate a product’s health. Understanding the different components for a product’s health means understanding what your key performance indicators (KPI) are for success. Different KPIs exist across the product lifecycle and different industries, but there are some fundamental metrics that every product manager should track. How well are you growing your user base? Once users log on, are they doing what they’re supposed to be doing? How well are you penetrating the market? How happy is your team? What’s your technical debt balance?
We’ve compiled a shortlist some metrics that can be used to help track product health. It’s not meant to be exhaustive, but it should serve as a good foundation to identify where to start.
User
Whether your product is for the general public, customer-facing, or internal, you should always track and measure the way they use your software. Here are a few key user-based metrics that will help measure your product health.
User Adoption
User adoption can be broken into several facets. Broadly, one could ask how well a user is adopting the product, but a more granular approach could measure feature adoption. To break it down, even more, you could compare user adoption between cohorts of new vs. old users. Taking another step, one could break down how adoption looks for a new feature and a new user vs. an already existing user.
User Retention
How effective is our product at keeping its users? How long do we keep them for? Where do we lose users? This is one of the best indicators for product-market fit. Users staying and coming back to your product show consistent engagement.
User Growth
How well does our product add users over time? What are some trends I can see in user growth? Is there seasonality, peak time, etc.
Net Promoter Score
Net Promoter Score (NPS) is a customer advocacy score. This metric is used to analyze how much support you have from customers. Typically, the scores exist on a 1-10 (1 being the lowest/ 10 being the highest) and are asked to customers who’ve used the product.
Stickiness
Most people get retention and stickiness confused. While retention is more about keeping a customer engaged, there are ways to manufacture retention. For example, you can send a push notification or an email to a user to come back to your product, and it would be a great retention strategy. Stickiness is more about getting users to come back to your product on their own. Nir Eyal, a behavioral designer who wrote the book Hooked, would argue you can design habit-forming features that will continuously “hook” the user into coming back to a product. A good example of a stickiness metric is open rate or engagement metrics like daily active users, time spent on site, and content creation rate. Like most metrics, these will depend on the industry, product type, and user base.
Business & Strategy
When thinking about the health of your product and business, it’s vital to track and measure the impact on your business. The following metrics will not only help you measure the success of your strategy but the health of your product as well.
MRR
Monthly Recurring Revenue helps track a product’s total revenue in a one month span. It’s generally calculated by taking the total monthly revenue of the month and any gained monthly revenue from new customers and subtracting the delta of downgrading customers and monthly churn. This is specifically helpful SaaS type businesses.
ARPU
Average Revenue Per User shows you how much revenue each user brings into the company. Depending on the need, ARPU can be calculated in different ways, per month, per cohort, per annum, etc. ARPU is critical to track as it helps you when you need to make decisions on pricing and marketing promotions. The standard way to calculate ARPU is dividing monthly recurring revenue by the total number of customers.
LTV
The Lifetime Value of a customer is what you use to identify how much profit you stand to gain from an individual customer. You’d take the ARPU and multiply it by the average amount of time a customer stays a customer. LTV helps teams make decisions like the type of users they should be going after or most effective / profitable channels.
CAC
Customer Cost of Acquisition is the cost it takes to acquire a customer. For example, Your ad spends on Facebook for the week brings you a hundred new customers. The budget for the ad spent divided by the 100 customers would get you CAC. While it could get a little more complex (adding salaries, etc.), its calculation depends on what level of the business is executing the analysis and for what purpose.
CAC and LTV are very powerful tools that can be used to identify whether a customer is worth the effort based on profitability. These two metrics can also be used to identify how promotions and price changes might affect your business model.
Product Team
Measuring the product team is just as important as measuring the user’s happiness and the business goals. You won’t continue to be successful if your team isn’t happy and productive. The following includes a few team-based metrics that you can measure to ensure the health of your product.
Team Velocity
Team Velocity is calculated by the total amount of points completed in a certain period of time. Typically, velocity is measured on a sprint to sprint basis, which depending on teams, lasts one to two weeks. Each user story is assigned points based on the level of effort. The number of points a team can complete in a given sprint become the team’s velocity. Ideally, a team would track their velocity over time and identify an average velocity. This helps to better schedule and predict delivery.
Budget/ Resourcing
Most product teams have a burn rate that translates to the optimal amount of resources staffed to the team. This includes engineers, designers, and specialty resources (data science, UX researcher, etc.). It’s best to identify the budget for teams and make sure the hours you’ve projected align with the forecasted velocity and the amount of work needed to be completed for the roadmap.
Technical Debt
Technical Debt is a term used in software development to describe the cost of taking on an easier solution or process knowing the longer and/or tougher solution will cost more in terms of time or money. Too much technical debt could lead to user-facing bugs, angry stakeholders, and less time to focus on delivering key features. Not all debt is created equally. Ideally, a product manager would work with their engineering team to identify tradeoffs on taking on technical debt.
Support/ Bugs
Tracking support is helpful so teams can understand overall product quality and how the team responds to customer challenges with the product. One metric to track is the volume of tickets and treat it similar to a funnel (how many get escalated?). Another useful metric to track is the number of days a support ticket is open. These all help to show a commitment to product quality.
Testing
Testing is another KPI that can help teams identify product quality. This will most likely require product manager and engineering collaboration to identify key areas for improvement. Some metrics that can be focused on include; automation percentage and/or the number of tests completed successfully.
Team Happiness
Team happiness is one of the most underrated health metrics. Outside of strategic KPIs and objectives, a product is only as good as how the team feels about working on it. Does the team feel their decisions are making an impact on the business? Do they feel connected to their work? Does the team feel they are making a difference for their customers?
Conclusion
There are a lot of other KPIs that teams should track to evaluate product health. It isn’t about tracking as many KPIs as possible but prioritizing which are key drivers for your team and optimizing toward the few select KPIs. The most important part is to learn from what you’ve measured and continue to build according to what you’ve learned. KPIs help us align our learnings with what we should build and how we’re doing as a product team.
