How to increase the value of a product — What’s old is new again
This isn’t a story about a product release or a new product feature. It’s a story about providing additional value to your customer long after you’ve initially gone to market.
Data Overload
You could say that the world is awash in data. An even more accurate statement is that the product world is awash in data.
Being a product leader these days mean you need to be able to gather, analyze and digest data every day. It’s a good thing all that data is all organized, easy to extract, simple to collate and even easier to interpret. Oh, you mean your data isn’t? 🤣
As a product manager, your job is to make sense of the information at your disposal. And these days, there are many (too many?) tools a product manager has access to — product usage, product analytics, feature adoption, user behaviours, and cohort segmentation. And that’s just the beginning.
However, none of these data sets tells you what you should do for a particular feature or how you should make a business decision.
Let me tell you how we used data to make those decisions for a new feature we released 6 months ago.
Initial Launch
In June 2021 we released a new native eSignature solution for PracticePanther, a legal practice management solution in the Paradigm family.
We had done all our research on the customer ask, competitor solutions, completed the build vs. buy vs. partner analysis and had gone through all the analysis (and aligned with engineering, sales and customer success) of what we need to release in v1, v2 and what to plan for the future. We had user feedback on expected usage, run cost models and projected adoption levels a year out.
After a successful EA (early access) program, we launched publicly with the full weight of the go-to-market teams behind us — blog posts, press releases, client testimonials, social media campaigns, webinars, help videos — you name it, we did it.
And… we blew away our projections — 10% of our entire user base enabled the feature within 48 hours of launch. Not 10% of users that were logged in and on the appropriate plan and were hit with an in-app message —not 10% with a bunch of qualifiers — literally 10% of our entire user base. And if you know anything about feature adoption in B2B SaaS companies you know that’s huge.
6 Months Later
Fast forward six months and we are looking to answer two different questions:
1. How has this new eSignature feature performed?
2. Do we need to build out additional features and revisit what we had pushed off after our fast follows?
To get some answers, we start gathering some data from various places.
Sales
Sales tells us this new feature is helping to close deals and is also helping to reduce discounts. Admittedly, these metrics are hard to measure concretely but anecdotally there is broad alignment that this new feature has been great and I’ll take those two wins!
Adoption
Next we turn to adoption data on who is enabling the feature. (FYI, we are using Mixpanel to track events in our app). We see that after the original launch month, things level off to a steady state — with one exception:
On the first or second of the month we see upticks in enablement of this new feature.
Is there some signal in the noise? Yes
Can we crystalize what this means? No
However, we have some smart people and we believe it’s a simple to-do list that our users are getting to. A new month starts, let’s tick somethings off my list. This new feature is one of them.
Usage
Then, to learn about specific usage, we run some queries directly from our databases:
Tell me all the firms that have enabled the feature, how many documents have they sent out for eSignature, how many users at each firm, how many days has it been since then enabled the feature — we get as much data as we can get in as quick a time as possible.
We also look at the economics and see what this has cost us — in aggregate but more importantly on a user basis as that’s how we’re pricing our overall subscriptions.
Here’s where those Excel and Google Sheet skills start paying off. We slice and dice the numbers — what’s the mean, the median, the standard deviation, let me normalize and annualize the data. What can I learn about usage rates that I can make decisions on?
We find that the average usage is below what our expected usage was based on the user interviews we did and the surveys we sent out. The feature has been out for half a year so this doesn’t seem like usage is still ramping up. We can make make assumptions and chalk it up to users overestimating their needs by a small, but still meaningful, amount — likely.
In the end we find that there’s a relatively normal distribution of usage. As expected, some are using the new feature pretty lightly, some are using it quite a bit — but most are right in the middle of that bell curve. (An interesting tidbit we find is that about a dozen users — a very small portion of the overall users — are really taking advantage of the feature — 3+ standard deviations higher than the mean. On a few of them, we are likely losing money but in the grand scheme of things, we are building products for tens of thousands of users and a few users like this will not be material).
Luck
So that was the data we analyzed — sales data, adoption data, usage data. Then luck stepped in.
Serendipitously, I was talking to our Head of CS about churn and it came up that for a specific client, even with this new eSignature feature, we weren’t able to save them.
That got me thinking about this other data set that I had access to but wasn’t part of the regular product tool kit — it wasn’t analytics embedded in the app, it wasn’t capturing the behaviour a user did in completing a workflow, it wasn’t logging time spent on a specific task. It was literally a list of customers that had churned. A report that is clearly useful but becomes even more useful when used in a unique way.
I took that list and compared it against our users that had enabled the feature and saw something interesting. When looking at the data, we found that if the user enables the feature, they are 8% less likely to churn. That is literally money that you don’t need to find or add to a quota. It’s something that doesn’t walk out the door and even more important, it’s likely that the user is happier and going to stay with you longer and thus has a higher LTV.
A feature that increases sales, is adopted quickly and reduces churn — that’s definitely a home run in my book!
What We Did
Great — so we have this data that says this new eSignature is a pretty great addition to our product. What do we do?
We’ve found that when users hear about it, they enable it.
We know that it helps us with increasing sales and with reduction in churn.
By examining the data we know our usage stats and our costs. So, we decide to make this an all you can eat feature.
We initially launched with a cap on usage. The cap was 2x the high end of our survey responses but it was still a cap. We had no plans to charge for overages but wanted to be sure that our projections were spot on.
Now, with 6 months of real data, we were able to be very confident in that.
We came up with a plan:
In the new year, let’s provide additional value to our users. Not via a new feature, not via an enhancement but by removing a constraint. Let’s ease the burden of staying within a monthly usage limit. You may not ever hit it, but by knowing it’s there, your behaviour is going to be affected.
Working with the engineering team to remove the limit in the app was the first move. Once that was done, we teamed with our go-to-market team, and picked an announcement date — remember that uptick in enablement that we saw around the first of the month — this is where it came into the picture.
We crafted a message, aligned our teams (particularly the messaging and questions that may come into the support team) and hit the go button.
What Happened
The result? Well, in the first 48 hours we saw enablements skyrocket to 24x the average of the previous few months. In fact, we’ve had roughly 40% the performance of our original launch.
Maybe it was removing that mental obstacle of the cap, maybe it was targeting the message on the day that users would be more receptive. Maybe it was that spidey-product-sense that people talk about.
In the end we didn’t release anything new but we provided a hell of a lot of value to our customers and we are seeing them respond.
We did that by weaving together the story that wasn’t present in any single data set but was split across 4 different places.
Building or enhancing a new feature is a team effort. For this feature, that team includes Scott Gartenberg, who managed building this new feature, Grace Transue, who designed a beautiful, easy and functional user experience and interface, and Zach Kyte, who randomly spoke to me about churn on that one day (and about many other random things on many other days).
