Back in November, UserVoice founder and CEO Richard White shared his keynote speech, “The (Near) Future of What We Do: Agile Support & (Customer) Data-Driven Product Roadmaps” at UserConf San Francisco.

The following is a full transcript:

“In my talk, generally what I like to do is kind of set the groundwork for why we’re here. Like why we have the PM track this year, why did we even start UserConf, what is it all about? Or as my grandma would say, “What is it all aboot?” There are Canadians here, right? They get that. If I really loved my grandma I’d sneak in a picture of the queen in this presentation, but whatever.

So the secret to what UserConf is all about is actually just in the tagline of the conference–and the tagline says, “A one day conference about keeping your customers,” and then “happy” is in parenthesis. Now I’m responsible for the “happy” in parenthesis, and my design guys and multiple other people are like, “You’re an idiot, what does that mean?”

The reason I put happy in parenthesis is because it’s not about customer satisfaction, it’s about loyalty. It’s about customer retention. This whole conference was built around customer retention. Now I know a lot of people think it’s become a conference about customer support, and I”ll get to why that is, but it’s really a conference about how to have amazing customer retention.

And why is amazing customer retention important? 

Well, because it’s really the holy grail of any business these days. It’s really hard to build a good business these days with really really shitty retention. It’s very very hard to build a good business. You can grow very quickly but you can’t sustain it unless you’re actually providing something people want and keeping them around, and that’s why this is what this conference is all about. And it’s not about talking about how important that is because we assume you already know that, your businesses are already built around the impact of customer retention. What we’re here to talk about is how to improve customer retention.

There are really two main things that help you improve customer retention:

Let’s talk about the first one. The first way to improve customer retention is something I call “Answer the Damn Email.” Because if you are not answering your support emails, if people are falling through the cracks and not hearing from you, that’s the biggest opportunity to improve your customer retention, to which you’re all like “Wow I’m really glad I came to UserConf, I’m learning a lot right now.”

But that’s kind of trivial, right? In the kind of businesses we all operate in, this is actually kind of a hard problem, right? Because we have a lot more users than traditional businesses and how do we scale when we’re growing very fast as a business? How do we make sure things don’t fall through the cracks? How do we make sure we answer all the damn emails? It’s not trivial. 

Thankfully, over the last couple years it’s gotten easier, I mean this is why we started UserConf. We can look to the heavens for inspiration in the form of this book. How many people own this book? Ok. You don’t even have to buy this ticket here if you own this book. This is Sarah’s book, The Customer Support Handbook, it is awesome. It is kind of a lot of things we’ve been talking about at UserConf for 2 years about how to scale support, how to build great documentation, how to hire the right people, distilled into 216 pages. So if you don’t have it, I highly recommend it.

It’s also what we at UserVoice have been focusing on for the past few years, too. We’ve been focusing a lot on building tools that help people get self service. Self service is a huge part of how we scale to be able to answer the damn email, and a lot of the talks over the last couple years have been about scaling support because this has been the biggest problem,  which is why most people think UserConf is a conference about customer support.

You can’t get to thing number two if you don’t do thing number one. If you’re not answering your support emails, if you’re not giving people great customer service, you’re not going to get to my thing number 2, which is going to be unfortunate because it’s awesome. And that’s also why we still have this whole track today just on customer support. So we’ve actually just, we’ve added the product manager track, but first we wanted to double down on both because before we were having a mix of product manager conversations and customer support talks and we think both of these are important. So that’s number 1: improving customer care. I say that’s the lowest hanging fruit to improving customer retention. If 50% of people that contact you and have a problem don’t hear back from you–that’s your best place to improve customer retention.

Let’s talk about the 2nd thing, because right now you’re like “Aww yiss, we answer that support, we so good at this, we’re gonna be on top of this,” and I’m here to warn you that I’m proud of you, really proud of you, but the second thing is gonna be a lot lot harder than the first one. What is the second thing? Well the second thing is, if we’re answering all of our emails, how many people are we actually talking to? 2%? 5%? On a monthly basis, right. (I hope you’re not talking to 50% of your users through support, cause that’s a whole other problem.)

What I’m saying is there’s a ceiling on how much you can do in support because there’s a ceiling on how many people contact you. You can raise that ceiling by breaking your product intentionally, but you don’t want to do that. So what’s the second way we can improve customer retention? It’s gonna be fairly obvious, but it’s much much harder and less straightforward and that’s by improving our product, to which you’re also saying like “Hey asshole we do that every day, we have an engineering team, we’ve got product managers, we improve our product every day it’s always getting better.”

 I’m gonna let you finish…but…I question whether you’re actually improving the right things. 

Because when I’ve talked to product managers, and when I look at the art of product management, which is really to me sequencing things in the right order, really understanding customer needs and making sure you’re prioritizing the right projects at the right time, I see a lot more art than science, and you’re probably like, “What?!” If Mark from UserVoice is watching this, this is for you, Mark.

So anyway, I know this is a little confusing and not everyone’s in product management, so let me give a parallel: the parallel I want to give is Marketing. Let’s talk about what marketing used to look like:

Back in the day, if you bought an ad, how do you know it was a good ad? How do you know it was the right ad to run? Was it effective? Well the way you know it’s effective is because some guy in some business meeting draws a line that goes up after you bought that ad and says “Oh shit, we should keep buying more of those ads, clearly it worked.” And this is called, correlation. Clearly not causation, and part of this is because back in the day, marketing was very Mad Men, right? The right campaign, the right ad was the right ad because the right people were convinced it was. You might have had some data, “look revenue went up, we bought this ad” but it’s not science, right? It’s art and it’s really the art of persuasion.

What is marketing like today? Marketing today is science. There’s still room for creativity in marketing for sure, but there’s a shit ton of science right now. There’s a lot of data, there’s a lot of analytics, you run an ad campaign right now, you know whether that worked right? You can follow the user all the way down every single click.

We’ve had this revolution in marketing where we’ve gone from this pure art, creative thing where a few people make all the decisions and just come to their own conclusions based on this very limited set of data to, “Wow! This is as much art as science, and I need to have a math degree to do this stuff.” [cta id=’2003897′]

What about product management? We’ll I’m sad to say that product management is kind of where marketing was 15 years ago. PMs are a lot like Don Draper, right? We don’t have enough data. They’re cherry-picking data here and there to make their claim for what the right feature is. So I would assert that the right roadmap in your company is the one which everyone’s been convinced is the right roadmap.

We don’t have enough data between all of these things so product managers are in that vacuum kind of acting like politicians, getting by on convincing you, “Yeah, this makes sense because of this.” We don’t have a dashboard that explicitly says “Yes, these are the right things to do,” and “Yes, we can track and see whether they worked.”

I’m not just talking out of my ass. We surveyed 300 product managers at the beginning of the year and I shared some preliminary results at UserConf Chicago but we dug a little deeper so I wanted to come back to it. One of the questions we asked product managers, “Who has the most influence  over the roadmap?” Anyone want to take a guess? Don’t all go at once, I’m going to make you ALL drink Fireball if you don’t start talking. Sales? That’s a good guess actually. The answer is actually: product managers are first, (kind of makes sense) executives, and then sales and then everyone else. Well this is interesting, I mean it’s not completely unsurprising. I don’t think anyone’s completely shocked by this, this is very similar to what I was saying:

Product Managers really only talk to a small group of people to get the data they need to make decisions. 

We went a step further than we did earlier this year and we started looking at, “Alright well those are 300 product managers from different size companies, with different levels of experience, different business models, different price points–” we looked at all these segments, and we found something really interesting when we dug into this and that is that none of that mattered. There was no change when we looked at any of those segments. So no matter what segment we looked at, we got the same results. 

Executives and product managers dominate the influence on what gets built and everyone else is kind of a distant second-third-fourth. 

Next thing we did is we looked at what feedback is valuable versus who has influence. So we asked, the second question was: “Whose feedback would you like to get?” and then we compared that to, “Well, who already has input?” and so as you can see there’s a group that’s already overrepresented–executives, sales, even engineering, slightly and you can see who’s under represented. So this was interesting too, and again we looked at these segments, we segmented this again and we said, well alright, let’s look at this again, and again….we found nothing.

Across all companies, no matter what stage of business, size of business, experience of the Product Manager, there’s a huge desire for PMs to get more feedback from users, support, and analytics.

And that’s why I say, “Yes, but are you improving the right things?” because this gap tells me you can’t possibly have the data to be making the most informed decisions you can about how you should be building your product. And so we dug a little further and we asked, them “Alright, well you know, why aren’t you getting feedback from these folks?” So we asked them “Do you have a formal process for how you get feedback from your sales and support team? 42% said, “Yes, we have a process.” We followed that up by asking “Is that process effective?” and about 1/3 of them said it was either effective or very effective, so when you add that up, only 14% of PMs have a process for getting feedback from sales and support.

We also found that it’s not just about feedback from support teams. That other group that people want to hear from–users–they don’t have a better process for them either.

One of the things that we’ve been working on for 6 years, and we were wrong for 5 of those years, because we assumed that if we gave product managers a lot more data they would be like “awesome!” It turns out we need, just like support, lots of process and workflow to make sense of that data. So that’s why we’re here today–that’s why we have a support track. That’s why we’re starting to add the PM track, that’s why we’re doubling down the PM track, we’re going to have it in the Spring.

We think you two folks: support and product, are going to be kind of the kings in customer retention for the future.

So, we want to get you guys in the same room on a regular basis. (Not too regular because this is exhausting to put on) But before I finish, I want to set our expectations even higher. I’d love if you came away from today and were like I had some great talks, I got some great ideas for how I can communicate with my product team better, or I got some ideas about how I can communicate and get data out of my support team better, but I want to set a real high bar:

I want us to be as data-driven about building our product roadmaps as we are about our marketing, and that’s gonna take a lot.

If we’re going to get to this point, we’re going to need a lot of data. Process and communication is the start, but we’re going to have to share a lot of data between departments, so I want to talk a little about something we’re experimenting with at UserVoice and how we’re sharing this data in an interesting way to kind of give you, “here’s what I think we need to get,” and hope we get your feedback on it later.

The way this process works, is we’re linking tickets from support to higher level concepts. I’ve talked to companies in the past that have done this, not a ton of them some of them called it agile support, some of them call it some other things, but that doesn’t matter for the time being. The idea is that when a support email comes in from a user, we can go ahead and label it, everyone does this–“oh, currently this is about search, so let’s set the product area about search”–and that’s okay.

We label all of our incoming tickets so that we can say “Oh jeez looks like we have a lot of  tickets coming in about search.” That’s useful, it’s not completely useless, but it doesn’t really. It’s not super actionable. It doesn’t tell us how many people, or where in search, and so we’ve started to take this a step further and we’ve said, alright well we want every single ticket to not just be labeled “This is a problem with search,” but we want it to be connected to an idea of how to improve to fix that problem, that could be missing documentation, that could be redesign this feature, add this feature etc. etc.

Every single ticket we are logging into a higher level concept. “This is a way to fix this” every single ticket should have a way to prevent it from ever happening again, and that’s what we want to link to.  So we do that. And what we’ve done is we’ve effectively connected the user that sent in the support ticket, to this idea. And that’s cool because this idea will live on well-beyond the usefulness of that ticket.

We can then close that ticket and say “cool, product knows about it, I need to go back to the other 5000 tickets in my queue because Rich told me I gotta answer the damn emails.” It’s also cool cause we can start adding more people to this idea–and start collecting a lot of them, and now we can kind of get a slightly more informed insight, and can say “Well there’s actually 50 customers who have contacted us about that idea, and they’re about 5% of our support queue.” Then we can go a step further, and as I’ve mentioned, we’re doing this with UserVoice data, but you can do this with any system you have.

So now we want to enrich this data with satisfaction data, right. Let’s look at all the people that are connected to this idea, what is their satisfaction level? Well that gives us kind of another layer of insight: There’s 50 customers that contacted us about this idea and when we look at them as a group, they really aren’t happy compared to our average folks. Ok that’s good to know. We’re still not quite there, right?

There’s more data, we’re making more informed decisions but that’s not everything we need.

So let’s add revenue data. The bigger person here kind of represents the more money they’re worth or whatever value equation you want to put on a user, could be engagement, could be referrals, could be spend. Now we can say, (and we’ll use spend, ok) “And this group represents $23K of our monthly revenue.” Wow that’s impressive. I’m starting to get a real sense of the impact of whether my engineering team followed up and built this as opposed to just “Hey here’s some things that are wrong in search, let’s start to fix them.”

So now I can start quantifying the impact of these various ideas. If I can quantify the impact of a singular idea, that means I can compare it to the impact of other ideas. So that’s kind of an interesting way to have support data–it’s great to go to the meeting and say “We’re seeing a lot of tickets coming in about search, here’s 10 ideas we have for this.” That’s great.

Product managers are very data-driven people too, they’re gonna want to see your primary source data. This is one way to do it.

And it sounds like it might be exhausting, but as I think a lot of you know if you do support, you get a lot of the same questions over and over. So if you’re actually linking them to the right concept, it’s not as much work because you’re not creating a bunch of new things all the time.

Support isn’t the only person with a say into what goes into product–they’re not the only people communicating with customers.

And I go back to another survey question we asked, and this one’s the one everyone really enjoyed last time: “The feedback I get is high quality and can be trusted” Does anyone wanna guess who’s last place? That’s right, salespeople–lie like a motherfucker. It’s not so much they lie, right–cause my sales guys can hear this. It’s that they just remember the last thing, like “Oh my god, we’ve gotta have this feature, I got this big deal in the pipeline and I lost two deals last quarter because of it.”

Every single salesperson has that one feature. That one–like the girl that got away. It’s like “if only i had that feature, I would rock my quota.” 

What if we extended the same system of connecting these concepts–connecting these users to tickets to connecting them on sales opportunities? Well then we could even go a step further and we could look at any given idea and say “Wow this idea not only has an impact on our existing customers, but $20K of our deals in the pipeline and $30K in lost deals last quarter were related to this concept or related to this idea.” Now we’re getting really interesting ways to compare different ideas and compare what we put onto our product roadmap–and so this is what I’m talking about in terms of “Let’s be as data driven about our product building as we are about our marketing.”

The intersection of art and science is really a beautiful thing If we make it happen, and I think right now, as I said before–a lot of what product management is is art, it’s politicking, it’s selling. And we’re not going to get to this idea of a fully data-driven product roadmaps next quarter, or by the next UserConf, so we need to take the first step.You need to listen to the rest of the talks this morning, you need to figure out how you can politic with the product managers for what you’re hearing from people in support.

I think this sort of vision puts us on the path to finding the holy grail; to finding that amazing customer retention, the kind of customer retention that just makes you a blow away awesome business. 

As I said before, the best thing about UserConf, other than all the wine backstage, is you guys. You guys are some of the smartest people in the industry. So I came here with this concept because we’ve been talking to people about it, it’s interesting and I’d love to hear what you think about it.

Is this something you guys are doing in your business?

What kind of data are you using to build informed roadmaps?”