Panels at conferences usually suck. But earlier today I was on a panel at DeveloperWeek (where UserVoice is a Top Innovator award-winner) about using analytics to make a better product…and it was actually quite good! Aside from myself, it featured Chris Kelly from New Relic, Dan Kador of Keen IO, Hugh Reynolds of Swrve, Bill Magnuson of Appboy, moderator Geoff Domoracki, and myself. Here’s what came up.
Yes to data (right away)
In case you were somehow unconvinced, everyone on the panel agreed that data is incredibly powerful and we should all be collecting it. There was a definite sense of urgency about this. Start measuring as soon as you launch your app; it might only be a little data (though more is better) and you might not analyze it right away, but there’s no reason to wait.
Data is about humans
Don’t forget that your data is about real human activity. Don’t let the lure of 1% increases in activity or deep data analysis distract you from that. Bill described it this way: “Analytics are a band-aid because we can’t go watch every customer use our app.” [Tweet this] So use data to find human problems, and then make sure to follow up with actual user feedback. It’s easy to misinterpret data, so always pair it with customer feedback. (Not sure how to start? Check out UserVoice Feedback.)
Data only informs decisions
Despite being so pro-data, the group was careful to point out that data should only inform decisions, not make them for you. I harped on the fact that talking to users can often clarify data that you might have otherwise misinterpreted. Hugh recommended rolling out changes quickly to small groups of people to see if your assumptions are actually correct (his best suggestion: roll it out to your free customers, rather than crucial paying customers). If you’re scared of being controlled by customers or data, don’t forget that you can say no.
Analytics should be a regular part of your process
Analytics, the panel insisted, are not something you look at once a quarter or every board meeting. They should be part of your process (whether you’re using agile or waterfall methodologies) every cycle, otherwise you are missing opportunities. Chris said it best: “You don’t wait to change your oil until the ‘check engine’ light goes on…you change your oil regularly and get tune-ups to avoid that light.” [Tweet this] Another great observation came from Bill: “The most dreaded word that customers say is ‘still’. ‘They still haven’t fixed the login screen!’” [Tweet this] You can build all the great features you want, but if you’re not fixing the things that are continuing to bug your customers, you’re going to lose them.
Thanks to my fellow panelists and our moderator for a great panel!