Our minds, for better or worse, are wired a certain way. We hear someone’s problem and we immediately want to solve it, even before they’re done describing it. This doesn’t even sound like a bad thing does it?
The first version of a Thing is built. It’s purpose is to take a process that sucks and make it suck less. Its creators think about the Thing a LOT, the process it’s fitting into, what the Thing is going to do, how it’s going to be useful to its intended end-users. They work hard. But they don’t think nearly as much about how the Thing’s users are going to tell them whether it worked for them. Or where it kinda didn’t. Or what other stuff those users still have to do manually, that sucks as much. There’s probably an email alias. That’s fine right? Users will send emails there when they hit bugs, so…. they can totally use the same alias to tell us about any bigger problems too surely.
What they think about even less than that, is how the more empowered users on the ground might themselves answer the questions – “Who is using this thing? what parts of it are they using? Who is not using this tool at all? Can we see any evidence of what else they are using instead?”.
The Thing is deployed and soon reality has moved on a bit. The people who use the Thing are always the first to notice, if only because they start bending how they’re using it – maybe abandoning one part, maybe circumventing the Thing entirely sometimes. They probably think about emailing the alias, or maybe they do but it’s not really a “defect” so they don’t. If they know a person on the inside, maybe they’ll ask them about it next time they’re chatting.
at any rate, this doesn’t have an easy answer and remember they have a problem still. They have to solve that, and they can think of a workaround so they move forward. Thus in that workaround is born the “Shadow Process”. As time goes on, the people in the know start using more Shadow Process.
Deprived of its most advanced and powerful users and their feedback, the official process withers further. The group owning the tool despairs! They may not even be aware that most of the workload they think is being done, is actually being borne by the Shadow Process. They seize on the officially reported defects and the feedback, most of which inexplicably has come from the consensus of internal stakeholders (and much of which interestingly concerns adding additional use cases and features) and they announce a grand plan to do a Next Big Version of the Thing.
Change one thing, and the story changes. Don’t release the product first and think about its feedback mechanisms last (never). Instead build the tools and/or processes that the end-users can use to easily send in feedback of all levels – from bugs, to enhancement requests, to full-on “Come To Jesus” moments.
Pretty much as soon as you start doing that, you’ll realize that those hero users on the ground don’t actually have enough data to do this well.
“I don’t know, it works for me. I have some defects reported but I passed those on. I think the other 193 users are probably using it fine.”
So realizing that, you’ll start spending time and energy giving those hero users some kind of little tools and processes of their own. For example so they can see “188 out of 193 agents are using this thing” and ask “Well.. what the hell are the other 5 doing??”. Or so they can see “That page that was supposed to be the workhorse of this whole thing, literally only one person has ever visited it in the last week”.
The entire purpose of these second order tools, is to enable the local heroes to send in and/or summarize these collected facepalms back to the mothership. So because they’ve been enabled to do so, following through and sending the data back in, even if it’s a screenshot, will be a natural next step. And hey that screenshot is more valuable than any three of the feature suggestions from your Product Council.
The solution I think starts with admitting that you suck. That you haven’t been doing this at all. I suck too and I haven’t been doing nearly enough of this. But I’m resolving to fix this in the next releases of all our products (before first giving a way to see if the fix is working).