Measure before you redesign

A lot of projects start with the same sentence: "We all know it's bad." It's usually true. Everybody does know. What nobody has is how bad, for whom, and where.

That gap matters more than it looks. Six months after the new version ships, someone will ask whether it worked. If nobody wrote down where things started, the honest answer is "we think so." You can't take credit for improving what you never counted.

So on any existing product, the first thing I do is take a baseline. Before anyone draws a screen.

What a baseline looks like

It doesn't have to be elaborate. It has to be specific, and it has to be repeatable. Depending on the product, it might be:

• a short survey with a score people can answer again later
• task success and time on task for the two or three jobs that matter most
• the support tickets or calls that keep coming back
• where people drop out of a form, from your analytics
• a pass against well-known usability principles, so "you can't tell if it saved" becomes a named, fixable problem instead of an opinion

The test is simple. Could someone run the same measure again in six months and compare? If not, it's an impression, not a baseline.

A bad number is a good start

At LISC, staff using a case-management platform across a national network were asked how likely they'd be to recommend it to other partner organizations. Thirty-nine answered. The result worked out to an NPS of −38.5. More than half were detractors. Thirty-six of the 39 named at least one challenge that got in their way.

That's not a flattering number. It's a useful one. It turned "the system is painful" into something a team could aim at and check against later.

LISC's own team carried out the build. I wasn't part of it, so I don't have the after-number. But the starting line is on paper, and whoever measures next will know exactly what they're comparing against. A negative score is a better thing to have at the start of a case study than a good one. It says what you walked into.

Aim at a number, not a feeling

On Spectrum's activation work, the number came from Charter's own call tracker: in September 2019, 23% of activation-related calls ended in a truck roll. A technician driving out to finish something the customer was supposed to be able to do alone.

That number shaped the work. The journey map didn't just show the happy path. It drew every failure branch: the confirmation email that doesn't arrive, the activation that fails, the call that ends with a technician on the schedule. Each branch was a phone call, and each call was a chance at a truck roll. Self-install became the primary activation path for new customers.

I'll be straight about the miss. I don't have the truck-roll rate from after the change. If it exists, it belongs right next to the 23%. Until I can show it, I don't claim it.

Say whose number it is

Measuring honestly also means being clear about what you measured and what you didn't.

Pacific Clinics has published outcomes for its behavioral-health access program, including a 538% increase in referral access and crisis response cut from 50–60 minutes to 15–30. I built the client-facing portal, mostly from a design partner's design with some of my own. Those numbers belong to the program, across the whole initiative. I didn't measure them, and I don't claim a share of them. Salesforce's own customer story cites a 300% increase in client referrals, over a different window. Both numbers get reported. Picking the bigger one would be easy, and it would be wrong.

The same goes for numbers I can't find. At Mattress Firm, the project ran NPS on the redesigned sales tablet app. I don't have the before-and-after pair. So it isn't in my case study, even though it might be the best number on the page.

What it costs, and what it buys

Less than you'd think. On a bigger project, a baseline is a day to a couple of weeks at the start. On a small one, it might be a short survey and an afternoon in your analytics.

What it buys you:

• A fix list in the right order. At LISC we ranked problems by how much time they cost people, not by how loudly they were complained about. You spend money where it hurts most.
• A real answer at the end. A month after launch, run the same measures again. Then you know which changes moved which number, and which didn't.
• Something to show your board or your funders. "We redesigned it" is a sentence. "Here's where we started, and here's where we are now" is evidence.

A design without a before-number is an opinion. It might be a good opinion. But the people using your product are usually tired, usually mid-task, and never reading the manual. They deserve more than a good opinion.

If that sounds like your situation, let's talk.