Comparing services to inform a product vision

A comparative analysis is a method of identifying similarities and differences between designs, products or services. It’s often used to identify best practices and make decisions.

I’m not a big fan of this method. Let me explain why.

Similar to personas, comparative analyses are often completed so that teams can check off a required step in the design process. You do the exercise, share your findings, and then, very often, the artifact collects dust.

Like any research or design method, a comparative analysis is only helpful if you’re clear about why you’re doing it. And sharing findings might be a step, but it’s not an outcome.

Defining outcomes that matter

At the right moment, a comparative analysis can be a useful artifact to provoke thought. It’s easier to look at something and say what you like or don’t like about it, than it is to come up with a product vision starting from scratch. This is particularly true when you bring together a group of people who have different ideas of what a service is, does or could do.

At the U.S. Department of Veterans Affairs, my team recently replatformed Ask VA to VA.gov. Ask VA lets Veterans and their support networks ask the VA a question online. 

Since launching, we’ve been having conversations about the future direction of Ask VA. We want to create a product vision in collaboration with our partners. And to do this, we need to articulate the things we like or don’t like about the current product and the direction we recommend.

I started by writing research goals. To understand how we could meet expectations and provide a more effective service, I planned to compare Ask VA to other contact services to learn:

• how other services provide help before asking someone to submit an inquiry
• how many fields are included on other service’s contact forms
• how other services ask for information that lets them categorize or route an inquiry
• how being signed in changes the experience of contacting an organization

I shared these goals with product leadership at VA to align on objectives before I started comparing services. 

Visualizing different approaches to Ask VA

I reviewed each website and took notes in Mural. This is the point where I often see people start to wrap up a comparative analysis. You analyze how a service does something the same or differently to your own service, but it’s left to the reader to contextualize what this all means for your own service going forward.

To engage my team in discussions about these different approaches, I took it a step further. For each website, I illustrated how Ask VA would work if it was designed like that particular contact experience. For example, if Ask VA worked like National Grid’s contact service, I described and visualized how it would work differently.

Building alignment on our future direction

Doing this exercise helped me better understand Ask VA and how these comparative services work. I found that these 8 services demonstrated 3 distinct types of contact experiences:

• Type 1: Submit a contact form for review
• Type 2: Find a contact page for an individual service
• Type 3: Answer questions and receive a recommendation

I shared which contact experience Ask VA is most like (Type 1) and which it could be more similar to (Type 3). And in addition to articulating how Ask VA would work differently if this was the case, I also shared why I think this shift would positively impact Veterans and the VA, by streamlining services and reducing burden on staff.

In a separate exercise, I also compared Ask VA to other support channels on VA.gov. This allowed us to review tools that use the same design language as Ask VA and articulate how Ask VA should differentiate from other channels on VA.gov.

These exercises gave our team and partners the shared language and understanding to articulate the future direction of Ask VA. The artifacts have helped us visualize how Ask VA should evolve to become a service that guides a Veteran towards an answer. And we’re more closely aligned on the next steps to get there.