Insights
User research is about more than just testing whether a design works well or not. Insights from research should help teams make strategic decisions, like, what we’re building and why, our technical approach, product-market fit and even which order to sequence and prioritize tasks.
Technical capabilities don’t guarantee we meet user needs
At the U.S. Department of Veterans Affairs, my team recently replatformed Ask VA to VA.gov. Ask VA lets Veterans and their support networks ask the VA a question online.
Currently, submitters have to choose a category, topic, and subtopic to determine which department should receive their question. Despite the usability and content improvements we made based on research, we know this categorization process is still difficult and time-consuming for Veterans and other submitters.
Last year, our VA Product Owner and data engineers started building a natural language processing (NLP) model to predict these categories based on someone’s question. The hypothesis: by pre-filling these selections and letting submitters review the predictions, we could reduce burden and improve the experience.
It’s an exciting idea with real potential. And there’s an opportunity for research to help shape how the technology is implemented.
Technical capabilities alone don’t guarantee we’ll meet user needs.
Research insights drive strategic decisions
I proposed a research study to test early concepts with Veterans, so we could understand how an NLP model should surface predictions to submitters. We tested different prototypes, for example, surfacing a single prediction versus a list of options.
We learned that people prefer choosing from a list of narrow options rather than the tool jumping to a conclusion immediately. And while people are good at identifying when a suggestion is wrong, it’s difficult to choose the correct option without further explanation.
These insights drove our strategic approach. Alongside data engineers and our UX designer, we developed a solution: show one or more options that the model is really confident about.
If the model is really confident in one option and not at all confident in the rest, we choose for the user. But, if the model is pretty confident in a few options, we let the user choose from this short-list. This balances providing choice (which people want) with reducing cognitive load (which is our goal) while avoiding suggestions the model isn’t confident about.
Our team could have made assumptions and invested months into developing a complete solution with this technology. But we would have been building something based on technical capabilities and assumptions. Instead, we took a step back and asked how technology should interact with people to solve their problems.
This type of strategic research should be happening early and often, when teams are exploring what to build, why, and how it should work. By bringing in research as a strategic partner from the start, product teams can make informed decisions guided by user needs rather than assumptions, while saving development time and resources. When research insights about user needs inform early decisions, teams build better products that actually work for people.