The future of customer research

One of the product leads who supported our early work wanted to see his product managers in the office no more than one day a week. He wanted them out spending time with customers.

When we started PoDojo in 2011, our focus was customer-centred agile product development. One of our biggest surprises was how few of the product managers attending our Product Dojos were conducting customer interviews themselves. For many, going out and speaking to customers was unfamiliar territory.

Helping them do that became a central part of our work. Discovery interviews, small experiments and usability tests became regular ingredients of our Dojos. We were enthusiastic about Lean Startup and Customer Development, and about helping teams learn what customers needed while they were still figuring out what to build.

We weren't user-researchers. My own understanding of research came from product management. I had read Steve Portigal, and once shared a fish platter with him at the Productized conference in Lisbon. But I had a fairly naive idea of how professional research worked inside organisations and what user-researchers considered sufficient evidence for a claim.

Two experiences that changed my view of research

In one project, we were working with a medical technology startup. Clinical trials were part of its work, so the team needed scientific expertise and evidence that could meet the standards of those trials. When we proposed our lean experiments, the researcher was very surprised. He didn't really take us seriously, and his comments on the statistical validity of our results left us feeling painfully naive.

We had been using the language of hypotheses, experiments and validation, drawing on Lean Startup and David Bland's Testing Business Ideas. But we were mixing two domains: clinical research needed scientifically defensible findings, while product discovery needed to help us find customer value and decide what to try next. The startup needed both, but we struggled to distinguish which questions we were answering and what kind of evidence each required. I realised that words like "validated" made our findings sound more certain than they were. I felt embarrassed, as if we had been showing off with scientific language without really knowing what we were talking about.

But I still believed that provisional findings could help us make better product decisions, provided we stayed honest about how little we knew and kept those decisions within the limits of our evidence.

In another project, our small team was developing a consumer product for the mass market. The researcher wanted evidence about the whole problem and was reluctant to decide without it. For me, that is what good research is about. But we needed to choose a small segment and start prototyping with limited evidence, knowing it might not be the best starting point.

It became a serious conflict. Looking back, we hadn't agreed on how much we needed to understand before making those first decisions, or how the research should fit the project's scope.

I still think about that project and what we could have done differently. How could the things we learn quickly help us decide where to investigate more deeply, and how could deeper research make our next experiments better?

My background is in product management and computational linguistics, and I'm still learning through working with AI and alongside professional user-researchers. These are a few thoughts from that experience, which I hope are useful to others exploring the same questions.

Making continuous discovery easier

Our current work with AI has brought me back to that question. At PoDojo, we are working with AI-moderated interviews, automated testing and ways for teams to ask questions of their existing research. Along with automated recruiting, these capabilities can reduce the effort needed to sustain the continuous discovery habits described by Teresa Torres.

I would not assume that this means teams will spend more time with customers or learn more from them. Which interviews we automate also depends on the empathy and understanding of our customers’ world we want to develop ourselves. Empathy can also grow from listening to recorded customer interviews, though direct conversations let us ask our own questions. For exploratory research, spending time with customers and interviewing them ourselves belongs in our toolkit. I would not outsource that learning to AI, any more than I would to a market research agency. Teams still need the time and willingness to engage with customers and act on what they learn.

Let's assume a team wants to add an idea to the roadmap. An agent working with a shared knowledge base could help them assess it against existing research and relevant product context before they commit to it. They could see what evidence supports the idea, where they are still making assumptions and which questions need further research. Depending on what they find, the next step might be to move forward, run a small test or investigate more deeply.

Bringing different domains and team contexts together in an LLM wiki goes beyond this article. I'll write about that separately.

Observations from these conversations and tests need to become part of the same body of knowledge as deeper research, with their different limits still visible. Audio and video diaries of colleagues trying the product themselves could add another source of observations to the wiki. We should be able to trace findings back to their sources and recognise when they no longer apply. Talking to the agent could help us see what our research supports, where evidence is missing or contradictory, and formulate questions for further research.

This is the connection I hope agentic systems can help us build. For this knowledge to compound, new evidence has to help us question, correct and deepen what we already know. Quick tests would have a better starting point, and deeper research would stay connected to the decisions teams are trying to make.

What user-researchers bring to the system

As we build these systems together, research and AI engineering are growing into a shared practice. Domain knowledge and research methods shape how we design, evaluate and improve them. That requires substantial technical expertise, developed through training and practice alongside experienced engineers. User-researchers also bring stakeholder knowledge and influencing skills. I think findings carry more weight when someone colleagues trust can explain and stand behind them. We recently put together a job profile for one of our customers and would love to hear your thoughts on it.

For me, evals are the heart of this work. User-researchers define what a well-supported answer looks like, what evidence it needs and when the system needs to acknowledge that it doesn't know enough. They turn those expectations into test cases based on the questions people actually ask.

These evals develop as we learn. If an insights wiki develops a strong bias, we investigate what is happening and add checks that make the problem visible. We then revise the synthesis, the selection of sources or the agent's instructions, and test whether those changes address the problem. This is an ongoing part of maintaining the system's quality. Alongside these evolving evals, we keep a stable set of questions to check whether answers improve as the knowledge grows.

Skills come next. They turn methodological knowledge into reusable instructions that guide how agents work with colleagues across the organisation. User-researchers use them to help colleagues clarify the question, add missing product context and decide whether the available evidence can answer it, or whether a different research method is needed. As we find weaknesses, we revise the skills and check the results through our evals.

The third part is the social side of research. Our current wiki project has shown me the potential of Slack threads where product managers, user-researchers and an insights bot work through questions together. The bot can synthesise details from existing research around the question at hand. People add context, discuss interpretations and judge what the evidence supports. Having someone colleagues trust explain the findings matters here, too. The aim is to draw new insights from existing research as different questions arise in the conversation. It is complex work in progress, and Slack gives us a promising place to develop it together.

User-researchers keep doing research themselves. Their investigations deepen their understanding of the domain, reveal things the system misses and help them question answers that sound convincing. What they learn informs new studies and feeds back into the evals, skills and corrections that keep the shared knowledge useful.

Before an observation enters the shared knowledge base, we check that it represents the source faithfully and keeps the context and limits of the evidence. Problems with automated meeting notes led us to build our own transcription pipeline, using the interview context to improve accuracy. Engineering builds and operates the technical infrastructure in collaboration with user-researchers, who take responsibility for its methodological quality and develop the criteria by which we judge it. For me, this is part of research's strategic role in helping organisations understand their customers and make informed decisions.

Learning by building

With coding agents such as Codex or Claude Code, everyone on a research team can quickly build small systems of their own. I think it is essential that everyone experiments with this themselves. Research is knowledge work, and working directly with agents, wikis and toolkits gives us a practical understanding of how to organise that work in a more agentic way. For me, the starting point is playful: building something, changing it and seeing what happens.

That does not mean everyone on the team takes responsibility for building and maintaining the shared team system. There is value in everyone gaining experience through their own experiments, while the team agrees on who develops and looks after the system they use together.

The second step is a pilot project. User-researchers work with AI Engineers to develop a system around real research questions and product decisions. This is where the evals, skills and shared conversations described above become part of the team's daily work. The aim is to build something useful for the research team and other stakeholders across the organisation, helping them use what they know about customers to make better product decisions. We also test whether the answers actually help teams make better decisions.

When we started PoDojo in 2011, we wanted product managers to spend more time with customers. Today, we can reduce much of the work needed to make that a regular part of product development. Whether teams use the opportunity still depends on how they choose to work.

Those uncomfortable conversations about our Lean experiments taught me to look more carefully at the claims we made. That is why I want user-researchers involved in building the systems we use to inform product decisions. Research expertise gives us a basis for questioning an answer, and learning to express that expertise through evals and skills makes it available to others as they work.