AI is rapidly moving from novelty to infrastructure. We are no longer asking AI only to recommend a restaurant, summarize a document or rewrite an email. We are putting it into experiences involving money, health, employment, education and identity, where a confusing, incorrect or overconfident response can have significant consequences. That changes the job of UX research.
Drawing from my experience researching an AI-powered financial tool within a financial institution, this talk explores what happens when traditional UX research practices collide with probabilistic technology in a sensitive domain. With conventional products, researchers often evaluate whether people can complete a task, understand an interface or find what they need. With AI, that’s only part of the story. An experience can be usable and still be wrong. It can sound confident and still be misunderstood. It can respond while eroding trust.
This talk examines the questions researchers need to ask about trust, uncertainty, expectations, comprehension, perceived authority, failure and human judgment. It explores the distinction between whether people can use an AI experience and whether they understand when they should trust it, question it or ignore it.
AI research also presents another challenge: the product you’re studying doesn’t always behave the same way twice. Traditional usability research assumes reasonably predictable system behavior. Generative AI complicates that assumption. Researchers need to study not only interfaces and workflows, but the range of responses and behaviors the system can produce, including uncomfortable edge cases.
Most importantly, this isn’t about creating perfectly trustworthy AI. It’s about designing and researching AI that helps people maintain their own judgment.
Using lessons from researching AI in financial services, I’ll introduce a practical framework for researching AI in sensitive and high-stakes contexts: what to test, where conventional methods fall short, how to investigate failure and how to distinguish between an AI experience that merely works and one that demands people’s confidence.

Talk outline
1. When “usable” is no longer good enough
• The fundamental shift from conventional UX to AI UX
2. What working on an AI financial experience changed for me
• Use my internal AI tool as the case study rather than the entire talk. The realities of researching AI around
3. The new research object: the user’s mental model of the AI
• Researchers traditionally study people’s mental models of products. AI adds another dimension: people construct a mental model of the intelligence behind the product.
4. Trust is the wrong question
• Rather than simply asking: “Do people trust the AI?”
• Researchers should investigate things like: transparency, agency, calibration and Competence
5. Research the failure
• What happens when there is not just the happy path
6. A practical framework:
• USE
• UNDERSTAND
• TRUST
• ACT
7. What AI changes about the researcher’s job
• AI researchers increasingly need to think beyond screens and flows.

Four Key Takeaways
• Understanding how an AI experience can be easy to use while still producing misunderstanding, inappropriate trust or poor decisions.
• To make sure that AI experiences are calibrated to trust, where people’s confidence appropriately reflects what the AI can and cannot reliably do.
• What research reveals in high-stakes experiences so that technologists do not make mistakes.
• A research study cover more than interactions with an interface. This talk discusses the importance the of relationship among system behavior, human interpretation and the consequences of acting on AI-generated information.

Jen’s qualifications
As a Principal Experience Researcher, Jen has been conducting user research with an AI-powered, personally-catered, financial experience within a person’s banking ecosystem. This gives her direct experience investigating how people understand, use and respond to AI in the sensitive context of personal finances.
Her perspective combines deep qualitative research experience, mixed methods analysis, with hands-on work examining emerging AI experiences. Her research in the AI space, allows her to move beyond theoretical discussions of “responsible AI” and talk about what researchers actually encounter when real people interact with AI and their decisions around issues that really matter.