Many emerging AI companies strive to ensure their technologies benefit humanity. However, the rapid growth of AI applications presents new ethical challenges. As AI technologies are increasingly applied to the Agile development ecosystem—impacting everything from defining requirements to completing code and conducting automated testing—agile teams must be vigilant against the unintentional embedding of biases. Large Language Models (LLMs) offer remarkable capabilities; however, their training on vast datasets from the internet can reflect and amplify societal biases. This may negatively affect end-user satisfaction, user interface interactions, and team dynamics.

This hands-on workshop will arm Agile practitioners with essential strategies to audit their AI-enhanced toolchains for hidden biases. Participants will engage in practical, interactive exercises to:

• Diagnose points within their Agile workflows where AI tools are employed.
• Understand the origins and pathways of AI bias.
• Utilize state-of-the-art tools to uncover and address these biases.
• Foster productive cross-disciplinary dialogues that emphasize ethical product development.

Target Audience:

This workshop is tailored for Scrum Masters, Agile Coaches, Product Owners, Technical Leads, and Development Team Members who are committed to ethical AI integration within their practices.

Participants will leave with practical templates, checklists, and techniques they can immediately apply to make their Agile practices more bias-aware and responsible.

Learning Outcomes

Participants will exit the workshop equipped with actionable templates, checklists, and strategies to enhance their Agile methodologies with a keen awareness of bias.

They will:

• Map AI Bias within Agile Toolchains: Identify and document potential bias sources within their toolchains.

• Comprehend Bias Sources and Mechanisms: Gain insight into various AI bias types, including data, algorithmic, and human-induced biases, enhancing their ability to mitigate these in real-world scenarios.

• Apply Tools for Bias Detection: Explore and apply AI fairness tools and frameworks like Fairlearn and AI Fairness 360, along with demonstrations of industry-leading solutions like Google's AI Fairness toolkit.

• Enhance Cross-Functional Collaboration: Understand the importance of collaboration between developers, data scientists, ethicists, and business leaders in ensuring that products are developed and deployed ethically, and are aligned with organizational values and ethics guidelines.

Time Breakdown for 90-minute workshop

Introduction and Context (10 minutes)
• Brief overview of AI bias and its impact on society
• Opening exercise: Participants will work in small groups to map their current agile toolchain

Understanding AI Bias Sources and Propagation (20 minutes)
• How biases in training data can propagate through AI systems
• Group exercise: Identify and discuss real-world examples of AI bias in various industries

Low-tech and high-tech techniques for Bias Detection (30 minutes)
• Tool demonstration: Using multiple GPTs to detect bais in AI responses
• Tool demonstration of Open-source libraries: e.g., Fairlearn, AI Fairness 360
• Tool demonstration of Commercial tools: e.g., Google's AI Fairness, Microsoft's Fairness, Accountability, and Transparency (FAT)

Cross-Functional Collaboration for Ethical Product Development (20 minutes)
• Discussion on the importance of cross-functional collaboration when using AI-driven toolchains for product development. This outcome emphasizes the need for a holistic approach involving legal experts, data scientists, ethicists, and business leaders.
• Closing exercise: With knowledge gained, participants will work in small groups to revise their agile toolchain map

Q&A (10 minutes)
• Additional resources and next steps

Presentation History

https://speakerdeck.com/billagileinnovator

Agile 2024 - Technical Practices for Detecting Bias in AI: Building Fair and Ethical Models

Beer City Code - Technical Practices for Detecting Bias in AI: Building Fair and Ethical Models

BITCON2024 - Becoming a Super Learner - AI in Education