Gender Bias in LLMs User Research Toolkit
- The toolkit
- # User Research Toolkit
- See the larger project from Digital Futures Lab and the findings from this research at:
- These cards were developed as a part of an exploration into possible tools to explore gender and language in LLM development and use.
- # Context
- The research aimed to uncover challenges and opportunities in deploying LLMs in diverse linguistic and cultural contexts, thereby contributing to the development of more inclusive and responsible AI.
- https://www.digitalfutureslab.in/projects/from-code-to-consequence-interrogating-gender-biases-in-llms-within-the-indian-context
- Our user group were low-resource users, non-English speakers, and had never interacted with an LLM chatbot from Mumbai, Bangalore and Goa. They varied across gender, profession, education level, age and digital literacy.
- # The Team
- https://cdn.kinopio.club/CEwdX2suIhhND76v11f46/DFL-logo-inverse.png
- https://cdn.kinopio.club/X51kXDFyJ0qs922ojevSk/PHOTO-2024-08-28-21-56-27-2-Background-Removed.png
- Research Deck Development Team: Meredith Stinger, Rimjhim Surana, Dhaval Kothari, Yuvraj Jha
- HOW TO USE THIS!
- 1. Understanding user contexts with Digital Use Profiles
- 1. Understanding user experiences with technology and chatbots: Unguided Prompting & Chatbot Interface “Skins” Testing
- 3. Gauging attitudes towards gender bias, and testing those against LLM applications: Predicting & Reflecting on Culturally-Specific LLM Interactions
- 2. Understanding prompt-generation and getting feedback from users on scenario-specific usage of LLM applications: Guided prompting with Life Journey Scenarios
- 4. Exploring futures of gender bias or its absence in localised LLM use cases : Speculative Cards
- 2. User interactions: Unguided prompting
- 4. Guided prompting with Life Journey Scenarios
- A diary study approach with users’ own devices could yield more naturalistic insights. High levels of hesitancy were observed among some participants, particularly those with lower digital literacy levels.
- 3. Usability & Accessibility: Chatbot Skins
- 5. Emergent cultural and social themes with culturally-specific tech interactions
- https://cdn.kinopio.club/OKurkalEGXcoHSTaxPNWS/1.-Expectations-versus-reality.png
- https://cdn.kinopio.club/Pa69KqAA6sstJLnIJJ8eR/2.-Expectations-versus-reality.png
- 6. User-led speculative foresight cards
- https://cdn.kinopio.club/dRpsL8GNjOu3Y-QW2HjtG/IMG-5031.png
Gender Bias in LLMs User Research Toolkit
- The toolkit
- # User Research Toolkit
- See the larger project from Digital Futures Lab and the findings from this research at:
- These cards were developed as a part of an exploration into possible tools to explore gender and language in LLM development and use.
- # Context
- The research aimed to uncover challenges and opportunities in deploying LLMs in diverse linguistic and cultural contexts, thereby contributing to the development of more inclusive and responsible AI.
- https://www.digitalfutureslab.in/projects/from-code-to-consequence-interrogating-gender-biases-in-llms-within-the-indian-context
- Our user group were low-resource users, non-English speakers, and had never interacted with an LLM chatbot from Mumbai, Bangalore and Goa. They varied across gender, profession, education level, age and digital literacy.
- # The Team
- https://cdn.kinopio.club/CEwdX2suIhhND76v11f46/DFL-logo-inverse.png
- https://cdn.kinopio.club/X51kXDFyJ0qs922ojevSk/PHOTO-2024-08-28-21-56-27-2-Background-Removed.png
- Research Deck Development Team: Meredith Stinger, Rimjhim Surana, Dhaval Kothari, Yuvraj Jha
- HOW TO USE THIS!
- 1. Understanding user contexts with Digital Use Profiles
- 1. Understanding user experiences with technology and chatbots: Unguided Prompting & Chatbot Interface “Skins” Testing
- 3. Gauging attitudes towards gender bias, and testing those against LLM applications: Predicting & Reflecting on Culturally-Specific LLM Interactions
- 2. Understanding prompt-generation and getting feedback from users on scenario-specific usage of LLM applications: Guided prompting with Life Journey Scenarios
- 4. Exploring futures of gender bias or its absence in localised LLM use cases : Speculative Cards
- 2. User interactions: Unguided prompting
- 4. Guided prompting with Life Journey Scenarios
- A diary study approach with users’ own devices could yield more naturalistic insights. High levels of hesitancy were observed among some participants, particularly those with lower digital literacy levels.
- 3. Usability & Accessibility: Chatbot Skins
- 5. Emergent cultural and social themes with culturally-specific tech interactions
- https://cdn.kinopio.club/OKurkalEGXcoHSTaxPNWS/1.-Expectations-versus-reality.png
- https://cdn.kinopio.club/Pa69KqAA6sstJLnIJJ8eR/2.-Expectations-versus-reality.png
- 6. User-led speculative foresight cards
- https://cdn.kinopio.club/dRpsL8GNjOu3Y-QW2HjtG/IMG-5031.png