Soliciting participation from stakeholders is a crucial control before and after deploying an AI system. Incorporating stakeholder perspectives and feedback ensures that AI systems are aligned with user needs, expectations, and ethical standards. Stakeholders, broadly defined, should minimally include individuals affected by the system. Feedback can capture system performance, fairness, safety, and more, contributing to continuous improvement, transparency, and overall success of AI projects.
Implementing feedback mechanisms can be as simple as incorporating buttons in a UI to rate model output or report an error, or it can involve a more complex process where Stakeholders are involved in the design process.
Participatory Methods Framework
One major framework used for soliciting participation and feedback from stakeholders both pre- and post-deployment is the
Participatory Methods Framework.
The method categorizes stakeholder participation into four levels: Consultation, Contribution, Collaboration, and Co-design.
Consultation – External feedback mechanisms post-deployment.
Involves participation where input occurs outside the core AI development process.
- Online feedback portals for issue reporting: Platforms where stakeholders can submit feedback, suggestions, and concerns post-deployment. These may include mechanisms for alerting developers of potential system issues, such as errors or bias.
- Surveys and questionnaires to collect structured input: Distribute mechanisms by which to collect structured feedback from a broad range of stakeholders.
Contribution – Limited participation in specific development stages. Participation is time-limited to one stage of the AI development pipeline, where stakeholders complete necessary tasks such as data collection, labeling, or validation.
- Crowdsourcing for real-time user feedback.
- Stakeholders interact with the AI system and provide real-time feedback, helping to identify usability issues and areas for improvement.
Collaboration – Creating multiple opportunities for stakeholders to influence AI models and features throughout the development pipeline. Participatory practices with multiple touch points along the AI development pipeline, allowing stakeholders to contribute to model and feature shaping.
- Focus Groups: Organizing discussions where stakeholders share experiences and concerns and provide detailed feedback. to shape AI improvements.
- Advisory Boards & Committees: Establish groups comprising key experts or stakeholders to offer ongoing guidance and feedback throughout the AI project lifecycle.
Co-Design – Engaging stakeholders at multiple stages throughout the pipeline, discussing their needs, values, and priorities regarding the issue and technology.
- Co-Design Workshops: Interactive sessions where AI designers and end-users work together to refine system functionalities. Facilitate collaborative feedback and idea-generation exercises, leveraging the collective knowledge and perspectives of diverse stakeholders.
- Stakeholder Interviews: Conduct one-on-one interviews with key stakeholders to gather detailed feedback and insights to understand nuanced concerns and expectations with diverse perspectives.
