Beyond the Algorithm: Building Inclusive, Equitable, and Just AI Products
Inspired by Navrina Singh, CEO of Credo.AI - Principles for Building Ethical, Effective, and User-Centric AI Products
In a previous article, I wrote about Navigating the gray areas of product management and how to do a deep dive into the ethical challenges of product management. As a continuation, this article is about what Product Managers should consider as they build an AI product.
In an interview, Navrina Singh, CEO of Credo AI has given some valuable tips for PMs to consider which are woven into the narrative below.
A deeper and diverse approach to research
Being customer-centric is key to product management. In a world of AI product management, customer research plays an even bigger role. Customer insights come from
Traditional customer research methods.
AI products produce significant amounts of data that are a great source of customer insights and customer behavior.
AI products might use different user experiences like conversational interfaces. User testing and customer feedback mechanism for such interfaces can be highly scalable to gain customer insights
Navrina emphasizes the importance of bringing in new voices from a variety of disciplines, including legal, policy, and business perspectives, into AI research and development to efficiently govern AI. Bringing in experts from outside of the technical field can help to identify potential ethical issues, and ensure that the product is designed and developed in a way that is consistent with legal and ethical principles.
Understand Intent vs Use
Products are not always used how their creators intended them to be used. Customer research focuses a lot on understanding intent. But understanding use is also critical to ensuring products minimize harm and can be improved. Understanding intent and use and the gap between them is critical to AI product management to ensure we are assessing all use cases to minimize issues like bias. Understanding intent and use is possible through
Customer research through direct customer interaction
Analyzing user behavior from data
Monitoring customer feedback through customer service, customer reviews on app store, etc.,
Data analytics can also indicate a lot about how products are being used, providing insights into customer usage patterns and trends.
Navrina explains that building an AI product that is fit for purpose requires a deep understanding of the problem that the product is intended to solve.
Fit For Purpose
Just like any product, AI products should be designed and developed to meet the specific needs and requirements of the intended user. Navrina explains that building an AI product that is fit for purpose requires a deep understanding of the problem that the product is intended to solve. Product managers must work closely with customers to identify the key pain points and requirements. Data collection, assessment, training, and testing models are key practices that will help us build AI products that are fit for customer purposes.
Minimize Harm As A Key Design Principle
Minimizing harm is key to AI product management even more than traditional product management because
AI models can be difficult to understand and their decision-making processes may not always be transparent. Hence, it is important for AI product managers to be transparent about how their products work and how they make decisions. It is critical to have models that are explainable.
They are complex and can have unintended consequences. The risk is amplified because we can’t predict these consequences.
AI models are only as unbiased as the data used to train them. If the training data is biased, the AI product can perpetuate those biases. This can lead to harm by excluding certain groups or reinforcing harmful stereotypes. As a result, it is important for AI product managers to be aware of these biases and take steps to mitigate them.
Understand Potential Impact
AI products can have a significant impact on people's lives, and it is essential to understand this impact before, during, and after developing the product. Building in practices like those below will help Product Managers continuously learn about the impact of their AI products
Conduct a risk assessment to identify potential negative impacts that the AI product may have on various stakeholders, such as users, customers, employees, and society at large. This assessment should consider potential harms such as bias, privacy violations, discrimination, or unintended consequences.
Analyze the potential benefits that the AI product may offer to its users, customers, or society at large. These benefits may include increased efficiency, improved accuracy, cost savings, or new opportunities.
Regularly review relevant regulations and ethical guidelines to ensure that the AI product is being developed and deployed in compliance with legal requirements and ethical standards.
Continuously testing with customers through qualitative interviews, surveys or quantitative methods to get feedback from potential users and understand how they perceive the product’s potential impact.
Foster A Culture Of Ethics
Product managers should foster a culture of ethics within the development team and the company as a whole. This includes promoting open communication and transparency, as well as encouraging team members to speak up if they identify potential ethical issues.
Design for User Privacy
User data should be handled responsibly and transparently. Companies should have clear policies and processes in place for collecting, storing, and using user data.
Continuously Monitor And Evaluate
Monitoring product performance is required for regular products. For AI products, monitoring plays an even bigger and more critical role. AI products should be monitored and evaluated regularly to ensure they continue to operate ethically and effectively. This includes monitoring for potential biases and unintended consequences. Monitoring and assessing AI product deployment to identify any intended consequences or negative impacts that weren’t identified during product development is extremely critical.
Overall, product managers must approach AI product development with a strong commitment to ethical principles and a clear understanding of the potential risks and benefits. Following a few of these guidelines will help companies create AI products that are effective, trustworthy, and ethically responsible.
Follow Navrina Singh on LinkedIn
To stay updated on the latest trends and developments in responsible AI subscribe to Credo AI's newsletter.
Share this article with others who might benefit from this
Subscribe to receive future posts directly in your inbox
Subscribed
Like or comment below to share other helpful resources