Three Actions You Can Take To JumpStart The Journey:
- Visit the Responsible AI Developers Hub
- Bookmark the Responsible AI Developers Collection
- Complete the Responsible AI Dashboard Learn Module
What is Responsible AI?
If you've built AI applications or trained a machine learning model, you've probably heard the term responsible AI being used in combination with words like "evaluation", "fairness" or "explainability". So what do these mean? And why should you care about responsible AI? And most importantly, how can you skill up on this with self-guided learning resources?
By one definition Responsible AI is an approach to "assessing, developing and deploying AI systems in a safe, trustworthy and ethical manner". Rapid growth in Large Language Models (LLM) and Generative AI (apps) is accelerating adoption of AI in every aspect of our daily lives. And this is raising concerns around data privacy, and the ability of these systems to operate fairly and reliably when making decisions that have the potential to cause harm to individuals and society.
6 Principles Of Responsible AI
To address this, Microsoft identified six principles to guide AI development and usage. These are:
- Fairness | AI Systems should allocate opportunities or resources in a fair and unbiased manner, across all users.
- Reliability & Safety | Systems should work correctly and reliably, even under conditions or contexts it may not have been designed for.
- Privacy and Security | Systems should protect data and preserve user privacy in all operations.
- Inclusiveness | Systems should function in a manner that is inclusive to people of all abilities.
- Transparency | System should be understandable to its users so that it is not misused or misinterpreted.
- Accountability | Humans should be accountable for system operations by means of appropriate oversight.
The Responsible AI Developer Hub
How can we apply these principles in practice, as we build our machine learning models or generative AI applications? How can we integrate these practices into our developer workflows? To help developers answer these questions and more, we setup the Responsible AI Developer Hub - a site with training materials and resources for the self-guided learner.
The site currently has three workshops that help you learn responsible AI practices for model debugging, prompt engineering (and LLMOps) and content safety for generative AI experiences. Let's take a quick look at each.
Responsible AI Dashboard Workshop
Building a machine learning model for your application domain with a relevant data set? How can you ensure that your model follows responsible AI practices? Traditional model performance metrics (e.g. accuracy) provide aggregated results which are insufficient to pinpoint where biases or model errors exist that could negatively affect people. This hands-on lab teaches you how to debug your model using a number of tools (error analysis, model performance, data representation, interpretability etc.) to assess it for fairness, transparency, inclusiveness and more. You'll start by training a model with a given dataset, registering the trained model on Azure, then creating a customized Dashboard to help you debug this visually.
The Dashboard is a single pane of glass experience that lets you seamlessly integrate and use a variety of toolkit components from the same UI.
Azure Content Safety Workshop
The Dashboard can help you debug the underlying model for fairness, reliability etc. But ensuring responsible AI experiences with generative AI content poses additional challenges in making sure that the prompts and responses do not include offensive, harmful or inappropriate contnet in text and images.
With the Azure Content Safety tools, developers can now moderate text, image and multimodal content using out-of-the-box AI models with built-in blocklists that you can customize further. In this workshop, you'll learn to detect and flag text that is unsuitable for end users, block inappropriate images, and create applications that present a safe and friendly tone to your audience.
Prompt Flow Evaluation Workshop
Generative AI applications rely on prompt engineering to improve the quality and relevance of the responses generated by the underlying model. Prompt engineering it itself a tedious process requiring orchestration of multiple steps in a "prompt flow" before the final model prompt can be generated, and the result returned to the user.
To infuse responsible AI tools and practices in your LLMOps it is not enough to understand the best practices as identified in that article above. Instead, you will need to iteratively test and refine the process, evaluating the results each time until you get the desired outcome. Azure, with Prompt Flow, provides a comprehensive evaluation toolkit with sample flows that you can use (and customize) to evaluate metrics like groundedness, relevance etc.
In this workshop, you'll learn to create a chat application using an LLM with a RAG architecture - then use the QnA RAG Evaluation template to test the flow for relevant metrics.
Related Resources
Want to learn more about Responsible AI? Bookmark our Collection and watch for updates. You can also follow the authors on dev.to for news on upcoming events and content to help you skill up further.
Last but not least, check out the Responsible AI Toolbox suite from Microsoft Research, for an open-source version of the tools that you can use for a customized end-to-end responsible AI experience.
Build AI Apps. And Build Responsibly.🛡
Three Actions You Can Take To JumpStart The Journey:
Top comments (4)
Love the content. Will try to find time to go through the course. Thanks for taking time and putting the effort on this.
Couple of things that would add more value is
1) Maturity of CI pipeline (i.e., how can we automate and ensure data pipeline and ML versioning is managed Systemically) and a dashboard view of the same
2) Another interesting space would be the Carbon impact. its extension of Emission Impact, but more from the ML workload point of view.
Thanks for the suggestions @balagmadhu - both of those are really good ones!! Keep an eye out on this series for updates. Quick note for now:
We have work in progress on an end-to-end workshop that will walk you through the entire process of app development with an LLM Ops focus in mind. The prompt flow workshop is a good starting point but we have a more complex example that will also showcase some CLI/SDK options for code-first tooling.
The sustainability issue is one that we have had a green advocacy team look at in more detail. However, I think what you are asking about is the ability to potentially integrate that kind of feature/knowledge into your LLM Ops flow so you can track that alongside other things? Interesting space for sure and I hope to post more about this as we expand the series.
Thanks for the update @nitya . on the second point I was more thinking of the CO2 impact on the cloud compute in general. LLM is very specific. Looking forward for more updates. Love the work that you and team do..
Thank you!! ❤️
Ah - okay!! We have an Green Software Advocacy lead so I will ask him for more recommendations to share. But I know of a few resources:
Hope that helps