Performance That’s Measured against the Things That Count

It’s no longer enough to measure model accuracy. Ethics, explainability, and wider performance are key KPIs that must be considered as equally crucial priorities.

AI systems are automating processes across our lives, but this is nothing new. It began in the last century with a handful of applications such as hand-written postcode reading and spam filtering. When internet advertising and retail grew, companies like Amazon, Google, Microsoft, and Yahoo used AI-based recommendation engines in core products. In the last decade, the trend has expanded far beyond internet retail and a few other disparate applications: Adobe have put Reinforcement Learning (RL) at the core of their marketing tools, Apple use Machine Learning (ML) for credit loan applications, Tesla build cars with AI-based automatic driving functions, Netflix, Disney, Uber, Spotify, the list goes on. Today, everyone wants a piece of the action, and there is an army of Small-to-Medium-sized Enterprises (SMEs) waiting to cater to this demand.

All this activity has captured the attention of regulators and even governments. High-profile cases, such as Apple’s discriminating loan allocation algorithm, have galvanised efforts to regulate the data economy. Significant new legislation has been introduced, with GDPR being just one example local to the EU. Best practice guides for how to implement AI effectively and ethically within your industry and organisation have proliferated. The OECD and the EU Commission have collaborated on bringing these many guides together into a single framework for ethical and sustainable AI development based on excellence and trust.

AI certainly has much value to offer data-generating businesses. However, it is not a magical new technology that “just works out of the box”. It takes highly technical data and scientific expertise to properly manage the raw data, design AI systems on top of it, and deploy these systems within a business. It takes deep domain knowledge to ensure the AI systems being developed solve valuable problems for the business. And it takes an understanding of both these components to properly govern digital transformation within a business. That is a lot of expensive human expertise to hire or train.

In addition, the experts have differing views on how best to govern AI systems. The law is being continually tested and challenged both in the courts and in academia right now. Meanwhile, the field of ethical algorithm design is a young one. How, for example, should a company ensure an end-user has an actionable “Right to an Explanation” when implementing an automated decisioning system? It is neither sufficiently clear what this right constitutes nor is it sufficiently well understood how the requirement could be met algorithmically for best practice to dictate exactly what needs to be done. This lack of clarity on how to enact sufficient best practices leads to either a dysfunction of governance of AI system deployment or a stifling of the innovation that such deployment could bring.

The Mind Foundry Solution

As part of our mission to enable our users to take advantage of AI for the benefit of our society, we believe that the performance of AI must not be measured solely in accuracy, but also in fairness, privacy, safety and other ethical concerns.

Mind Foundry has developed an AI Governance Platform that brings multiple parties together across multiple disciplines and departments through an intuitive and easy-to-use interface. The system enables stakeholders from disparate backgrounds to record the whole process of governing an ML model. At each stage of the process, state-of-the-art technologies for meeting existing compliance requirements can be offered either within a solution or recommended out of the solution. Despite the major advances in AI, accountability will never be surrendered to a computer, so key individuals are recorded, and their responsibility for algorithms and decisions is tracked through the solution.

Capturing this information in a regulated manner allows us to provide a historical audit trail of decisions made in relation to ethics and performance and quantify how the system delivers on its requirements in these important fields. Many quantities, such as privacy, can be extraordinarily challenging to measure and verify through an automated process, so the involvement of those who built the system in this audit and the sign-off process is a vital component in delivering a comprehensive picture of the system performance.

An Ongoing Responsibility

The initial sign-off of AI applications for use in production is seen by many as a stumbling block but is, in fact, the tip of the iceberg. AI models are not simplistic constructs that may be deployed and forgotten about until they are removed from service. It is vital that the performance and adherence of AI to ethical standards are continually measured and verified against existing benchmarks. Failure to implement adequate monitoring frameworks leaves organisations open to the possibility of silent AI failure and the ensuing havoc that this can cause. Mind Foundry’s monitoring and audit framework provide the capability not only to sign off systems at the point of deployment but to rerun and continually monitor agreed quantities of interest, providing historical results and performance evidence in a centralised single record, acting as a system passport giving assurance of fitness to function.

Regulations on the Horizon

Regulations in the field of AI ethics have yet to catch up with the capabilities and concerns that abound in the field, but the movement is already evident in this. The introduction of GDPR in 2018 rapidly led to a rush by companies to retrospectively fix legacy systems that were in violation of the regulations, leading in many areas to degradation or reduction of availability to customers. It is vital that AI models being deployed today avoid falling prey to this same fallacy; regulation does not exist yet, but it is coming. By providing a robust, comprehensive audit trail of decisions and technologies employed to deliver solutions more sensitive to ethics, practitioners place themselves in a far more favourable position in the face of this quickly changing regulatory landscape.

From Education to Understanding

A collaborative workflow allows many stakeholders to understand the process behind model audit and benchmarking, make their opinions heard, and discover for themselves the underlying workings of the system. This setting has the added benefit of providing a natural space for new employees and other stakeholders to learn from their peers in the process of applying the compliance framework. Understandable and intuitive presentation of data and information is vital if those who are controlling the system are to be held accountable for their decisions and support of systems that have been through the process. We go even further, and provide educational course material — content for training the next generation of AI compliance officers and ethically conscious AI solution developers.

Responsible, by Design

AI has the potential to drastically impact our lives. When applied in areas like education, healthcare, and security, this impact is felt throughout society. When done responsibly, it can improve human outcomes for millions of people and transform the world for the better. But the reverse is also true. It is this potential for negative impact that makes it vital that the performance and adherence of AI to ethical standards be continually measured and verified against existing benchmarks.

Failure to implement adequate monitoring frameworks leaves organisations open to the possibility of silent AI failure and potentially catastrophic consequences. Only by empowering people to verify and take responsibility for the performance of the AI systems they implement can we be assured that appropriate safeguards are in place. Mind Foundry’s monitoring and audit framework provide the capability not only to sign off on systems at the point of deployment but to rerun and continually monitor agreed quantities of interest, providing historical results and performance evidence in a centralised single record, acting as a system passport giving assurance of fitness to function.

AI must be built responsibly by design and it can only truly be used ethically in our society if the appropriate values are built into it from the ground up. That means AI must be developed with considerations for ethics that are finding a voice in national policies and international frameworks. These often include a requirement that an ML model’s decisions preserve privacy, are non-discriminating, and can be explained to a variety of stakeholders. Many governments are now launching AI Strategies to try and combat these issues.

AI has gone beyond simple applications of customer churn, spam filters, and email optimisation. It’s time the standards AI systems are held to match the importance of their application.

Find out more about AI regulation around the world.