By Francesca Gomez
for the 2024 Risky Women Writing Competition, presented in partnership with Bovill Newgate
Follow one professional’s journey into AI Safety, highlighting the challenges faced and the skills acquired from a background in digital and operational risk management. This article explores the growing importance of AI Safety and Governance, the risks associated with advanced AI models, and how Governance, Risk, and Compliance (GRC) professionals can contribute to shaping responsible AI practices. Learn about resources for up-skilling and opportunities to engage in this vital field.
My story
Earlier this year I started thinking about how I could start working in AI Safety. I had worked a number of years in digital and operational risk management within Financial Services (American Express, HSBC, Tandem Bank, Fintechs) and almost two decades ago (eek!) I was awarded a BSc in Artificial Intelligence so I thought I would be fairly well qualified to work in the field.
A significant number of job rejections later, I realised was wrong.
However, that did not put me off and I have spent the last few months trying to fill in the gaps in my knowledge. The good news is that the field of AI Safety is still relatively new, there are high quality and low cost ways of up-skilling, and I believe techniques and experience from Governance, Risk and Compliance could add major value and shape future AI Safety approaches.
What is AI Safety?
You will be aware about the recent explosion in Artificial Intelligence capabilities and the release of Large Language Models (LLMs) and multi-modal generative AI models, such as Open AI’s ChatGPT, Google DeepMind’s Gemini and Meta’s Open Source Llama. You may even use these regularly or even have just tried them out (if not you should take a look – there is already a gender gap in the use of these).
While these have impressive capabilities, along with risks to manage today such as misuse and bias, AI Safety is focused on longer term, more catastrophic risks. The companies developing these models – OpenAI, Google, Meta, Anthropic – are on a self-proclaimed mission to develop general AI, an intelligence comparable to or surpassing human cognition. While this may sound impossible, these are being funded with billions of dollars and backed by some of the world’s most prominent tech leaders and investors.
If you are still sceptical, it’s worth bearing in mind that some of the world’s most prominent philanthropic organisations, such as Open Philanthropy have risks from Advanced Intelligence as a focus area for investment and the impact-focused non-profit 80,000 Hours has ranked risks from AI as the number one most 0pressing world problem.
Concerns stem from the fact that AI models with advanced capabilities may surpass human oversight, especially as even today’s models sometimes exhibit deceptive or sycophantic behaviour, prioritising pleasing responses over factual accuracy. Additionally, the internal decision-making processes of these models remain largely opaque, making it hard to fully explain how they arrive at their outputs. This lack of transparency complicates efforts to provide assurance that these systems will consistently work as intended.
AI Public Governance and Legislation
Although I have spent more time focusing on AI Safety so far, AI Governance is another rapidly growing area where I believe GRC professionals can bring useful expertise, particularly from regulatory compliance and internal governance.
AI Governance is concerned with addressing the risks of AI through standards and regulation, and foreign policy approaches to promoting AI safety internationally. As AI capabilities expand, governments are stepping in to assess what regulatory frameworks may be required to help ensure AI is developed and used responsibly. The European Union is leading with its AI Act, but countries like the US, UK, China, and Canada are also considering implementing regulations aimed at balancing innovation with safety, ethics, and accountability.
The gap between AI Safety and GRC
AI Safety is largely a technical field, with a focus on how to carry out research that makes models more aligned, explainable or robust to adversarial attacks or misuse. While work has been done which is more familiar to traditional risk management, such as the taxonomies to classify AI risks released by researchers at the MIT (MIT AI Risk Repository) and the Foundation Model Risk Management Standards published by UC Berkeley, roles within AI Safety at the organisations developing Frontier AI models tend to be technically focused, and often even require PhD level qualifications.
To date, Safety Teams in AI have drawn on methodologies from safety-critical industries like nuclear and aerospace, focusing on approaches such as Safety Cases. I believe there is a current gap in leveraging the insights and expertise of governance, risk management, and compliance professionals from other industries, such as Finance, in addressing the risks posed by AI.
Get involved
Based on my experience, I would recommend interested people working in GRC to consider the following paths:
- Read up on the risks. The MIT AI Risk Repository brings together contributions from research into taxonomies and a database to classify AI risks. The Foundation Model Risk Management Standards (UC Berkeley) are detailed guidelines encompassing recommendations from the NIST and ISO standards and and BlueDot provide a more accessible summary of AI risks.
- Complete a course. Bluedot, a non-profit organisation, run AI Safety Fundamentals and AI Governance courses throughout the year. I recently completed the AI Safety Fundamentals course and found it very useful. If you are a IAPP (International Association of Privacy Professionals) member, you could consider the certification for AI Governance Professionals (IAGP).
- Carry out independent research. There are grants and competition opportunities for original research in the area, and this can also be great way to learn new skills. For example Open Philanthropy have an open Request for Proposal (RFP) in this area.