Podcast S5E7 | Risk Analytics: Ximena Zambrano

TRANSCRIPT

Kimberley Cole 0:01
This is Risky Women Radio – a show to connect, celebrate and champion women in risk, regulation, and compliance. Sharing insight and perspectives from the most influential members of our global Risky women network on the latest developments, we need to think about, the challenges we should all talk more about and the innovation we are most excited about governance, risk, and compliance. Bringing together the hundreds of senior women professionals already connected with a new emerging group of leading women and men. I’m Kimberley Cole, your chief risky woman.

Shaheen Dil 0:25
Hello, and welcome to the next episode of our series on Risky Women Radio. My name is Shaheen Dil, and I am a Senior Managing Director at Protiviti, a global risk and advisory company. In this episode, I am thrilled to be here with Ximena Zambrano, a seasoned risk executive with broad experience in deploying advanced analytical frameworks and processes across all lines of business. Ximena is currently a Senior Vice President and Head of Model Validation for Wells Fargo’s corporate model risk group focusing on qualitative models. Previously, she led consumer analytics at Ally Financial and was the Chief Risk Officer for the international operation. And even earlier, she was a senior credit officer in charge of different businesses across Latin America for Citibank. So you can see she comes with an impressive set of experiences and education from MIT, Fuqua School of Business at Duke and the Universidad de los Andes, Bogota, Colombia. Her experience in applied analytics, model development and validation across large and midsize banks is mixed with a strong data science and analytics background, which positions her squarely at the intersection of technological advances in data science, and risk analytics, our topic for today. So welcome Ximena. And thank you for taking the time to speak with us. I know I introduced you already. But tell me one interesting and one fun fact about yourself before we start.

Ximena Zambrano 2:11
Great Shaheen. Thank you so much for the incredibly kind introduction. I’m thrilled to be here. I do want to clarify that my comments today are my own and do not necessarily reflect the views of Wells Fargo. So with that, let’s see something interesting about me is that maybe as much as I enjoy analytics, I have always been deeply in love with humanities. And I have a minor in Classical Studies specifically, you know, I ended up being a teaching assistant of Athenian Democracy in the fifth century, and classical Greek theater of all things.

Shaheen Dil 2:48
Cool!

Ximena Zambrano 2:49
Yeah, I still love it. I find it fascinating. Fun Facts. Well, you know, a lot of people think risk people are not a lot of fun, but maybe this is fun. I will look for any excuse to plan a party Shaheen. I love to host people in my house. I am a cake decorator, I will obsess over every little detail of planning a party from the place settings, the menus. For example, during COVID lockdown, I would put together these game nights with our friends where I would deliver game kits to their house along with a home made meal so that we could all play the same game and share a meal together. And it was a lot of fun.

Shaheen Dil 3:31
I want to come to one of your parties.

Ximena Zambrano 3:34
I love to have people over.

Shaheen Dil 3:37
All right. So tell me you’ve obviously had a very successful career in risk management. What is the single biggest risk that you’ve taken yourself in your career?

Ximena Zambrano 3:49
Hmmm, that’s a great question. Well, you know, I was once told by one of my great mentors, listen Ximena, if you want to become a really good risk manager, you got to go where there are risks to be managed. So there was once this opportunity and a choice I had between two roles. One was a promotion into a very well controlled business, very mature business, very evolved risk management practices. And they were well within my area of expertise. And there was this other role in an entirely new area. They had immense challenges, criticism and issues to overcome. And I knew that I was going to be stretched day one, and to add gravy on top people will actually tell me, why would you take that role that it’s like a third rail, you touch it and you die. I ended up taking the role and it was one of the most incredibly rewarding experiences of my career, the people I met the learning was exceptional. The team I was fortunate to build was wonderful. So I do not regret the decision at all.

Shaheen Dil 4:57
Awesome, great so, one more question. Is there a risky woman in particular who serves as an inspiration for you?

Ximena Zambrano 5:06
Oh, gosh, Shaheen so many women, obviously, all of my mentors. But I do think all trailblazing women are risky women, you know, women willing to push the boundaries and challenge the status quo. So I would pick probably Nancy Grace Roman. So she was the first female executive at NASA, at a time where women were not a very common sight in STEM fields, let alone in an executive field. She played such a key role in developing the astronomy program in NASA and putting the ground work for the Hubble telescope. And most of all, she was a fierce defender and proponent of women in science, which is something that I feel very strongly about. So you can tell I’m a big fan of hers, as she must have been so gutsy to face that environment, and to still excel in it.

Shaheen Dil 6:04
And of course, these are the real rocket scientists, and people sometimes called quants, rocket scientists.

Ximena Zambrano 6:10
Oh, yeah.

Shaheen Dil 6:11
My sister is an astrophysicist. She’s worked at NASA sometimes. She’s currently at the Harvard Astrophysical Observatory. So yeah, you’re right. These women are truly truly impressive.

Ximena Zambrano 6:22
Oh, my God, dream job!

Shaheen Dil 6:24
Dream jobs. Exactly.

Ximena Zambrano 6:26
I would love to meet your sister!

Shaheen Dil 6:28
So getting down to our topic for today, I think everybody is aware of the vast increases in data availability, in the advances in technology that has led to huge amounts of computing power available to every single person, when I first started in the analytic space, it would take a whole room in the CMU university to have the computing power, and I had to book 15 minutes or half hour of time in the CMU supercomputer to do the kind of work that I can now do on my iPhone.

Ximena Zambrano 7:06
Yeah, it’s crazy. I can relate to that.

Shaheen Dil 7:08
This is amazing changes. So given this huge impact, all of these dramatic changes, where do you see the key emerging risks in the next two to three years?

Ximena Zambrano 7:21
Great question. Great question. As you said, we are just at this immense crossroad of all the accumulation of data in every possible aspect of our lives in every one of our transactions, coupled with the ability to compute that data, but also coupled with the availability of algorithms that can interpret that data and turn it into usable applications. At the same time. It’s like a dream sandbox for people in data science and analytics. So as you said, gone are the days where you in order to create an advanced model, you had to write every line of code yourself. So this is great opportunity, exciting time to be in risk. I love it. Anyone in business intelligence analytics is loving it. But there are very significant risks. These are a few that come to mind Shaheen. So for example, these can come with issues in fairness, transparency, and explainability. What data are we using to train these algorithms? How are we using this data in terms of considering privacy and disparate impact and equity issues? That’s one that comes to mind. Another risk that comes to mind is well, because these are readily available, they can lead to maybe superfluous understanding of the use, and sometimes even abuse of these algorithms and techniques, right? Especially in cases where new data sources are used. Truly being able to understand the data and their relationships observed is important. So spurious correlations has always been a challenge in risk management and always been a thing that we are on the lookout. But with this extra availability of data, and the potential for opaqueness in these algorithms, there’s additional risks. And then finally, one that comes to mind is, then all of this demand for new applications comes with a demand in talent and people and skills. So how to attract and retain talent becomes very important, a prevalent risk for all firms.

Shaheen Dil 9:43
You’re raising a very important issue and we’ll come back to that later and spend a little bit more time in detail on the question of talent.

Kimberley Cole 9:53
This episode is brought to you by Protiviti. Protiviti is a global consulting firm with deep expertise in transformation, risk management and compliance, partner with Protiviti and face the future with confidence.

Shaheen Dil 10:12
In your experience, how have risk management practices evolved most dramatically over the past decade as a result of all these changes?

Ximena Zambrano 10:21
I think, I think about it, I don’t know, 10 years ago, in the late 2010s, the industry was still exploring even practical, usable applications of things like neural networks. Fast forward just a few years in the mid, you know, around 2015, and to now, and we have all sorts of applications of neural networks, from image processing, to fraud detection, to smart assistants, it’s all over the place. So with that, and while risk management has always been a field that leverages data and analytics extensively, given this rapid pace of change, there is an ever increasing focus and the need to explore and understand and solve problems leveraging data and technology. But there’s the obvious counter side to that, which is, then thinking about that the more maybe potential that these algorithms have to be opaque, or complex. This is where the more good old disciplined risk management can make a difference. So it is kind of a confluence of applying traditional, rigorous risk management principle to these environments, and traditional model risk management principles around some things sort of come to mind. One, understanding your data fully and how you’re using it, how you test your model, and understand what happens when data shifts, do you have a good monitoring mechanism? Do you understand your limitations and your risks, and you know how to mitigate them. So if you look at it, it’s kind of the traditional risk management cycle applied to this discipline. So it’s a little bit ironic that the practices and the principles still apply, even though there’s such an advanced and rapid change of the underlying technologies and techniques being used.

Shaheen Dil 12:30
That’s good to know. At the same time, I think one of the big fears that I hear a lot of conversation about is the thought that because of the use of these neural networks, and machine learning algorithms, and so on, and so forth. The algorithms themselves transmute and are transformed as they acquire and ingest new data. In other words, people are not changing the algorithms, the algorithms are evolving and changing themselves. How is this transformation affecting the output of these algorithms? And how can we control or test that the outputs are still resulting from the right questions and giving us the right answers?

Ximena Zambrano 13:16
I’m not going to go into these very intense dark paths where self learning algorithms can go into so I’m not going to go into the Matrix or Terminator type of environments. But let’s think of more close applications..

Shaheen Dil 13:31
So you’re saying the machines aren’t ready to take over yet?

Ximena Zambrano 13:33
Not just yet! But you know, there are several things to keep in mind. One, not all machine learning algorithms are self learning algorithms. That’s one thing to sort of understand. And in fact, very strong applications are actually not self learning algorithms, meaning the data is not necessarily updating in real time run of the implementation of one of these algorithms, but I think several things are important to keep in mind, and how do you gain a sense of comfort around these techniques? One of them is Do you understand the data that’s going into the model? And do you understand how to tune the model, so people have the tendency to spend a whole lot of time tuning what we call the hyper parameters into the model, when in reality, the variables selection and the factor selection itself can be a lot more impactful to the outcome. So sometimes really stopping and asking the questions. Can you truly explain the relationships that you’re seeing in the data? Do they make business intuitive sense? Can you explain those? Do you have the ability to explain aka is there explainability in these models is super important. The other one is, have you tested your model? And understand what happens when data shifts? Is your model robust to the shifts? Or if it isn’t, understand why no model is going to be perfect models are imperfect by the very definition of a model. But do you understand the imperfections? That is very important. Finally, as they get deployed Can you monitor them? Can you keep track really good track of what they’re producing as an output? And is it what you were expecting or not. And the other thing that I think is important Shaheen is that the amount of explainability required out of these algorithms is aligned to the type of application of these algorithms. So for example, the closer these are to customers into affecting individuals, the higher the bar should be for the amount of explainability and ability to track, monitor and understand that you should have around these types of algorithms.

Shaheen Dil 15:59
Yeah, yeah, great answer. Great answer. You’ve highlighted the fact that it’s not that these types of new models, these new algorithms using neural networks and machine learning tools, they are not really completely new creatures. They’re really evolutions of the kind of regression models that we’ve always been using. But even the traditional regression models sometimes had unintended consequences. In your opinion, are these more advanced predictive models, are there more unintended consequences? Or is it just a sort of a progression along the same continuum?

Ximena Zambrano 16:39
That’s a good question. I do think there can be more unintended consequences, Shaheen. And you’re right. If you look at strict definitions of you know, for example, a neural network, you do have sort of embedded mini regressions in them depending on things like the activation function, you pick, or those kinds of sort of more technical details. But ultimately, in terms of the unintended consequences, the added level of complexity coupled with how readily available they are, bring about several things in my view. One, we’ve always heard that there is the potential of misuse, as you said, of even a simple regression algorithm can be used, right. But we have seen recently all the way from intentional misuses and applications like the creation of the infamous fake news or deep fakes, to more unintentional consequences, like an application that is created with the best of intention, but has disparate impact and produces inequitable results to different social or ethnic groups. I worry about this aspect of unintended consequences in these applications, especially in industries that do not have evolved risk management disciplines like the financial services industry does. I mentioned it earlier, readily available algorithms. So there are these readily available bots, or applications that you could almost use out of the shelf or maybe offered by individual firms that might not have gone through the level of scrutiny and testing that they require to manage these risks that we were talking about earlier. And finally, Shaheen, there’s just obviously this excitement and hype around these algorithms. And I’m a huge fan, of course. But you got to ask yourself, if you have a traditional statistical model that provides a stable output and an accurate answer, you may not need to go with the cool, you know, machine learning technique, given the complexity, the difficulties for implementation, the need for very careful fine tuning the computational intensity, you may not need any of that if you’re already getting a pretty stable, accurate solution. And in many cases, these traditional methods still provide very reliable solutions.

Shaheen Dil 19:11
So one shouldn’t sort of jump too quickly into your advanced analytics or machine learning tools because in the vast majority of cases the traditional models and tools work just as well.

Ximena Zambrano 19:25
I think the exploration is absolutely required and needed and we should be doing it. But always asking ourselves the question if the extra risk is worth it, and the extra complexity is really required, there’s no need to over engineer everything, right?

Shaheen Dil 19:40
Good point. Good point. So how have all these changes impacted the drivers and skills required of people in risk management teams as a leader of a significant team yourself. Have you changed your hiring practices? Have you changed what you’re looking for in the people that you hire and train?

Ximena Zambrano 20:02
Ah ha, great question. So yes and no. So we’ve always had, you know, very obvious needs for people with analytical capabilities in risk management for decades, this is not new that we’ve been making data driven decisions and analytical driven decisions in risk fields for a long, long time, right? What has probably changed reasonably is that given the rapid pace of change of all of these three things coming together, that we talked about earlier, I think then the need to be resilient, and a lifelong learner becomes more important. So in a way, the skill to learn how to learn becomes even more important, or at least as important than, you know, having foundational technical and business acumen to begin with. So that skill becomes something very, very valuable.

Shaheen Dil 21:06
I’m comforted to hear that you’re saying that you don’t need to hire only PhD quants, and so on and so forth from CMU and MIT and whatnot. But that, in fact, traditional risk managers work just as well. And that everybody now needs to have some modicum of knowledge about these new tools and developments, right?

Ximena Zambrano 21:28
You’re spot on Shaheen, you’re spot on, I think it’s the mix of the skill sets that really makes a difference. While you’re truly open minded and willing to embrace advances and new techniques, while at the same time keeping the risk management rigor and discipline in how these get deployed.

Shaheen Dil 21:52
Is there more need or the same need as before for collaboration between those who are fully conversant with the business context of a problem, and the analytical skill set that are brought to bear upon some of these problems?

Ximena Zambrano 22:09
Oh, there is increased need of collaboration, Shaheen because the speed of deployment and the need for the speed of deployment is increasing. So the only way you can successfully do this is if you have these sort of multifaceted teams working together to solve these problems. So long gone, it’s never been a good practice. But now even less than ever, can you afford to have your quants in an ivory tower and the business team in another place? It has to be pretty well integrated.

Shaheen Dil 22:43
Well, thanks Ximena, that was a great answer. How have all these changes impacted the drivers and skills required in the kind of people that you hire in your position as a leader of a risk management team?

Ximena Zambrano 22:57
Uh huh. Yes, good point. So obviously, in risk management, we have been needing people with deep analytical skills for decades now, right, it’s not a new thing that we have been making data driven decisions and analytically driven decisions for a long time and risk management disciplines. What maybe changes now is that, given the pace of change, the ability to be resilient, and to be a lifelong learning becomes even more important. So in a way, Shaheen, the skill to learn how to learn is incredibly valuable. So maybe that is something that as a hiring manager you’re looking for more and more is sort of that ability to adapt and learn. And, you know, learn new techniques, because just the pace of change is so big to begin with. So it became almost as important as actual having the foundational training and business acumen and knowledge to begin with.

Shaheen Dil 24:04
So you’re saying you not only want to hire the D quants, and the PhDs from MIT and CMU, but you also want to sort of utilize people with deep business context knowledge, but you want to make sure that everyone in the organization has a basic understanding of these new tools and algorithms and how they work. So they’re, in a sense, citizen data scientists as opposed to being your advanced quantitative data scientists.

Ximena Zambrano 24:33
You’re spot on Shaheen. I think that mix is what drives really applicable good solutions. While on the one hand, you have the ability to dive deep into new techniques, new technologies, with the right depth of understanding and right level of use of these tools. But you also have a layer that makes sure that these are being applied with the right rigor in the right business context, and one without the other sort of falls apart. So I think the collaboration between these groups, and the integration between these groups is pretty important to come up with truly applicable solutions and models.

Shaheen Dil 25:19
That’s terrific, Ximena. Finally, you’ve obviously been a very successful woman leader yourself in a significant institution. Do you have any insights to share with younger women going into risk management, from your experiences as a senior woman leader that will help them advance their careers? I imagine there are quite a few of them on this audience.

Ximena Zambrano 25:42
Well, let me think what could have been something that I wish I could have heard earlier in my career. So couple of things. One, maybe don’t try to be a man, be who you are. For a while, when I started my career, I thought that I needed to maybe conform to like a playbook or a set of expected behaviors. And only when I started embracing sort of my particular passions, my interests, maybe my own style, I started feeling a lot more comfortable, a lot more confident, and a lot more able ultimately to execute and deliver. So that is something that I would think it’s important to keep in mind. And that I’m encouraged to see that it’s a little bit easier to do these days than maybe Shaheen when you and I started our careers, all those years ago, where, you know, maybe the expectations were a little bit different. So I think that’s important.

Shaheen Dil 26:45
I still remember having to wear dress suits every day with little bow ties.

Ximena Zambrano 26:50
Oh, yeah, absolutely. Absolutely. And, you know, we can tell stories of the past, it probably needs a couple of drinks for that though!

Shaheen Dil 26:58
Exactly, exactly.

Ximena Zambrano 26:59
The other thing I can think about is be supportive and relentlessly supportive of other women, bring people along with you in your career, not only because it’s obviously the right thing to do, but because it’s such a rewarding experience, to see people who you’ve worked with, grow and rise into their own confidence and skills and ability and see them develop their own careers. So I think that is super important to continue to help drive the change that we want to see in women’s representation. And then the final one, I talked about it a little bit earlier in terms of the skills that I think are important, but I feel passionately about this one. And it’s be a lifelong learner. Find the joy in continuing to learn about whatever floats your boat. But continue to learn, don’t fall into temptation of thinking that you’re done. First, because it can get very boring if you do that. And second, because it just opens up so many doors and so many experiences and so many opportunities. So I think that is a pretty important one.

Shaheen Dil 28:11
Thank you so much, Ximena. This has been fascinating. Now that brings another episode of Risky Women radio to an end. But I think the conversations and thought processes that this discussion has triggered will reverberate with me for a long time. So thank you again, Ximena, for joining us today, and for your thoughts and valuable insights.

Ximena Zambrano 28:34
Thank you so much Shaheen, I really enjoyed our conversation. And thank you so much for having me today.

Shaheen Dil 28:41
Thank you.

Kimberley Cole 28:48
Thank you for listening to this exciting episode of Risky Women Radio, to connect, champion and celebrate women in risk, regulation and compliance. I’m Kimberley Cole, based in Hong Kong. For more information on the Risky Women global network, head to our website, and the episode notes and please be part of the ongoing conversation by subscribing to this podcast, connecting with us @RiskyWomen on Twitter, or even reaching out to me directly by email.