Diversity, AI and decision-making blind spots

Diversity, AI and decision-making blind spots


What links financial markets, artificial intelligence and counter-terrorism? Whilst there could be several links, one that might surprise you is: workplace diversity.

It may not be obvious that financial markets and counter-terrorism agencies have 'workplaces', never mind artificial intelligence, but the human diversity which lies behind each can be crucial to their success, and central to their failures.

Being a highly regulated industry, financial services have been feeling the pressure for some time to adopt more inclusive hiring, retention and promotion practices.

But the principled argument for diversity is being joined by an increasingly well-documented business case – a lack of diverse thinking can severely hinder a company, agency or machine's execution of its objectives.


The white, male CIA that failed to spot 9/11


A recent BBC analysis pieced together commentary suggesting that a racially diverse Central Intelligence Agency would have been more concerned about Osama Bin Laden in 2001. Specifically, Muslim counter-terrorism officers might have seen the cultural symbolism that imbued the Al-Qaeda leader's seemingly parochial style. In 2015, the CIA director John Brennan concluded in an internal report:


"the CIA simply must do more to develop the diverse and inclusive leadership environment that our values require and that our mission demands."

Whether there is any weight to the counterfactual is difficult to measure in such unique circumstances. But 'perspective blindness', the phenomenon which occurs when a group's composition is so consistent that it can't perceive it's own weaknesses, is not confined to high-stakes intelligence work – in fact, it can be measured in a range of fields.




Culture and conduct


In 2019, the Financial Conduct Authority (FCA) took serious action against Andrew Tinney, the Global COO of Barclays Wealth and Investment Management, for his attempts to suppress a report into his firm's allegedly risk-taking and anti-compliance culture. Culture and governance issues are being investigated increasingly by the regulator – there were 70 open investigations in March 2019 compared to 61 the year before.

FCA Executive Director Christopher Woolard recently set out the need for diversity in the asset management sector. He highlighted how workplace diversity leads to more scrutiny and therefore better decision-making, and noted that:


"When firms haven’t made diversity a priority, it's a good indication they may not value challenge and internal debate. And in those sorts of cultures, misconduct can flourish."

Mr Woolard's speech also referred to 'blind spots', and in particular those perpetuated by hiring 'people like us'. The FCA's role is to tackle instances of misconduct, but the clear link with diverse thinking gives it a strong mandate.


Artificial intelligence, a new challenge


Whilst the link between diversity and performance in humans is now obvious, the same cannot be said as clearly where machines make decisions. The current practice when designing artificial intelligence (AI) programmes is to 'teach' the software using whatever large datasets are available to cash-strapped entrepreneurs.

Those in the industry have observed that 'algorithmic bias' can occur when AI draws inferences from datasets such as online forums which pervade discriminatory ideas. Thankfully, entrepreneurs are able to reprogram their software before it goes to market in most instances, but biased AI has been shown to influence whether firms invest in female entrepreneurs.

Because of the probabilistic nature of AI, it is harder for a financial institution to spot biased reasoning in the coding which underpins AI decision-making. The best solution is to ensure neurodiversity in the dataset and coding which are used to build the software, and this is already an area of focus for the UK government.

In the near future, rules about the AI programmes that financial services and businesses more generally can use (and the datasets which can be used to build them) could be the next phase in championing legal diversity.


Related Articles:
Latest Articles:
About the author:

Rory is a solicitor in the Financial Regulation team at Pinsent Masons LLP, with personal academic interests in private, public and international law approaches to emerging financial technologies.