Algorithms with Machine Hearts and Machine Minds
The issue of bias built into AI decision making boils down to the absence of empathy in the algorithmic learning process. No one would rationally accuse a statistical process like a random forest or a neural network of lacking a heart. Unfortunately, there have been manifold examples of AI producing thoroughly heartless outcomes; be that in the context of race, sex and various other societal categories. This begs the question: is a society that passively accepts AI injustice worthy to claim to have anything other than a machine heart?
The issue is clothed in complex layers. Often these algorithms are not overtly making decisions based on these societal variables. For example, an algorithm that decides to grant credit to a man over a woman is not necessarily using sex as a factor in its decision-making process. Rather the discriminatory outcome is a manifestation of wider societal bias which is a feature of the data the algorithm uses – the training data. In this case, the well reported income disparity between the sexes is the decisive factor in credit being withheld from women.
This white paper argues that, while AI can undoubtedly be a force for good, a fatalistic acceptance of bias in machine learning will only lead to the reinforcement of egregious inequalities that have enduring historical roots. The time has come to stop hiding behind R-squareds and F statistics, there is need to engage with the wider implications for society of unfettered AI.
This white paper will briefly look at: the history of AI bias, why this bias occurs, why it is a societal problem and what we can do to minimise the occurrence of AI bias.
A History of AI Bias
One of the earliest instances of algorithmic bias occurred in the late 70s. Dr. Geoffrey Franglen was frustrated by the time-consuming nature of the admissions process for St. George’s Hospital Medical School. Dr Franglen’s solution was to write an algorithm that closely replicated the actions of human assessors. This was thought to mark a triumph in fairness since the element of human error would be eradicated and each applicant would have a dispassionate non-bias machine deciding their fate.
The outcome of the implementation of this algorithm diverted St. George’s far from the path of fairness. In the years following automation of admissions, staff became increasingly concerned about the lack of diversity among students. The UK Commission for Racial Equality launched an investigation which uncovered flagrant prejudice inherent in Franglen’s algorithm. The Commission found that candidates would be classified as either “Caucasian” or “Non-Caucasian” based on their name and birthplace. Having a non-European name would knock 15 points off an applicant’s score. This would potentially be compounded for females who were on average docked 3 points by virtue of their sex. Franglen had unwittingly created a racist and sexist monster.
Despite the widespread media attention of the St. George’s case, instances of this kind of bias continually reoccur. The Commission ruled that race and ethnicity were inadmissible as factors for future algorithms to consider, however there are innumerable examples of algorithms using variables that act as a proxy for things like ethnicity or sex which produce the same unfair outcomes.
Just recently, Co-founder of Apple Steve Wozniak commented how shocked he was to find out that he was offered a credit limit that was 10 times larger than his wife, despite sharing assets and accounts. Pro Publica, an investigative journalism outfit, found that software to predict future criminals had a heavy bias against African Americans. Last year a research paper from MIT illustrated how Amazon’s facial recognition software had a dramatically higher error rate for dark skinned women compared to light skinned males.
The history of AI is littered with examples of clear prejudice against the most marginalised groups in society. It is a far from enlightening that the discussion around transparent AI has been lying dormant over several decades.
Why is there Bias in AI Algorithms
The aforementioned instances are instructive as to the reasons why AI can produce such marked unequal treatment. Clearly in the case of St. George’s, the fault lies with the algorithm using features like name and birthplace which produced a clearly discriminatory outcome.
The failing of the Apple Card algorithms is more subtle. It is unlikely the algorithms were explicitly using sex as a variable in the decision making process, but since various studies have shown that the difference in median incomes between men and women is favourable for men, this being reflected in the training data meant men tend to be offered a higher credit limit.
Fair representation of ethnic minorities is a crucial ingredient missing from most algorithms’ training data. It is hardly surprising that Amazon’s AI software had a 0.8% error rate for light skinned men compared with a 20% error rate for dark skinned women, since the training data was 77% male and 83% white.
Until there is full transparency regarding the inner workings of algorithms and data collection which is weighted more accurately to reflect the general population, the statistical community is doomed to repeat the failures of the past.
Why AI bias is a societal problem
The St. George’s case of AI bias perfectly demonstrates the point that AI decisions invariably reflect the patterns embedded in society. In the years succeeding the introduction of Dr. Franglen’s algorithm, student applications were double tested comparing the results from the computer and what the human assessors’ decisions would have been. The algorithm was in agreement with the human selection panel 90-95% of the time. What the algorithm served to do was make human discrimination more systematic and scientific, ensuring that all individuals were subjected to inherent prejudicial biases. The fact this was done by a computer served to undermine any dissenting voices about the negative consequences of this admissions process, because how could you accuse a computer of containing human prejudice?
The sobering takeaway is that AI has untold potential to cement an inequitable status quo, since it can legitimise a discriminatory outcome, which would be unpalatable if the same result was generated from human actions. What is unacceptable in the form of a face, is deemed justified from a computer because there is a perception it is unscientific to argue against it.
A secondary and more fundamental problem with algorithms that produce a prejudicial result is the outcome becomes further woven into the fabric of society. There will be a negative reinforcement of the existing societal inequalities. If minority groups are discriminated against in financial markets, education and housing then there will be a persistent lack of opportunity presented to them.
There is a strong moral argument that the hallmark of a progressive society is the ability to nurture and develop the most disadvantaged groups. If not swayed by this, then there is also an argument on efficiency grounds. A society that holds back certain groups is clearly not efficiently utilising their resources. Research the by OECD found that the single biggest impact on economic growth is income inequality and the main propagation mechanism is the lack of educational opportunities afforded to low income groups. A carefree approach to the long-term impacts of AI decisions may be a threat to general prosperity.
Computers are rightly seen as an advancement on human capability and even a humanising force for good. The crucial caveat being that if the algorithms are fed data points collected from a historical record of discrimination against women, people of colour and other minority groups, invariably the computer will assume it is desirable to preproduce this reality. The bleak consequence is the legitimation and reinforcement of prevailing societal injustices.
What can be done?
Demanding greater transparency regarding the mechanics and composition of training data seem like an obvious place to start. Clearly the example of Amazon’s facial recognition error rate bias could be solved by a greater representation of both women and non-whites in the training data. Unfortunately, it will not always be a simple case of adjusting the training data. The majority of algorithms are trained on historical data which serve as a record of the existence of a societal bias. As with the Apple Card bias, it is impossible to make an adjustment to financial history data - which echoes the financial disparity between men and women – without outwrite data tampering.
If it is impossible to reweight the data, it follows the key to the solution is greater transparency in the workings of the algorithms. Marketing Metrix have long been advocates of ‘white box’ machine learning. This concept is in direct opposition to ‘black box’ unsupervised learning, which essentially lets algorithms loose on the available information and affords the statistician a far more limited role in deciding which input features algorithms can use to achieve their goal. The starting point is to ensure that there is an audit process whereby the variables that drive the algorithm’s decision making are not used in a way that can perpetuate bias.
Positive steps have been made towards better AI scrutiny. In 2016 Google, Facebook and Apple joined to form a partnership with the intention of increasing research into AI ethics. Companies like IBM have even designed bias detection algorithms to counteract this problem.
The plethora of recent examples of AI bias points to the conclusion that there is undeniably more work to be done in this area. The AI bias problem is the clearest indictment of a society which has hitherto allowed the growth of inequitable outcomes. Dr. Rumman Chowdhury, Accenture’s lead for responsible AI, summed up the predicament aptly “With societal bias, you can have perfect data and a perfect model, but we have an imperfect world”. The first step begins with the acceptance that privilege and power is a greater driving force in distributing resources than merit and inherent ability. It hardly strains credulity that an algorithm that selects the best people for a job selects the same type of people that have historically occupied that job. Therefore, while there is still a role for AI in society, our excitement must be tempered by foresight of its limitations and potential for regressive results.