Building Fair, Unbiased, Transparent Decision Assistance Systems
A few ideas on Building Fair, Unbiased, Transparent Decision Assistance Systems
Published by Karthik Bacherao on Oct 12, 2022.
Building fair, unbiased and effective deep-learning based hiring systems is challenging; particularly those trained on large amounts of data. First, for a system to be effective, the machine-learning system needs to be trained not just on large amounts on data, but also on large amounts of quality data. Next, for the system to be unbiased, either the data needs to unbiased or the system should be capable of handling and eliminating bias. In addition to this, accounting for fairness would involve either teaching machines all factors that affect fairness or providing granular algorithmic controls to incorporate fairness.
Effects of Bias in AI:
First, for a hiring system, how do we eliminate bias: do we eliminate bias in algorithms while building deep-learning models, or do we eliminate bias in data by labeling and identifying all possible forms of it. Starting with an example, a deep learning model to assist hiring decisions would focus on answering this question: “Who is most likely to be successful in this role”. Or to be more specific: “Out of these five hundred candidates, who are the top two candidates who are most likely to be successful in this role”. The learning system would then likely optimize for a set of criteria, identify a set of features (parameters to filter candidates on) and provide probabilistic outcomes for each of the five hundred set of candidates, identifying the top two.
Often times, most of these “features” or patterns that the system learns are not well understood, even by the ones building such deep learning systems, particularly for deep learning models that have billions of parameters. More often that not, most of the data, which a system trains on, are just numerical values that it identifies and optimizes for and may or may not result in associations to actual real-world features, but in certain cases it may be possible to identify patterns in data that point to correlations to real-world features. For example, in an image, an identifiable feature would be a “nose”, or "color of hair". For a machine learning system that assists hiring decisions, an well-understood feature could be “score in an interview”.
Effective systems usually optimize for a specific outcome. Hiring systems typically optimize for what it’s trained on, particularly focusing on what the hiring company cares about. Labeling data to teach the system what to optimize for would be key to building effective AI systems. But focusing just on specific outcomes could result in a biased system.Identifying Bias
Identifying Bias in a deep learning system is challenging because of how such systems are built. To begin with, in order to identify bias, we need to understand what the deep learning system has “learned” and is basing its decisions on. This is currently an area that’s still being worked on [needs citation]. Certain types of bias are easy to identify. Let’s take an example. For the question, “Who’s likely to be successful in this role”, if the system always favors a specific attribute say a specific skin color, based on data it’s trained on, ignoring all others, that’s a form of bias, which results in discrimination, that's well understood and perhaps is even easy to identify and eliminate.
Let’s take a more complex example. A more sophisticated deep learning hiring-recommendation system trained on large amounts of data, identifies a set of “hidden features” that it bases its decisions on. While it may be an effective one, understanding the “features” that contribute to the decision may be just as important as identifying whether the deep learning system is a biased one. Say, the system, trained on large amounts of data, learns that only those with a “specific color of eyes” or “specific size of ears” are successful for a particular job. Would this be considered a form of discrimination? First, how do you identify all such forms of it. Also, how do you build deep learning systems that eliminate them?
Understanding Features in Deep Learning
The key thing to keep in mind while building self-supervised or semi-supervised deep learning systems is that, in most cases, features — patterns that a machine-learning algorithm identifies to predict outcomes — are determined “automatically” i.e., it learns what features are relevant to a task automatically based on labeled training data. In other words, the ML algorithm doesn’t learn based on “skin color” but instead identifies features as random strings of 0's and 1's or floating point numbers identifying patterns that it associates with success. While in some cases, it may be possible to identify discriminating factors easily, in most cases, it’s important to note that identifying “features" that can result in discrimination would require a deep understanding of how the model learns.
In the previously stated example, if a deep learning system learns (implicitly) out of ten thousand candidates that it has seen previously, only those with a specific “eye color” are successful in a role, and is biased towards that specific eye color, how do you build systems that can identify that specific form of bias. In fact, even identifying all features that a machine-learning model has learned by itself seems like an arduous task.