Let's Talk About Bias Training

4/23/2018
​​By Kelsey Medeiros, PhD
 
With Starbucks’ announcement regarding the closing of 8,000 stores to administer unconscious bias training to 175,000 employees, a debate has emerged over whether or not unconscious bias training can reduce bias. Much of the discussion has focused on a review article which found mixed results as to whether or not it works. Many have taken this as hard evidence that Starbucks’ training is a waste of time. Although the reported results certainly do not provide a compelling case that it will reduce bias, it is important to carefully evaluate the method by which researchers came to this conclusion. In doing so, it becomes clear that we really don’t know much about whether or not it will work!
 
Outside of the training content, there are three key elements to consider: 1) number of studies being evaluated, 2) training rigor, and 3) how effectiveness was measured.
 
Let’s begin with the number of studies included, or sample size. This report summarizes the results of 18 training efforts. Although 18 may appear enough to understand whether or not this training works, it is a relatively small number of studies to use to draw conclusions about a topic. Generally speaking, the larger the sample size, the more stable the results and the more confident we can be in our conclusions. Along these lines, the authors evaluated the included studies for rigor, with only 10 of the 18 being considered “very” or “extremely rigorous.” As I’ll discuss in the following sections, this can be problematic as it may mask true changes in attitudes.
 
Why does training rigor matter? The rigor with which a training or study is conducted can impact results! For instance, studies that follow best practices in the training literature, incorporating principles such as cases, feedback, and multiple sessions may be considerably more effective and thus, show larger effects. In comparison, if a less rigorous training, such as a 2-hour lecture with no feedback or practice opportunities, is conducted, results may suggest small to no changes because the training was not executed according to best practices. If the sample mainly includes less rigorous studies, this again may hide how truly effective these efforts can be when executed well.
 
Along these lines, how and when effectiveness is assessed matters. Our understanding of whether or not a training is effective is dependent on what metrics we use to assess the training effort. It is critical to note that measuring implicit attitudes such as unconscious biases is extremely difficult and must be done carefully. The Implicit Association Test is often used to evaluate the effectiveness of these efforts. However, scholars have critiqued this measure, arguing that it does not relate to actual behavior. (Note the sample size reported in that link – 499 studies!). Thus, our understanding of whether or not these trainings influence behavior, is limited by the assessment choice.
 
Additionally, measuring attitudes and behavior and pre- and post-training allows for an assessment of how attitudes and/or behavior change after training. However, administering an assessment immediately after training only provides part of the story. Have you ever taken a training and learned a bunch of new information only to forget it one week later? Me too! Without proper reinforcement from environmental cues and management support, training content may be easily forgotten. To remedy this, it is important to take additional measurements several weeks to months after a training initiative. This provides insights regarding the “stickiness” of training content. In other words, it can tell you about the enduring effect of training.
 
This discussion is not meant to dismiss the research that has been done in this area. Much of this work is pioneering and informative. However, as with all scientific studies, it is important to carefully evaluate the evidence in context. In the case of unconscious bias training, we do not have ample evidence to truly determine whether or not it will be effective.
 
One of the goals of Ethics Advantage is to improve our understanding in this area. We have taken careful steps to scientifically validate our assessment and demonstrate evidence that our assessment relates to ethical decision-making and ethical behavior. We are committed to furthering research in this area and continuing to grow our understanding of bias training and assessment.
 
Will Starbucks’ effort be effective at reducing employee bias? Without knowing how their training will be executed, it’s hard to say.