Selection Test Validation and Adverse Impact in a Call Center
A call center hires about 500 customer-service representatives each year and uses a 50-item cognitive ability test as its first screening hurdle. The test has a criterion-related validity coefficient of .35 against supervisor ratings, and the current cutoff of 70 produces a 60% pass rate for White applicants and a 35% pass rate for Black applicants. The company wants to raise the cutoff to 80 to improve performance. A job analysis shows that the job requires following scripts, handling irate customers, and entering data accurately under time pressure. Turnover is high, and training takes six weeks. An industrial–organizational psychologist is asked whether the test is defensible and what selection system should be recommended.
1. Evaluate the validity and adverse impact evidence and determine whether raising the cutoff is defensible.
Read the worked answer
The test has modest but real criterion-related validity (.35) for supervisor ratings, and the job analysis supports some cognitive demand. However, the evidence is not sufficient to justify raising the cutoff without more analysis. A cutoff is a policy decision that should be linked to expected job performance and business necessity, not simply to the desire for a higher-scoring workforce.
The adverse impact is substantial. The ratio of the Black pass rate to the White pass rate is 35/60, or 0.58, well below the four-fifths rule of .80. Raising the cutoff to 80 would likely reduce the Black pass rate further and increase adverse impact. The psychologist should compute pass rates at the proposed cutoff, examine the test's relationship to actual job performance above and below the cutoff, and consider whether the supervisor ratings are contaminated or subjective.
Raising the cutoff is defensible only if the employer can show that the higher score is needed for safe or effective performance and that less discriminatory alternatives are unavailable. The current data do not establish that. The psychologist should recommend a validation and utility analysis before changing the cutoff.
2. What alternative selection system should the psychologist recommend, and what evidence would support it?
Read the worked answer
The psychologist should recommend a multi-hurdle or compensatory system that preserves validity while reducing adverse impact. Options include a structured interview, a job-knowledge or situational-judgment test, a work sample involving irate-customer scenarios and data entry, and a conscientiousness or customer-service orientation measure. These can be combined with the cognitive test in a weighted composite rather than using the cognitive test as a rigid first hurdle.
The evidence should include a criterion-related validation study using actual job performance, not only supervisor ratings. Objective criteria such as call handling time, quality monitoring scores, absenteeism, and turnover would strengthen the case. The psychologist should also examine whether the cognitive test adds incremental validity beyond the other predictors and whether banding or score adjustment is justified. Any change should be monitored for adverse impact by group and reviewed periodically.
The recommendation should be practical: pilot the new system, track pass rates and performance, and document the job-relatedness of each component. The psychologist should avoid promising that a new test will eliminate adverse impact; the goal is a defensible balance of validity, utility, and fairness.