Problems with testing

Article in the Palm Beach Post by Tom Blackburn 6 July 2009

Reports a case where fire fighters were selected for promotion based on a written test. No African-American passed; 18 white firefighters did. The case went to court due to alleged discrimination. "The white guys won."

The article includes some interesting insight into testing. One of the first tests was for Intelligence Quotient (IQ), which the author claims were "misused almost at once." The test writers looked at how different sexes answered questions and edited those which showed a bias. People have since used the results of IQ tests to show men and women have the same IQ, without realising that the question set had been specifically designed to achieve that result.

"The misuse of testing fills a need to believe that we can take the human factor out of judgement calls. Human error sneaks back in as part of the design of each test, but we act surprised every time we find it." We all seem to be reassured that tests give numerical, 'objective' results. The reality is that most tests have little evidence to show the results are anywhere near correlated to what supposedly was being tested.

"It's easy enough to devise a test that will tell you whether someone has memorized the times tables." But many factors such as leadership (as required by the firefighters in the case cited) is immeasurable.

"It's no surprise that we do a poor job of testing for leadership. After 100 years of testing for intelligence, we still are not clear about what IQ numbers really mean."