Employment Testing, USAF Edition

This is interesting:

MEMORANDUM FOR TECHNICAL OFFICER SURVEY PARTICIPANT, 4 June 2009

FROM: AF/A1, 1040 Air Force Pentagon, Washington DC 20330-1040

SUBJECT: AFOQT Revision – Technical Officer Survey Participation

We are initiating a major revision of the Air Force Officer Qualifying Test (AFOQT). The AFOQT, in its many forms, has had an effective track record of predicting officer training and pilot training success since it was implemented in 1953. Its screening has saved tens of millions of training dollars by reducing attrition and contributed to an effective officer accession process. Since initial development, the AFOQT has been revised 17 times. The current version, Form S, was implemented in 2005. With our rapidly changing mission and Air Force culture, we want to ensure the next revision assesses all of the current critical officer skills.

To better identify theses technical skills, we are conducting a survey . . . .

The survey goes on to ask participants drawn from technical and aviation military career fields to rate the importance to success in their careers of 54 different mental, physical, and social facilities:

• Cognitive Abilities: Oral Expression, Listening Comprehension, Reading Comprehension, Written Expression, Mathematical Computation, Mathematical Reasoning, Inductive Reasoning, Deductive Reasoning, Memorization, Perspective Prioritization, Task Management (Multi-tasking), Pattern Recognition, Planning Resourcefulness, Foresight, Technology Literacy, Spatial Orientation, Visualization, Adaptability, Situational Awareness, Information Processing/Sensor Management, Electro-Mechanical Science, Earth/Weather Science, Critical Thinking, Perceptual Vigilance, Aviation Knowledge

• Psychomotor Abilities: Static Strength, Physical Fitness/Stamina, Finger Dexterity, Arm-Hand Steadiness, Multi-limb Coordination, Choice Reaction Time, Rate Control, Hand/Eye Coordination, Color Vision, Depth Perception, Auditory Acuity, Visual Acuity

• Interpersonal Abilities: Persuading/Influencing, Mediation, Cooperating, Assuming Responsibility, Responsiveness, Decisiveness, Resilience, Teaching/Mentoring, Work Effectively in Isolation Settings, Work Effectively in Stressful Situations, Empathy, Self-Assessing, Self-Discipline, Integrity, Selflessness

I once had an opportunity to ask a military engineer how much opportunity she had to actually apply her engineering education. The dirty little secret, she said, was that most military engineers did what she did: technical management. Specifically, most of them spent most of their time engaged in some facet of the military procurement process, a byzantine administrative function in which the Armed Services spend a great deal of effort educating officers to perform effectively.

But on the other hand, she went on, their experience in bringing in non-ABET-accredited degree holders was not good, even though the work they did was not, strictly speaking, engineering. People with B.A.s, even in management, tended to get lost in the technology. It was just easier to train engineers to be managers than to train managers in technology.

I was thinking about this in the context of the Ricci decision. As articulated by Justice Kennedy, the test of "business necessity" for purposes of avoiding disparate impact liability is whether the employment standard is "manifestly job related". But what if the evidence for job relatedness is only statistical? Ricci's exam avoided that problem because its questions were obviously specific to firefighting. But what if someone objected to the disparate impact of the AFOQT (and you know there is one) on the grounds that pilots and engineers did not actually have to solve the kinds of problems on test in their day-to-day jobs? The military could presumably point to the correlation between AFOQT scores and performance, but would that satisfy the courts?

I think the survey mentioned above wisely chose to rate broad categories of ability rather than narrow ones. I suppose that, say, mathematical computation and reasoning are measured by the ability to solve a second-order differential equation, but while professional engineers only seldom face the second problem, the first is indispensable to success in the profession.