Thursday, July 16, 2009

Employment Testing, USAF Edition

This is interesting:

MEMORANDUM FOR TECHNICAL OFFICER SURVEY PARTICIPANT, 4 June 2009

FROM: AF/A1, 1040 Air Force Pentagon, Washington DC 20330-1040

SUBJECT: AFOQT Revision – Technical Officer Survey Participation

We are initiating a major revision of the Air Force Officer Qualifying Test (AFOQT). The AFOQT, in its many forms, has had an effective track record of predicting officer training and pilot training success since it was implemented in 1953. Its screening has saved tens of millions of training dollars by reducing attrition and contributed to an effective officer accession process. Since initial development, the AFOQT has been revised 17 times. The current version, Form S, was implemented in 2005. With our rapidly changing mission and Air Force culture, we want to ensure the next revision assesses all of the current critical officer skills.

To better identify theses technical skills, we are conducting a survey . . . .

The survey goes on to ask participants drawn from technical and aviation military career fields to rate the importance to success in their careers of 54 different mental, physical, and social facilities:

  • Cognitive Abilities: Oral Expression, Listening Comprehension, Reading Comprehension, Written Expression, Mathematical Computation, Mathematical Reasoning, Inductive Reasoning, Deductive Reasoning, Memorization, Perspective Prioritization, Task Management (Multi-tasking), Pattern Recognition, Planning Resourcefulness, Foresight, Technology Literacy, Spatial Orientation, Visualization, Adaptability, Situational Awareness, Information Processing/Sensor Management, Electro-Mechanical Science, Earth/Weather Science, Critical Thinking, Perceptual Vigilance, Aviation Knowledge

  • Psychomotor Abilities: Static Strength, Physical Fitness/Stamina, Finger Dexterity, Arm-Hand Steadiness, Multi-limb Coordination, Choice Reaction Time, Rate Control, Hand/Eye Coordination, Color Vision, Depth Perception, Auditory Acuity, Visual Acuity

  • Interpersonal Abilities: Persuading/Influencing, Mediation, Cooperating, Assuming Responsibility, Responsiveness, Decisiveness, Resilience, Teaching/Mentoring, Work Effectively in Isolation Settings, Work Effectively in Stressful Situations, Empathy, Self-Assessing, Self-Discipline, Integrity, Selflessness

I once had an opportunity to ask a military engineer how much opportunity she had to actually apply her engineering education. The dirty little secret, she said, was that most military engineers did what she did: technical management. Specifically, most of them spent most of their time engaged in some facet of the military procurement process, a byzantine administrative function in which the Armed Services spend a great deal of effort educating officers to perform effectively.

But on the other hand, she went on, their experience in bringing in non-ABET-accredited degree holders was not good, even though the work they did was not, strictly speaking, engineering. People with B.A.s, even in management, tended to get lost in the technology. It was just easier to train engineers to be managers than to train managers in technology.

I was thinking about this in the context of the Ricci decision. As articulated by Justice Kennedy, the test of "business necessity" for purposes of avoiding disparate impact liability is whether the employment standard is "manifestly job related". But what if the evidence for job relatedness is only statistical? Ricci's exam avoided that problem because its questions were obviously specific to firefighting. But what if someone objected to the disparate impact of the AFOQT (and you know there is one) on the grounds that pilots and engineers did not actually have to solve the kinds of problems on test in their day-to-day jobs? The military could presumably point to the correlation between AFOQT scores and performance, but would that satisfy the courts?

I think the survey mentioned above wisely chose to rate broad categories of ability rather than narrow ones. I suppose that, say, mathematical computation and reasoning are measured by the ability to solve a second-order differential equation, but while professional engineers only seldom face the second problem, the first is indispensable to success in the profession.

4 comments:

Trumwill said...

This is only tangentially related, but my father cut his professional teeth with the DoD (civilian-side) as an aerospace engineer. Anyway, when he decided to go to graduate school, he went for an economics degree and was in administration thereafter in cost estimation, auditing, and supervising auditors.

Trumwill said...

Actually, a better example might be my brother. Earlier this year he shifted from being an engineer to a contract analyst for his contractor. He now monitors efficiency, progress, overruns, and so on. Unlike Dad, his Bachelor's and Master's were both in engineering.

(My other brother is a database engineer, which ironically is what I trained to do but haven't done since my first job. His degree is in Biology. Totally unrelated to your post, but thought that I would fill out the family tree).

Φ said...

Wow, you come from a whole family of technical types! I'm the first one in my family to get an engineering degree; all my progenitors were Arts & Letters people.

S. said...

My brother was interested in the whole pilot training a while ago, but his focus right now is getting his degree in engineering. The employment testing explained above sounds like one of the toughest versions of assessment that a Professional Employer Organization administers. I'm not sure if my brother would still be interested in being a pilot if he knew the kinds of testing he had to go through. Not that he is lazy or anything, but he knows that he is not great at testing just because of nerves and the pressure to do well.