As proprietor of a test-preparation school geared toward promotion exams in the NYPD, I wrote an article in this newspaper in December of 2006 questioning the fairness of promotion exams in the Police Department.
In response, a professional from the Department of Citywide Administrative Services wrote in a letter to the editor that exams are developed by working very closely with a panel of Police Department Sergeants and superior officers. And therein lies the problem. There and a few other places.
An Unpredictable Mess
The process of developing exam questions is unprofessional. These panels are made up of members of the rank who have no particular education or experience in creating questions that properly assess the ability of the test-taker. The result is an unpredictable mess of minutia with no professional standards.
The process does require input from the fieldespecially in policing. But the crafting of the actual questions and the structure of the overall examination should be done by experienced police professionals who have the necessary knowledge and experience of the field but also have education, training and experience in how to create a professional exam.
Little Relation to Job
DCAS is supposed to play that role but doesnt. A recent example is the last Lieutenants exam where the process was left entirely in the hands of one individual Lieutenant who holds a PhD in English. The exam questions were so far removed from the job description of Lieutenant of Police that 20 of the 100 questions were thrown out or changed. That is unacceptable.
To its credit, DCAS realized that mistake and, for the next Captains exam, brought in a Captain who actually works for us in our test-preparation school, The Key, and so has the necessary experience. Mr. Mifsud supervised the preparation of a Captains exam that was administered and promulgated with no changes.
But Captain Mifsud is now retired. These two anecdotes illustrate the instability of the process. Students just dont know what to expect. Especially after the most recent Sergeants exam administered this past October.
Piling on the Questions
Three weeks before the October exam, an exam which most students spent the better part of six months preparing for, DCAS announced that the test would contain multiple-choice questions which will be used for research purposes only.
Every exam for the last 20 years consisted of 100 questions and lasted six hours. Notice that the above-quoted last-minute announcement did not say how many of these research questions would be included in the still-six-hour exam. There were 30. The test-takers found this out when they sat for the exam, not a minute before.
There are standards in the industry of testing where we see research or, as the College Board calls them, variable questions such as on the SAT. However, it is made known well in advance exactly how many variable questions/sections will be asked and each section is allotted its own necessary time period.
DCAS gave us half a loaf. If industry standards are to be followed, they should be followed for the entire process. The College Boards variable sections are developed by Test Development Committees which are experts in their fields and oversee all aspects of test development to ensure that the test is carefully designed.
Why Fewer Cops Compete
Is it any wonder that members of the Police Department have lost faith in the process? Over the last 20 years the percentage of people in each rank that sit for these exams has dropped dramatically.
Will Commissioner Bratton take steps to fix this problem? If history is any judge, he might.
When Bratton became Police Commissioner in 1994, there had not been a Lieutenants exam administered in nearly eight years. During his tenure, a test was announced and developed by an outside professional entity which was well-versed in crafting a promotion exam. The 1996 Lieutenants exam still stands as one of the more-challenging-but-most-job-related exams in recent history. Lets hope history does repeat itself and that faith in the promotion exam process gets restored.
Mr. Monaghan, a retired NYPD Captain, holds a master's degree in public administration from the Kennedy School of Government at Harvard University and is proprietor of The Key, a test preparation school for NYPD promotion exams.
