Good
Morning!
My name is Matt
Ruble and as I have stated in previous postings, I have been developing a
procurement certification examination for Maryland's State Highway
Administration. I was given the opportunity to create 75 question multiple
choice and true/false questions. In order to create these questions, I was
given both a job analysis document and various training documents. After
reviewing the documents, an initial pool of 106 questions were created.
These questions were then sent to SME's for their input and editing.
Furthermore, I created a "rating form" which was completed by SME's, asking the
difficulty, time to complete, and relevance of each question. Using that
information, I will be able to create alternate forms of the examination with
the same proportion of difficult questions.
After various editing meetings, the pool initial
questions was complete. I "piloted" the test with a small group of SME's
to verify the content of each of the questions. After having completed the
"pilot", I removed so-called "tricky" questions, and returned the questions to
the person requesting the examination. A meeting was held with other
SME's determining the final 75 "perfect" questions.
After a training
conducted, the test will be given to verify the information has been
learned. This test had been created to ensure ethical procedures are
followed during drafting of contracts for various merchandise ranging up to
$25,000 throughout the state agency. This test will be given to
approximately 75 to 100 employees. Furthermore, anyone appointed,
promoted, or hired in the future will have to pass this examination within 6
months of his or her start date. After the test has been
distributed, I plan to run an item analysis to determine the difficulty and
appropriateness of each question from an actual sample of test-takers. I
will examine the "very difficult" questions and ensure it's validity through SME
input.
If you have any
questions or input, please feel free to contact me.
Thanks!
Matt Ruble
Testing Analyst
Maryland State Highway
Administration
Recruitment and Examination
Division
707 Calvert Street
Baltimore, MD 21202
410.545.5574
The information contained in this communication (including any attachments) may be confidential and legally privileged. This email may not serve as a contractual agreement unless explicit written agreement for this purpose has been made. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication or any of it's contents is strictly prohibited. If you have received this communication in error, please re-send this communication to the sender indicating that it was received in error and delete the original message and any copy of it from your computer system.