Good Morning!
 
My name is Matt Ruble and as I have stated in previous postings, I have
been developing a procurement certification examination for Maryland's
State Highway Administration.  I was given the opportunity to create 75
question multiple choice and true/false questions.  In order to create
these questions, I was given both a job analysis document and various
training documents.  After reviewing the documents, an initial pool of
106 questions were created.  These questions were then sent to SME's for
their input and editing.  Furthermore, I created a "rating form" which
was completed by SME's, asking the difficulty, time to complete, and
relevance of each question.  Using that information, I will be able to
create alternate forms of the examination with the same proportion of
difficult questions.  
 
After various editing meetings, the pool initial questions was complete.
I "piloted" the test with a small group of SME's to verify the content
of each of the questions.  After having completed the "pilot", I removed
so-called "tricky" questions, and returned the questions to the person
requesting the examination.  A meeting was held with other SME's
determining the final 75 "perfect" questions.  
 
After a training conducted, the test will be given to verify the
information has been learned.  This test had been created to ensure
ethical procedures are followed during drafting of contracts for various
merchandise ranging up to $25,000 throughout the state agency.  This
test will be given to approximately 75 to 100 employees.  Furthermore,
anyone appointed, promoted, or hired in the future will have to pass
this examination within 6 months of his or her start date.  After the
test has been distributed, I plan to run an item analysis to determine
the difficulty and appropriateness of each question from an actual
sample of test-takers.  I will examine the "very difficult" questions
and ensure it's validity through SME input.
 
If you have any questions or input, please feel free to contact me.
 
Thanks!
 
Matt Ruble
Testing Analyst
Maryland State Highway Administration
Recruitment and Examination Division
707 Calvert Street
Baltimore, MD  21202
410.545.5574
[log in to unmask]
 


The information contained in this communication (including any attachments) may be confidential and legally privileged.  This email may not serve as a contractual agreement unless explicit written agreement for this purpose has been made. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication or any of it's contents is strictly prohibited.  If you have received this communication in error, please re-send this communication to the sender indicating that it was received in error and delete the original message and any copy of it from your computer system.