Writing Good Multiple Choice Questions

Multiple choice questions

1. place severe limits on what you can test
2. can only test learning of facts
3. are easy to write
4. All of the above
5. a and b only
6. a and b and maybe c on a good day
7. None of the above

Answer: g

Strengths of Multiple Choice Questions

1. Easily administered to large numbers of students
2. Can cover a lot of material
3. Scoring is easy, objective, and, hopefully, reliable
4. Could be more reliable than subjectively scored questions
5. If written well, can measure learning objectives from simple to complex (facts to evaluations)
6. Item analysis is possible to include:
   1. a) % of students who got each question correct and % who answered each answer option
   2. b) Discrimination statistics comparing high scoring and low scoring students
7. Incorrect choices yield diagnostic information
8. Can compare performance from class to class

Weaknesses of Multiple Choice Questions

1. Constructing good items takes time (clear questions and options, one correct or best answer, plausible options)
2. Not very effective for measuring some types of thinking and achievement (e.g., synthesis)
3. Real world problem solving often involves proposing solutions versus selecting ones
4. Dependent on student's reading ability and instructor's writing ability
5. Lack of feedback on thinking process
6. Can be misinterpreted (students read into questions)
7. Tendency to test factual information only
8. May encourage guessing
9. May encourage surface learning
10. Doesn't demonstrate knowledge beyond answer options given
11. May be testing test-wiseness
12. May overestimate learning (students use elimination process)
13. Can be construed as picky or tricky

Developing a Test Bank

If possible, write questions and choices such that multiple test bank items can be generated from one basic question by simply changing the question or the answers.  For example, if a question asks, "which of the following best illustrates…" and is followed by four or five choices each of which correctly illustrates some course item, then that question becomes four or five possible questions simply by changing the question slightly. If a question asks, "which of the following is true about…" and is followed by four or five choices one of which is true, multiple questions can be generated by simply making the former true answer false and making a different choice true.

Do's & Don'ts – General

Minimize the amount of reading
Use words familiar to the students
Don’t never use double negatives
Assess course content versus other skills (test-wiseness, logic, grammar)
Avoid direct textbook quotes (could serve as a clue)
Be sensitive to diversity issues

Do's & Don'ts – Question (Stem)

Write questions testing various levels
Question should be clearly stated
Question should address a significant course item
State the question in a positive form
If use negatives, emphasize (bold, underline)
Check grammar and style (could cause reading confusion and can give clues)
Minimize "always", "never", "all"
Direct questions are preferable to incomplete statements
Include bulk of content in the question versus the choices

Do's & Don'ts – Choices (correct answer and distractors)

• Use 3 - 5 answer choices
• Put choices in vertical order
• Place options in logical order if there is one
• Avoid unnecessary repetition of text in the choices
• Include one definite best or correct option
• Avoid making the correct/best answer longer
• Make different letters/numbers correct and randomize (versus making "c" the most common right answer)
• Make distractors wrong, but plausible
• Avoid throw-away answers
• Avoid overlap and interdependence of distractors
• Use "none of the above" cautiously
• Avoid multiple correct answers (both a and c)
• Avoid fine distinctions between options (unless you're testing for that)
• Include common errors and misconceptions in choices as a means of assessing comprehension

Some suggested practices

• Before the exam - explain the nature of the questions, show examples, and give practice questions
• During the exam - be available to answer student questions, clarify and rephrase, announce changes if necessary
• After the exam - use the statistical analysis to critique and revise questions, review at least the most troublesome questions in class, accept student feedback on questions and their arguments and explanations for possible credit