College of Teacher Education

Second Semester, A.Y. 2020-2021

MODULE 7
Creating Test Questions

Introduction

This module entitled ‘Creating Test Questions’ would discuss pointers on how to create proper test
questions in multiple types of tests.

Date and Time Allotment

April 12 (6 hours)

I. Objectives

At the end of the end of this module, students should be able to:
1. Discuss the frameworks of making different type of test questions.
2. List the principles of making proper test questions.
3. Determine if given distractors are considered as effective.
4. Create appropriate and diverse test questions for different types of test.

II. Lecture
It can be argued that written tests are one of the most, if the most, versatile assessment tool for
learning as it can be created in a way that tests all levels of cognitive learning in the Revised Bloom’s
Taxonomy. Thus, it is in our best interest to know what test-writing technique could be applied in different
types of tests.

Binary Choice Tests

Binary choice test is also known as a true-false test, binomial choice test, or alternate response test
(Navarro, Santos, and Corpuz, 2017). This type of test utilizes two opposing answers where only one answer
is correct. Test-makers usually avoid this type of test due to the probability of getting a correct answer by
purely guessing which is exactly 50% of the time. Navarro et.al. (2017) suggested the use of a modified
binary choice test where learners will be required to identify the key elements of the sentence or to explain
their answers. The trio also suggested seven (7) rules in creating binary choice tests which are as follows:

1. Do not give a hint (inadvertently) in the body of the question.

What this means is that teachers must avoid giving clues in the question themselves. For example,
the statement “The Philippines, consisting of 7640 islands, is considered an archipelago” is obviously true
since the literal definition of archipelago is “a group of islands” (Merriam-Webster, 2021).

2. Avoid using the words “always”, “never”, “often” and other words that tend to be
either always true or false.
Consider the item “Water is always a liquid”. This statement could easily be found to be false because
of the word ‘always’ and the fact that a good percentage of students know of the existence of solid ice. What
we need to take away from this is that students might be able to deduce the truthfulness of an item by simply
looking for these words.

3. Avoid long sentences as these tend to be “true”. Keep sentences short.

This principle should be strictly followed if the teacher aims to measure LOTS as questions aimed at
these skills are factual and brief. Longer statements tend to be made up of phrases where some parts are
true and others are false which creates a problem for learners. For example, the item “Assessment is an
important part of teaching due to its ability to solely determine the skill of the learners in a classroom context.”
While it is true that assessment is an important part of learning, it could be argued that it could be used
outside of a classroom context which makes the second part of the sentence arguably false.

4. Avoid trick statements with some minor misleading word or spelling anomaly,
misplaced phrases, etc. A wise student who does not know the subject matter may
detect this strategy and thus get the answer correctly.
 This rule could be seen in the item “Ferdinan Marcos is the 10th President of the Philippines.” One
could easily notice the error in the name of the subject to which the student could easily surmise that the
statement is false even without being sure if the subject is the actual 10th President of the country thus making
the item useless in assessing the knowledge of the learning regarding the matter.

5. Avoid quoting verbatim from reference materials or textbooks. This practice sends
the wrong signal to the students that it is necessary to memorize the textbook word
for word and thus, acquisition of higher-level thinking skills is not given due
importance.
For instance, the item “Test-makers usually avoid binary choice test due to the probability of getting
a correct answer by purely guessing which is exactly 50% of the time” is not a good test item due to the fact
that it is copied from the previous parts of the module.

6. Avoid specific determiners or give-away qualifiers. Students quickly learn that

strongly worded statements are more likely to be false than true, for example,
statements with “never”, “no”, “all”, or “always.” Moderately worded statements
are more likely to be true than false. Statements that are moderately worded use
“many”, “often”, “sometimes”, “generally”, “frequently” or “some” usually should
be avoided. e.g., Executives usually suffer from hyperacidity. The statement tends
to be correct. The word “usually” leads to the answer.
This rule reiterates the need to remove indicators in items for binary-choice as a general rule.
Personally, only mathematical theorems should make use of these indicators.

7. With true or false questions, avoid a grossly disproportionate number of either true
or false statements or even patterns in the occurrence of true and false statements.
Traditionally, teachers use a discernable pattern of answers in binary choice tests for the ease of
checking. However, this becomes problematic when the students inadvertently discover the pattern and
based their answers on it without actually reading the items. Another problem is skewing the ratio between
true and false statements. For example, a teacher may create 9 true statements and only 1 false statement
which might encourage the student to answer all the items “True” when he/she perceived this disparity.
Thankfully, there are technology tools that could randomize the order of the questions in a binary choice test
to remove any pattern on the answers.
Note that these are mainly rules that should be followed most of the time. It is under the teacher’s
decision to move away from convention in order to pursue creativity as long as these changes still properly
assess the learners.

Multiple Choice Test

A multiple-choice test is basically a broader binary-choice test where it uses at least three choices
for each item. Technically, a multiple-choice item is composed of two parts: the stem and the options. The
stem refers to the body of the item or, basically, the question or statement of the item. The options are the
choices given to the students that could answer or correspond to the stem. The correct option is called the
key while the others are called distractors. The role of the distractors is to convince and appeal to non-
learners to choosing them while leaving those who actually learned unaffected simply because they know
the key. The way in which these distractors could be varied, allowing for practitioners to test HOTS even
through a written test. Navarro, et. al. (2017) has also laid out several rules in creating multiple choice items.

1. Do not use unfamiliar words, terms, and phrases. The ability of the item to
discriminate or its level of difficulty should stem from the subject matter rather than
from the wording of the question.
Suppose you have the stem “What is the main effect of having a negative Kurtosis value in an
estimated normal classroom setting?”. Some learners might not have an idea on what Kurtosis is. The same
goes for what defines a normal classroom. In this context, the stem is testing the student’s knowledge on
different concepts without focusing on the main idea of the question which is the effect of Kurtosis.

2. Do not use modifiers that are vague and whose meanings can differ from one person
to the next such as: much, often, usually, etc.
Stems like “Most of the freshwater resource can be found in _________” might not be an entirely
bad stem but it can be improved into “Majority of freshwater could be found in ______” or even “At least 68%
of fresh water on Earth could be found in ______”. It is altogether recommended to use precise modifiers
such as percentage, percentile, or ratio.
 3. Avoid complex or awkward word arrangements. Also, avoid use of negatives in the
stem as this may add unnecessary comprehension difficulties.
The stem “Considering the arrangements of the books on the cases, how many instances is present
where it is possible to discern the alphabetical order of subjects they contain?” is complex at a glance and is
even confusing when you try to determine what it asks for. So instead of using the former stem, one could
simply use “How many possible arrangements of the books follow an alphabetical order based on their title?”.

4. Do not use negative or double negatives as such statements tend to be confusing. It

is best to use simpler sentences rather than sentences that would require expertise in
grammatical construction.
If one has the stem “What are the instances that are not unconducive to learning?”, it is better to
simply remove the double negatives and go straight to the point. As such, it is better to have the stem “What
are the instances that are conducive to learning?”.

5. Each item should be as short as possible; otherwise, you risk testing more for reading
and comprehension skill.
Stems should be straightforward is stating what it wants to be answered. Long stems could only
serve to test the endurance of learners as they will repeatedly read it in order to gain a better understanding
of it.
6. Distracters should be equally plausible and attractive.
Imagine, if you will, a stem “Who is the father of Modern Psychology?” where the options are Sigmund
Freud, Rodrigo Duterte, Dolphy, and Arnold Schwarzenegger. Chances are the student will answer Sigmund
Freud not because it is the correct answer but because the distracters are obviously wrong.

7. All multiple-choice options should be grammatically consistent with the stem.

If we have the stem “The most strenuous task among the given is _______”, then the options should
NOT be: a) to dancing with a stranger, b) is swimming, or c) when dancing. It is better to use simple yet
grammatically cohesive options such as: a) swimming, b) dancing, and c) running.

8. The length, explicitness, or degree of technicality of alternatives should not be the

determinants of the correctness of the answer.
Consider the stem “What is the definition of a triangle?” with the options:
a) a three-sided polygon having where the total measure of the interior angles is 180 degrees.
b) shape
c) closed area
This item is not necessarily a good item due to the unequal length of the choices. Thus, it is advisable
to make the choices have a similar or near-equal number of words.

9. Avoid stems that reveal the answer to another item.

Consider the stems “Behaviorists assert that behavior indicates learning while cognitivists assert that
______” and “What do behaviorists measure in order to indicate learning?” are not compatible stems to put
in a test since the former provides a clue or answer to the latter.

10. Avoid alternatives that are synonymous with others or those that include or overlap
others.
Say we have the stem “Which of the following contribute to global warming?” with the choices:
a) Pollution
b) Planting of trees
c) Reforestation
d) Using bicycles
Notice that “planting of trees” is actually a part of “Reforestation” which makes the latter option
obsolete. At these instances, it is advisable to change the latter distracter.

11. Avoid presenting sequenced items in the same order as in the text.
This rule asserts that a teacher must not present distracters in the same way that they are presented
in any reference material.

12. Avoid use of assumed quantifiers that many examinees may not be aware of.
 For example, the concept of percentages within the normal curve may only be known to some of
your learners. If such is the case, it is advisable to not use stems that assume learners know the functions of
the normal curve.

13. Avoid use of unnecessary words or phrases which are not relevant to the problem
at hand (unless such discriminating ability is the primary intent of the evaluation).
The item’s value is particularly damaged if the unnecessary material is designed to
distract or mislead. Such items test the student’s reading comprehension rather than
knowledge of the subject matter.
If you have the stem “Emilio Aguinaldo led the Revolutionary Army during the Spanish reign on the
Philippines which led him to be the first President of the Philippines. Who is the 12 th President of the
Philippines?”, you can notice that the first sentence on the stem is unnecessary as the complete question
could be found in the second sentence thus making the first sentence completely irrelevant and should be
removed.

14. Avoid use of non-relevant sources of difficulty such as requiring a complex

calculation when only a knowledge of a principle is being tested.
This is usually only applied to mathematics. For example, one should not hint at the use of calculus
to solve for the area of a figure when it is possible to do so using analytical geometry.

15. Pack the question in the stem.

The stem “A triangle is ______” is NOT a good stem for use in a test since it could lead to various
answers which forces the learners to perform unnecessary cognitive tasks which could be detrimental to their
performance in the other parts of the test. Not to mention that it confuses the learners as well.

16. Use the “none of the above” option only when the keyed answer is totally correct.
When choice of the “best” response is intended, “none of the above” is not
appropriate, since the implication has already been made that the correct response
may be partially inaccurate.
17. Note that use of “all of the above” may allow credit for partial knowledge. In a
multiple-choice item (allowing only one option choice), if a student only knew that
two (2) options were correct, he then could deduce the correctness of “all of the
above”. This assumes you are allowed only one correct choice.
The general rule is that teachers must avoid using “all of the above” and “none of the above” as
options for a multiple-choice test as they create inconsistencies in the evaluation of the learners on that
particular item.

18. Having compound response choices may purposefully increase difficulty of an item.
As one may already know from my multiple-choice tests, there are some phrases that seem to be
correct but is not the correct answer due to another option being more appropriate for the given stem. This
technique is not easily done but it could be continuously practiced.
Again, there are other rules that one may follow in creating multiple choice tests. However, do try to
extract the rationale that the rules above set.

Matching Type
According to Navarro, et. al. (2017), a multiple-choice test is a “modified multiple choice type items
where the choices progressively reduce as one successfully matches the items on the left with the items on
the right.” However, the choices may not necessarily be reduced if a practitioner states that some of the given
options on the second column may be used more than once. Note, however, that matching type tests are
only suitable for testing the LOTS of the learners.
In a similar note to the tests above, Navarro, et. al. (2017) has also listed several guidelines in
creating a matching type test.

1. Match homogenous not heterogenous items.

This rule states that, for example, if a teacher wants to match a polygon with the number of sides,
then the options in the second column should all refer to numbers. None should refer to their definition or
such. See the table below for homogenous items.
 Direction: Draw a line connecting the shapes on the left column to their corresponding number of
sides in the second column.
Shapes Number of Sides
1. Octagon a. Three
2. Triangle b. Four
3. Hexagon c. Five
4. Pentagon d. Six
e. Seven
f. Eight
As one can see, all the items on the second column refer to sides. If one sees an option such as “an
area equal to square of the side” then the students will simply dismiss such an option which makes it useless.

2. The stem (longer in construction than the options) must be in the first column
while the options (usually shorter) must be in the second column.
This rule is stated due to the face that the learners will often reread the second column in order to
pinpoint the correct answer for a given stem. Not following this rule results to student reading fatigue which
is usually not the main evaluation goal of the test.

3. The options must be more in number than the stems to prevent the student from
arriving at the answer by mere process of elimination.
Technically, the probability of getting a correct answer in a matching type test without actually
knowing the correct answer with every option that you eliminate through correctly identifying one. This
increases to 100% if you are left to answer the last stem with only one choice remaining which renders the
last stem inefficient in evaluating the learners. Consider the matching type given above where there are only
4 stems but 6 choices.

4. To help the examinee find the answer easier, arrange the options alphabetically or
chronologically.
Since the options will be the one read often, it is advantageous for the learners to see a sort of
arrangement in the options to ease their answering experience. Teachers are testing their knowledge on the
items on the test anyway, not on their reading stamina.

5. Like any other test, the direction of the test must be given. The examinees must
know exactly what to do.
Direction is, obviously, important to any test as it shows whether a student could follow instructions
especially in complicated types of test. As a starter, it is better to use common words and short, precise
phrases in giving directions. Also state whether some of the options may be used more than once.
Following these rules allow teachers to create sufficient matching type tests. Some may even try to
incorporate creative ways in answering one.

Supply Type or Constructed-Response Type

A supply type test is a variation of the multiple-choice test wherein no choices will be given to the
students which forces them to supply their own. The item is composed of a stem and a blank where students
will complete the statement using the correct answer. An example stem is “The first element in the periodic
table of elements is _______”.
For constructed (as well as multiple choice tests), teachers must remember two general rules. Never
start with a blank like in the stem “_______ is the powerhouse of the cell” as this could trigger some negative
emotions in the learner when they are immediately exposed with an unknown. The other is that a stem must
start with a word, not a number of symbol. Thus, the stem “2 minus 3 is equal to what?” is NOT a good stem.
The effectivity of these types of items depends on how the item is made by the teacher. As such, the
only guideline in this type of test is to put the blanks at the later parts of the stem. A stem should not begin
with a blank.

III. Activity
For your activity, I will upload a set of Google Forms in our Google Classroom for you to answer.
This form has to be answered INDIVIDUALLY.

IV. Assessment
 For your assessment, you are going to create a binary choice test, a multiple-choice test, a matching
type test, and a supply type test items based on a selected topic from your major of specialization. You may
refer to your respective curriculum guides for the list of contents.
For a binary choice test, you are going to create 10 appropriate items. Make sure that you follow the
guidelines stated above. Be very mindful of your syntax and grammar.
For a multiple-choice test, you are going to create 20 stems with 4 options each. Be sure to follow
the rules above and ensure that the distractors and stem are appropriate enough.
For the matching type, you are going to create a matching type with 10 stems having at least 13
options. Make sure that the items are homogenous. If you want to relate literary works to their authors, make
sure that one column only contains literary works and the other contain only authors. Also observe the other
guidelines.
Make sure to put directions on each type of test. Make the directions clear and precise. If you want
students to encircle their answers on the multiple-choice test, then say so. If you want them to write their
answers on the blank preceding each stem, then say so, etc.
Be warned that copy-pasting at least one (1) test item from the internet will INVALIDATE your answer
for the entire assessment part of this module

V. Other References
• Navarro, Rosita L., Santos, Rosita G., and Corpuz, Brenda B. (2017) Assessment of
Learning 1 (3rd Edition) Lorimar Publishing, Inc. Quezon City, Metro Manila

Prepared by:

ALLEN CHRISTIAN V. TORRES, MSEd

Instructor, CTE

Checked by:

GLORIA C. NEMEDEZ, MAEd

Program Head, CTE

Approved by:
 EVELYN ABALOS-TOMBOC, DBA
President, University of Eastern Pangasinan