Paper—Natural Language Processing for Mapping Exam Questions to the Cognitive Process Dimension
Natural Language Processing for Mapping Exam
Questions to the Cognitive Process Dimension
https://doi.org/10.3991/ijet.v17i13.29095
Hindayati Mustafidah(), S. Suwarsito, Tito Pinandita
Universitas Muhammadiyah Purwokerto, Jawa Tengah, Indonesia
Abstract—Exam questions as a test instrument to measure educational
success must have good quality. This quality can be measured by the level of
cognition expected of students. The level of cognition reflects the mastery of
learning materials as a form of evaluation of the teaching and learning process
outlined in the curriculum. For this reason, the exam questions need to be mapped
into the cognitive process dimensions, namely based on the categories in the
Revised Bloom's Taxonomy (RBT). The mapping method used is Natural
Language Processing (NLP) as one of the fields of Artificial Intelligence
development. The stages in this mapping are pre-processing, including
tokenization, stemming, stop-word removal, and feature extraction using POS
Tagging. The output of this mapping process is in the form of categories of test
items into RBT: remembering (C1), understanding (C2), applying (C3),
analyzing (C4), evaluating (C5), and creating (C6). The classification results
obtained information that the exam questions prepared were still dominated at
the C2 cognitive level, which was indicated by the use of operational verbs in the
understanding category. The results of testing the method used produce an
accuracy of 82.22%. Thus, the NLP method can classify test items into Revised
Bloom's Taxonomy to determine the dimensions of students' cognitive processes.
Keywords—Natural Language Processing, exam questions, cognitive process
dimension, Revised Bloom's Taxonomy
1 Introduction
Education is a system, while learning is a form of educational activity. As a system,
education is a continuous activity. It involves many components, namely: a) raw input
(students), b) instrumental input (educators, objectives, materials/programs/curriculum,
methods, infrastructure, and facilities), c) environmental input (situation and conditions
of the educational environment, social, economic, cultural, security conditions), d) pro-
cess (implementation of education), and e) product (graduates) [1]. The availability of
educational components determines the quality of graduates both in quantity and qual-
ity and the functioning of each element according to its role in the implementation of
education as a system.
4 http://www.i-jet.org
, Paper—Natural Language Processing for Mapping Exam Questions to the Cognitive Process Dimension
An effective and efficient educational process will help assess the process and as-
sessment of learning outcomes; on the other hand, a good and correct, valid, and reliable
learning assessment will reveal the level of achievement and actual learning conditions.
The relationship between the elements of the educational process was proposed by [2]
which became the basis for other general models. The interpretation of Tyler's model,
in practice it is implemented into defining learning goals, organizing learning activities,
and evaluating the learning process implemented.
Educational objectives are formulated to describe student behavior reflected through
instructional objectives in the perspective of the course. The instructional goals are
elaborating educational plans [3]. The formulation of the proper educational purposes
will determine a good assessment. According to [4], a good assessment must describe
the expected learning outcomes. Assessment as a pedagogic ability [5] must be carried
out appropriately to get good feedback on the learning carried out. A good assessment
will obtain good feedback on the teaching carried out, which means that the quality of
education is also good.
The test is given to students as an assessment instrument in the form of items. Ref
[6] conveyed the stages of item development, namely: production, preparation, admin-
istration, reporting, documentation, and evaluation. Besides, ref [7] said that assessment
activities through written tests are traditional and common methods practiced in most
educational institutions today. Hence, questions must be given under the content of the
courses studied to meet learning objectives. Writing questions is a very challenging step
for questions makers. The situation is daring the teacher or items maker to compile
quality items that can be used to know the various cognitive levels. Thus, Bloom's Tax-
onomy or its revision named Revised Bloom's Taxonomy (RBT), has become general
references to guide for preparing test items for the teaching and learning process [8].
Interpretation of test results is conducted by making standard terms indicators of
what will be measured. These indicators must be derived from the formulated catego-
ries and subcategories, classifying educational objectives and the education taxonomy
classification. A goal formulation contains a verb which is a learning objective. This
verb describes the expected cognitive process, which includes six categories, namely
remembering (C1), understanding (C2), applying (C3), analyzing (C4), evaluating
(C5), and creating (C6) which previously consisted of knowledge, comprehension, ap-
plication, analysis, synthesis, and evaluation [9]. The six categories are known as Re-
vised Bloom Taxonomy, as shown in Figure 1.
Fig. 1. Bloom's Taxonomy and Revised Bloom's Taxonomy diagram [10]
iJET ‒ Vol. 17, No. 13, 2022 5
Natural Language Processing for Mapping Exam
Questions to the Cognitive Process Dimension
https://doi.org/10.3991/ijet.v17i13.29095
Hindayati Mustafidah(), S. Suwarsito, Tito Pinandita
Universitas Muhammadiyah Purwokerto, Jawa Tengah, Indonesia
Abstract—Exam questions as a test instrument to measure educational
success must have good quality. This quality can be measured by the level of
cognition expected of students. The level of cognition reflects the mastery of
learning materials as a form of evaluation of the teaching and learning process
outlined in the curriculum. For this reason, the exam questions need to be mapped
into the cognitive process dimensions, namely based on the categories in the
Revised Bloom's Taxonomy (RBT). The mapping method used is Natural
Language Processing (NLP) as one of the fields of Artificial Intelligence
development. The stages in this mapping are pre-processing, including
tokenization, stemming, stop-word removal, and feature extraction using POS
Tagging. The output of this mapping process is in the form of categories of test
items into RBT: remembering (C1), understanding (C2), applying (C3),
analyzing (C4), evaluating (C5), and creating (C6). The classification results
obtained information that the exam questions prepared were still dominated at
the C2 cognitive level, which was indicated by the use of operational verbs in the
understanding category. The results of testing the method used produce an
accuracy of 82.22%. Thus, the NLP method can classify test items into Revised
Bloom's Taxonomy to determine the dimensions of students' cognitive processes.
Keywords—Natural Language Processing, exam questions, cognitive process
dimension, Revised Bloom's Taxonomy
1 Introduction
Education is a system, while learning is a form of educational activity. As a system,
education is a continuous activity. It involves many components, namely: a) raw input
(students), b) instrumental input (educators, objectives, materials/programs/curriculum,
methods, infrastructure, and facilities), c) environmental input (situation and conditions
of the educational environment, social, economic, cultural, security conditions), d) pro-
cess (implementation of education), and e) product (graduates) [1]. The availability of
educational components determines the quality of graduates both in quantity and qual-
ity and the functioning of each element according to its role in the implementation of
education as a system.
4 http://www.i-jet.org
, Paper—Natural Language Processing for Mapping Exam Questions to the Cognitive Process Dimension
An effective and efficient educational process will help assess the process and as-
sessment of learning outcomes; on the other hand, a good and correct, valid, and reliable
learning assessment will reveal the level of achievement and actual learning conditions.
The relationship between the elements of the educational process was proposed by [2]
which became the basis for other general models. The interpretation of Tyler's model,
in practice it is implemented into defining learning goals, organizing learning activities,
and evaluating the learning process implemented.
Educational objectives are formulated to describe student behavior reflected through
instructional objectives in the perspective of the course. The instructional goals are
elaborating educational plans [3]. The formulation of the proper educational purposes
will determine a good assessment. According to [4], a good assessment must describe
the expected learning outcomes. Assessment as a pedagogic ability [5] must be carried
out appropriately to get good feedback on the learning carried out. A good assessment
will obtain good feedback on the teaching carried out, which means that the quality of
education is also good.
The test is given to students as an assessment instrument in the form of items. Ref
[6] conveyed the stages of item development, namely: production, preparation, admin-
istration, reporting, documentation, and evaluation. Besides, ref [7] said that assessment
activities through written tests are traditional and common methods practiced in most
educational institutions today. Hence, questions must be given under the content of the
courses studied to meet learning objectives. Writing questions is a very challenging step
for questions makers. The situation is daring the teacher or items maker to compile
quality items that can be used to know the various cognitive levels. Thus, Bloom's Tax-
onomy or its revision named Revised Bloom's Taxonomy (RBT), has become general
references to guide for preparing test items for the teaching and learning process [8].
Interpretation of test results is conducted by making standard terms indicators of
what will be measured. These indicators must be derived from the formulated catego-
ries and subcategories, classifying educational objectives and the education taxonomy
classification. A goal formulation contains a verb which is a learning objective. This
verb describes the expected cognitive process, which includes six categories, namely
remembering (C1), understanding (C2), applying (C3), analyzing (C4), evaluating
(C5), and creating (C6) which previously consisted of knowledge, comprehension, ap-
plication, analysis, synthesis, and evaluation [9]. The six categories are known as Re-
vised Bloom Taxonomy, as shown in Figure 1.
Fig. 1. Bloom's Taxonomy and Revised Bloom's Taxonomy diagram [10]
iJET ‒ Vol. 17, No. 13, 2022 5
