DEVELOPMENT OF A DIAGNOSTIC TEST INSTRUMENT TO MEASURE HIGHER ORDER THINKING SKILLS ON RESPIRATORY SYSTEM MATERIAL BASED ON SUSTAINABLE HEALTH ISSUES
DOI:
https://doi.org/10.23969/jp.v11i03.64932Keywords:
ADDIE, Higher Order Thinking Skills (HOTS), Respiratory System, sustainable health issue, diagnostic test instrumentAbstract
The persistently low achievement of Higher Order Thinking Skills (HOTS) among Indonesian students, as reflected in PISA and TIMSS results, is partly attributable to the absence of test instruments specifically designed to diagnose weaknesses in analyzing (C4), evaluating (C5), and creating (C6), rather than merely identifying factual misconceptions. This study aimed to develop a HOTS diagnostic test instrument on the respiratory system topic contextualized within sustainable health issues (PM2.5 air pollution, smoking-related pathophysiology, and communicable respiratory diseases), and to examine its content validity, construct validity, reliability, and the resulting HOTS profile of Grade XI students at Nation Star Academy Surabaya. This Research and Development (R&D) study employed the ADDIE model and produced three parallel instrument sets (Set A, B, and C), each comprising six items (two multiple-choice items for C4, two multiple choice multiple answer items for C5, and two structured essay items for C6) that synthesized the Merdeka Curriculum Fase F Learning Outcomes with the Cambridge AS & A Level Biology Learning Outcomes. A limited field trial was conducted with 50 Grade XI Science students. Results showed very strong content validity (mean Aiken's V = 0.907). Construct validity, examined through corrected item-total correlations, sub-scale internal consistency, and exploratory factor analysis, provided initial but not fully confirmed evidence: the creating dimension (C6) proved the most empirically stable (item-total r up to 0.781), while analyzing (C4) and evaluating (C5) showed sensitivity to item format and stimulus heterogeneity (case specificity). Internal consistency reliability was acceptable for formative diagnostic purposes (Cronbach's Alpha = 0.623; McDonald's Omega = 0.642). The HOTS profile showed a declining gradient across cognitive levels: analyzing (C4) was high (72.00%), evaluating (C5) was moderate (46.67%), and creating (C6) was low (39.83%), indicating a gap in knowledge transfer toward argumentative evaluation and evidence-based creative problem formulation. The instrument is recommended as a formative diagnostic tool for teachers designing differentiated instruction, with the caveat that additional items per cognitive dimension and stimulus homogeneity are needed in further development to strengthen construct validity evidence.
Keywords: ADDIE; diagnostic test instrument; HOTS; respiratory system; sustainable health issues
Downloads
References
Book:
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman.
Arikunto, S. (2019). Dasar-dasar evaluasi pendidikan (Edisi revisi). Jakarta: Bumi Aksara.
Azwar, S. (2018). Reliabilitas dan validitas (Edisi 4). Yogyakarta: Pustaka Pelajar.
Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Alexandria, VA: ASCD.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Andover: Cengage Learning.
Mullis, I. V. S., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 international results in mathematics and science. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
Nitko, A. J., & Brookhart, S. M. (2014). Educational assessment of students (7th ed.). Boston: Pearson.
OECD. (2023). PISA 2022 results (Volume I): The state of learning and equity in education. Paris: OECD Publishing. https://doi.org/10.1787/53f23881-en
Reckase, M. D. (2009). Multidimensional item response theory. New York: Springer.
Retnawati, H. (2016). Analisis kuantitatif instrumen penelitian (Panduan peneliti, mahasiswa, dan psikometrikawan). Yogyakarta: Parama Publishing.
Wiliam, D. (2018). Embedded formative assessment (2nd ed.). Bloomington, IN: Solution Tree Press.
World Health Organization. (2013). Health literacy: The solid facts. Copenhagen: WHO Regional Office for Europe.
World Health Organization. (2022). WHO global air quality guidelines: Particulate matter (PM2.5 and PM10), ozone, nitrogen dioxide, sulfur dioxide and carbon monoxide. 2021 update. Geneva: World Health Organization.
Journal Article:
Alkhatib, O. J. (2022). An effective assessment method of higher-order thinking skills (problem-solving, critical thinking, creative thinking, and decision-making) in engineering and humanities. In 2022 Advances in Science and Engineering Technology International Conferences (ASET) (pp. 1–6). IEEE.
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139–148.
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104.
Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach's alpha for estimating reliability. Communication Methods and Measures, 14(1), 1–24.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41(4), 212–218.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749.
Nutbeam, D. (2008). The evolving concept of health literacy. Social Science & Medicine, 67(12), 2072–2078. https://doi.org/10.1016/j.socscimed.2008.09.050
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292.
Taber, K. S. (2018). The use of Cronbach's alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Pendas : Jurnal Ilmiah Pendidikan Dasar
This work is licensed under a Creative Commons Attribution 4.0 International License.
