Email this article Challenges and Barriers in Vocational Learning Assessment: A Case Study of Automotive Engineering Expertise Program in Vocational High Schools
This study aims to analyze the challenges and barriers encountered in vocational learning assessment and to propose a model to strengthen assessment practices based on empirical findings. A qualitative multiple-case study was conducted in one public and one private vocational high school in the Special Region of Yogyakarta, Indonesia, from March to October 2025. Six productive-subject teachers with at least 8 years of teaching experience were purposively selected. Data were collected through in-depth semi-structured interviews and document analysis of lesson plans, teaching modules, assessment instruments, and learning outcome records. We combined document analysis with in-depth interviews (triangulation) to maximize data validity. Thematic analysis was carried out with ATLAS.ti 22 through data familiarization, initial coding, theme generation, theme review, theme definition, and reporting. Trustworthiness was strengthened through source triangulation, member checking, peer debriefing, and cross-case comparison. The study identified multidimensional challenges in the cognitive, psychomotor, and affective domains. Cognitive assessment was constrained by low student literacy engagement, the dominance of multiple-choice tests, limited use of diagnostic assessment, and weak follow-up of item analysis. Psychomotor assessment was constrained by limited facilities, limited teaching time, equipment wear, and uneven opportunities for observing individual performance. These barriers differed between the two schools due to variations in management flexibility, team teaching, procurement, and industry involvement. Affective assessment was constrained by large class sizes, limited observation time, teacher reliance on informal impressions, and the absence of explicit attitude assessment rubrics. The findings indicate the need for a context-sensitive model that strengthens meaningful assessment, continuous diagnostic-formative-summative cycles, resource collaboration, and teacher capacity. The proposed model remains hypothetical and requires further validation and testing of its effectiveness.
Keywords: vocational learning assessment, cognitive, psychomotor, affective, case study.
Abdurrahman, I. S., & Mahmudah, F. N. (2023). Development of a digital-preneurship measurement Instrument: Alignment approach through project-based learning. International Journal of Educational Methodology, 9(1), 283–295. https://doi.org/ 10.12973/IJEM.9.1.283
Airasian, P. W. (2001). Classroom assessment: Concepts and applications (4th ed.). McGraw-Hill. https://eric.ed.gov/?id= ED446097
Ayub, M. R. S. S. N., Istiyono, E., Munadi, S., Permadi, C., Pattiserlihun, A., & Sudjito, D. N. (2020). Analisa penilaian soal fisika menggunakan model Rasch dengan program R [Analysis of physics question assessment using the Rasch model with the R program]. Jurnal Sains dan Edukasi Sains, 3(2), 46–52. https://doi.org/10.24246/juses.v3i2p46-52
Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2007). Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks. Educational Research Review, 2(2), 114–129. Doi:10.1016/j.edurev.2007. 6.001
Baartman, L. K. J., Prins, F. J., Kirschner, P. A., & Van der Vleuten, C. P. M. (2007). Determining the quality of competence assessment programs: A self-evaluation procedure. Studies in Educational Evaluation, 33 (3-4), 258–281. https://doi.org/ 10.1016/j.stueduc.2007.07.004
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80 (2). https://doi.org/10.1177/003172171009200119
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10. 1191/ 1478088706qp063oa
Chappuis, S., & Stiggins, R. J. (2002). Classroom assessment for learning. Educational Leadership. 60(1). http://hssdnewteachers.pbworks. com/w/file/fetch/50394085/classroom.assessment.for.learning.chappuis.pdf.
Carless, D. (2007). Learning-oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57–66. https://doi.org/10.1080/14703290601081332
Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches (2nd ed.). Sage Publications, Inc.
Firmania, N., Sudira, P., & Minghat, A.D. (2018). Literature review: Authentic assessment in vocational high schools. International Journal of Engineering & Technology. 7(4): 191–194. DOI:10.14419/ijet.v7i4. 33.23557
Fullan, M., & Quinn, J. (2015). Coherence: The right drivers in action for schools, districts, and systems. Corwin Press.
Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67–86. doi:10.1007/BF02504676
Heritage, M. (2010). Formative assessment: Making it happen in the classroom. Corwin Press. https://doi.org/10.4135/9781452219493
Istiyono, E. (2020). Pengembangan instrumen penilaian dan analisis hasil belajar fisika dengan teori tes klasik dan modern (Edisi Kedua) [Development of assessment instruments and analysis of physics learning outcomes using classical and modern test theories (Second Edition)]. UNY Press.
Jiang, C., Ma, H., Luo, Y., Fong, D. Y. T., Umucu, E., Zheng, H., Zhang, Q., Liu, X., Liu, X., Spruyt, K., & Meng, R. (2023). Validation of the Chinese version of the Perceived Stress Scale-10 integrating exploratory graph analysis and confirmatory factor analysis. General Hospital Psychiatry, 84, 194–202. https://doi.org/10.1016/ j.genhosppsych.2023.07.008
Lin, P. Y., Chen, P. Y., Liu, Y. C., & Yang, H. C. (2024). The impact of the Ok4R mobile platform on vocational students’ improvement of science reading comprehension and logical thinking. Journal of Baltic Science Education, 23(1), 90–103. https://doi.org/10.33225 /jbse/24.23.90
Mardapi, D. (2008). Teknik penyusunan instrumen tes dan non tes [Techniques for developing test and non-test instruments]. Mitra Cendikia Offset.
Mardapi, D. (2017). Pengukuran, penilaian dan evaluasi pendidikan (Edisi ke-2) [Measurement, assessment, and evaluation of education (2nd ed.)]. Parama Publishing.
McMillan, J. H. (2018). Classroom assessment: Principles and practice that enhance student learning and motivation (7th ed.). Pearson.
Misbah, Z., Gulikers, J., Dharma, S., & Mulder, M. (2020). Evaluating competence-based vocational education in Indonesia. Journal of Vocational Education & Training, 72(4), 488–515. https://doi.org/ 10.1080/13636820.2019.1635634
Mustapha, R., Sadrina, Nashir, I. M., Azman, M. N. A., & Hasnan, K. A. (2020). Assessing the implementation of project-based learning (PjBL) in the Department of Mechanical Engineering at a Malaysian Polytechnic. Journal of Technical Education and Training, 12(1 Special Issue), 100–118. https:// doi.org/10.30880/jtet.2020.12.01.011
Nadzar, A. H., & Supriyadi, E. (2020). Pengembangan media pembelajaran berbasis android database komponen listrik [development of android-based learning media for electrical component databases]. Jurnal Edukasi Elektro, 4(2), 103–109. https://doi.org/10.21831/jee.v4i2.35855
Nisa, C., & Retnawati, H. (2018). Comparing the methods of vertical equating for the math learning achievement tests for junior high school students. REID (Research and Evaluation in Education), 4(2), 164–174. https://doi.org/ 10.21831/reid.v4i2.19291
Ordóñez-Ávila, R., Salgado Reyes, N., Meza, J., & Ventura, S. (2023). Data mining techniques for predicting teacher evaluation in higher education: A systematic literature review. In Heliyon (Vol. 9, Issue 3). Elsevier Ltd. https://doi.org/10.1016/j.heliyon.2023.e13939
Pals, F. F. B., Tolboom, J. L. J., & Suhre, C. J. M. (2023). Development of a formative assessment instrument to determine students’ need for corrective actions in physics: Identifying students’ functional level of understanding. Thinking Skills and Creativity, 50. https:// doi.org/10.1016/j.tsc.2023.101387
Pambayun, N. A. Y., Munadi, S., Arifin, Z., Setiawan, C., & Retnawati, H. (2023). Industrial work culture education in Indonesian vocational high schools: Teachers’ perceptions and practices. Issues in Educational Research, 33(2), 713–732. https://search.informit.org/ doi/abs/10.3316/informit.303109822674581
Pratama, R. A., Pratiwi, I. M., Saputra, M. A., & Sumargono. (2022). Integration of STEM education in history learning. International Journal of Evaluation and Research in Education, 11(1), 313–320. https://doi.org/10.11591 /ijere.v11i1.22064
Prianto, A., Winardi, W., & Qomariyah, U. N. (2021). The effect of the implementation of teaching factory and its learning involvement toward work readiness of vocational school graduates. International Journal of Instruction, 14(1): 283–302. DOI: 10.29333/iji.2021.14117a
Presidential Regulation No. 8 of 2012 concerning the Indonesian National Qualifications Framework.
Randhawa, J. S., & Ahuja, I. S. (2017). 5S - A quality improvement tool for sustainable performance: Literature review and directions. International Journal of Quality & Reliability Management, 34(3), 334–361. https://doi.org/10.1108/ IJQRM-03-2015-0045
Putra, B. J. (2023). Study of stake evaluation model in the interest of program evaluation. Jurnal Sosial Humaniora Sigli. 6(1), 220–223. https://doi.org/10.47647/jsh.v6i1.1489
Rezkilaturahmi, R., Danni, R., & Retnawati, H. (2024). Investigating test equating methods: English examination test. International Journal of Education and Curriculum Application 7(3), 246–257. DOI:10.31764/ijeca.v7i3.25065
Rezkilaturahmi, R., Istiqlal, M., Putro, N.H.P., & Istiyono, E. (2025). Evaluating English language test items developed by teachers: An item response theory approach. VELES Voice of English Language Education Society, 9(1), 218–230. DOI:10.29408/veles.v9i1.27644
Santoso, A., Retnawati, H., Pardede, T., Apino, E., Rafi, I., Rosyada, M., Kassymova, G. & Wenxin, X. (2024). From investigating the alignment of a priori item characteristics based on the CTT and four-parameter logistic (4-PL) IRT models to further exploring the comparability of the two models. Practical Assessment, Research, and Evaluation. 29(1). https://doi.org/10.7275/pare.2043
Retnawati, H., Kardanova, E., Sumaryanto, Prasojo, L. D., Jailani, Arliani, E., Hidayati, K., Susanti, M., Lestari, H. P., Apino, E., Rafi, I., Rosyada, M. N., Tuanaya, R., Dewanti, S. R., Sotlikova, R., & Kassymova, G. K. (2024). A Systematic review of the use of technology in educational assessment practices: Lesson learned and direction for future studies. International Journal of Robotics and Control Systems, 4(4), 1656–1693. https://doi.org/ 10.31763/ijrcs.v4i3.1572
Şahin, M. G., Yıldırım, Y., & Öztürk, N. B. (2023). Examining the achievement test development process in the educational studies. Participatory Educational Research, 10(1), 251–274. https://doi.org/10.17275/per.23.14.10.1
Setiawan, H. J., & Islami, N. (2020). Improving critical thinking skills of senior high school students using the problem based learning model. Journal of Physics: Conference Series, 1655(1). https://doi.org/10.1088/1742-6596/1655/1/012060
Sudira, P. (2018). Metodologi pembelajaran vokasional abad XXI: Inovasi, teori dan praksis. [21st Century vocational learning methodology: Innovation, theory and practice]. UNY Press.
Suhartanta, S., Efendi, Y., & Ramadhan, A. (2023, March). Needs assessment of competency for human resource development in vehicle wheel alignment workshop. In AIP Conference Proceedings (Vol. 2671, No. 1, p. 020025). AIP Publishing LLC. https://doi.org/10.1063/5.0115997
Thanabalan, T. V., Siraj, S., & Alias, N. (2015). Evaluation of a digital story pedagogical module for the indigenous learners using the stake countenance model. Procedia-Social and Behavioral Sciences, 176, 907–914.
Thorndike, E.L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4(1), 25–29. https://doi.org/10.1037/h0071663
Tosuncuoglu, I. (2018). Importance of assessment in ELT. Journal of Education and Training Studies, 6(9), 163. https://doi.org/10.11114 /jets.v6i9.3443
Tsai, F.-H. (2024). Development and evaluation of an Internet of Things project for preservice elementary school teachers. Sustainability, 16(17), 7632. https://doi.org/ 10.3390/su16177632
Wagner, F. L., Sudacka, M., Kononowicz, A. A., Elvén, M., Durning, S. J., Hege, I., & Huwendiek, S. (2024). Current status and ongoing needs for the teaching and assessment of clinical reasoning – an international mixed-methods study from the students' and teachers' perspective. BMC Medical Education, 24(1), 1–17. https://doi.org/10.1186/s12909-024-05518-8
Wiliam, D. (2011). What is assessment for learning? Studies in educational evaluation, 37(1), 3–14. https://doi.org/10.1016 /j.stueduc.2011.03.001vv
Zhang, D., & Hwang, G. J. (2023). Effects of interaction between peer assessment and problem-solving tendencies on students’ learning achievements and collaboration in mobile technology-supported Project-Based Learning. Journal of Educational Computing Research, 61(1), 208–234. https://doi.org/10.1177/07356331221094250
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
View My Stats