Automatic Short Answer Assessment Using The Cosine Similarity Method

Samsu Bahri(1*), Mugi Praseptiawan(2), Winda Yulita(3),

(1) STKIP Setia Budhi Rangkasbitung, Indonesia
(2) Institut Teknologi Sumatera, Indonesia
(3) Institut Teknologi Sumatera, Indonesia
(*) Corresponding Author

Abstract


In the learning process, most exams to assess learning achievement have been carried out by providing questions in the form of short answers or essay questions. The variety of answers given by students makes a teacher have to focus on reading them. This process of assigning grades is difficult to guarantee quality if done manually. Moreover, each class is mastered by a different teacher, which can cause inequality in the grades obtained by students due to the influence of differences in teacher experience. Therefore the automated answer assessment research was developed. The automatic short answer assessment is designed to automatically assess and evaluate students' answers based on a trained set of answer documents.  The automated grading system uses the cosine similarity method to determine the degree of similarity of a student's answer to the teacher's answer. While the word weighting used is the Term Frequency-Inverse Document Frequency (TF-IDF) method.  The data used is a question totaling 5 questions with each question answered by 30 students, while the students' answers are assessed by experts to determine the real value. This study was evaluated by Mean Absolute Error (MAE) with the resulting value of 0.22.

Full Text:

PDF

References


H. Sun and Z. Shao, "Research on the Application of Students' Answered Record Analyze Model and Question Automatic Classify Based on K-Means Clustering Algorithm," 2019 10th International Conference on Information Technology in Medicine and Education (ITME), 2019, pp. 494-497, doi: 10.1109/ITME.2019.00116.

Z. Ke and V. Ng, "Automated essay scoring: A survey of the state of the art", Proc. 28th Int. Joint Conf. Artif. Intell., pp. 6300-6308, Aug. 2019.

G. Lv, W. Song, M. Cheng and L. Liu, "Exploring the Effectiveness of Question for Neural Short Answer Scoring System," 2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC)2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC), 2021, pp. 1-4, doi: 10.1109/ICEIEC51955.2021.9463814.

H. A. Abdeljaber, "Automatic Arabic Short Answers Scoring Using Longest Common Subsequence and Arabic WordNet," in IEEE Access, vol. 9, pp. 76433-76445, 2021, doi: 10.1109/ACCESS.2021.3082408.

T. -H. Chang, J. -L. Chen, H. -M. Chou, M. -H. Bai, F. -Y. Hsu and Y. -C. Chen, "Automatic Scoring Method of Short-Answer Questions in the Context of Low-Resource Corpora," 2021 International Conference on Asian Language Processing (IALP), 2021, pp. 25-29, doi: 10.1109/IALP54817.2021.9675160.

U. Hasanah, T. Astuti, R. Wahyudi, Z. Rifai and R. A. Pambudi, "An Experimental Study of Text Preprocessing Techniques for Automatic Short Answer Grading in Indonesian," 2018 3rd International Conference on Information Technology, Information System and Electrical Engineering (ICITISEE), 2018, pp. 230-234, doi: 10.1109/ICITISEE.2018.8720957.

A. Mishra and S. Vishwakarma, "Analysis of TF-IDF Model and its Variant for Document Retrieval," 2015 International Conference on Computational Intelligence and Communication Networks (CICN), 2015, pp. 772-776, doi: 10.1109/CICN.2015.157

M. Ramya and J. A. Pinakas, "Different type of feature selection for text classification", International Journal of Computer Trends and Technology, vol. 10, pp. 102-107, 2014

A. N. Khusna and I. Agustina, "Implementation of Information Retrieval Using Tf-Idf Weighting Method On Detik.Com’s Website," 2018 12th International Conference on Telecommunication Systems, Services, and Applications (TSSA), 2018, pp. 1-4, doi: 10.1109/TSSA.2018.8708744

Nazief, B. A. A. dan M. Adriani. 1996. Confix-Stripping : Approach to Stemming Algorithm for Bahasa Indonesia. International Conference on Information and Knowledge Management, : 560-563

Stephen Robertson, "Understanding Inverse Document Frequency: On Theoretical Arguments for IDF", England Journal of Documentation, vol. 60, pp. 502-520, 2005.

Patidar, A. K., J. Agrawal dan N. Mishra. 2012. Analysis of Different Similarity Measure Functions and Their Impacts on Shared Nearest Neighbor Clustering Approach. International Journal of Computer Applications. 40(16): 1-5.

W. Wang and Y. Lu, "Analysis of the Mean Absolute Error (MAE) and the Root Mean Square Error (RMSE) in Assessing Rounding Model", IOP Conference Series: Materials Science and Engineering, 2018.




DOI: http://dx.doi.org/10.30645/j-sakti.v7i1.608

Refbacks

  • There are currently no refbacks.


J-SAKTI (Jurnal Sains Komputer & Informatika)
Published Papers Indexed/Abstracted By:

Jumlah Kunjungan :

View My Stats