Item-analysis of the multiple-choice questions used in the formative assessment of introductory posting examination in Medicine and Surgery at a medical university in Southern Nigeria.

Abstract
Introduction: The aim of this study was to undertake quality assurance as post-examination analysis of multiple-choice questions (MCQs) used in formative assessment. Materials & Methods: Classical Test theory (CTT) and Item Response Theory (IRT) of 500 items (100 keys & 400 distractors) in single-best answer MCQs (A-type) in introductory medicine (IM) and introductory surgery (IS) from 62 medical students was done post-examination. Anonymised answer-scripts had item responses made binary as 0 and 1 and analysed using Microsoft Excel spreadsheet&JMetrik psychometric software to determine difficulty index, discrimination index, distractor efficiency and Cronbach's alpha coefficient.

Results: The mean score in IM was 60.83 ± 9.48 (95% C.I. 58.42 – 63.24)] Fifty-four students (87.10%) [95% C.I. 76.15 – 94.26] passed and 8 (12.9%0 [95% C.I 5.74 - 23.85] failed. Thirteen (20.7%) attained a score of 70 and above. The mean score in IS was 63.5 ± 7.1 (95% C.I 61.70 – 65.32). Sixty (96.77%) [95% C.I 88.8 3– 99.61] passed, and 2 (3.23%) [95% C.I 0.39 – 11.17] failed. Twelve (19.3%) scored 70 and above. Difficulty index (DIF-I) of keys set at <0.3 (too hard) was 10% in IM & 14% in IS. DIF-I > 0.8 (too easy) was 22% in IM and 40% in IS respectively. Discrimination index (DI) of keys set <0.1 (poor) was 44% in IM & 48% in IS and >0.3(good) was 10% in IM & 2% in IS respectively. The Cronbach's alpha coefficient was 0.62 in IM & 0.45 in IS respectively. Nonfunctioning and ineffective distractors (NFD) with a score of zero (0) was 28.8% in IM & 45.2% in IS respectively.

Conclusion: Item analysis in this study showed many easy questions with poor discrimination, low reliability index and poor distractor efficiency. We recommend post-examination item-analysis as part of quality assurance matrix after formative assessment.

KEYWORDS: item-analysis, discrimination index, difficulty index, distractor efficiency, key assignment, formative assessment