|
Introduction
According to the regulations established in 2002, the Thai Medical Council requires that all Thai medical students who matriculated as of academic year 2003 pass the Medical Licensing Examination of the Thai Medical Council in order to be a licensed physician in Thailand. There are three steps to the examination. Step 1, called comprehensive basic medical science, centers on the subjects covered in the pre-clinical curriculum (anatomy, biochemistry, epidemiology, microbiology, pathology, pharmacology, and physiology). Thai medical students sit for the Medical Licensing Examination of Thailand (MLET) Step1 which is a paper-based 300-MCQ examination covering basic science subjects at the end of their third academic year. The Step 2 examination, which is a paper-based 300-MCQ examination covering clinical science subjects, would be held at the end of their fifth academic year. The Step 3 examination, includes OSCE (objective structural clinical examination) covering clinical skills, would be held at the end of their sixth academic year.
The faculty of Medicine, Mahasarakham university, delivers a 6-year medical curriculum. High-school graduates qualify for admission by the Mahasarakham University Entrance Examination (MUEE) and an interview process. The first 3-year (pre-clinical) courses included several disciplines relating to basic medical science. Community medicine and introduction to clinical medicine are introduced during the second semester of the third academic year. The students sit for the MLET Step1 at the end of their third year.
This report describes the Mahasarakham medical students results in their first attempt sitting for the MLET Step1 examination and factors predicting their success or failure
Materials and Methods
MLET Step 1 tests the medical students knowledge in a problem-solving framework using clinical vignettes. It is a 1-day, 6-hour, multiple-choice examination (MCQ) divided into two blocks (each block containing 150 questions). The examination items are created in accordance with the Thai Medical Councils Table of Specification on Basic Science Subjects, similar to that of the United States Medical Licensing Examination (USMLE) Step1. Subjects include anatomy, behavioral science, biochemistry, epidemiology, immunology, microbiology, pathology, pharmacology and physiology. The examination items are developed by all Thai medical school faculties and pooled at the meeting of the Thai Medical Council Committee. The committee finally selects the 300 questions for the examination and establishes the minimum passing score based on the examination results. Three hundred and forty of the third-year Mahasarakham medical students (male : female = 126 : 214) who had first attempt for MLET Step1 since 2006 to 2015. Objective data on demographic factors (age, gender and residence), high school grade point average (GPA), Mahasarakham University Entrance Examination (MUEE) and GPAs at year 1, year 2 and year 3 in medical school were retrieved from the Medical Education Unit.
The system of grading for GPA is as follows:
A = grade 4.00, B+ = grade 3.50, B = grade 3.00, C+ = grade 2.50,
C = grade 2.00, D+ = grade 1.50, D = grade 1.00, F = grade 0.00
The GPA is averaged from each clerkship grades (weighted with the number of credits of the clerkship). Each clerkship grade is obtained by multiplying the grade with the number of credits of that clerkship.
The result of the Medical Licensing Examination Step1was obtained from the Thai Medical Council after the examination. Logistic regression analysis was performed with SPSS 15.0 software.
The logistic regression model was constructed using the forward selection procedure in an attempt to discover the predictors of MLET Step1 success and failure. At each step, the explanatory variable with the smallest significance predictors of MLET Step1 success and failure. At each step, the explanatory variable with the smallest significance level for the Wald statistic was entered into the model. The Wald statistics is a method in logistic regression to test the null hypothesis (H0) that the associate parameter estimates are not significantly different from 0. The default entry criterion for the explanatory variables was a p-value of 0.05. The Wald statistics for all variables in the model were examined and the explanatory variable with the largest p-value of the Wald statistic was removed from the model. The default removal criterion was p = 0.10. If no explanatory variables met the removal criterion, the next eligible variable was entered into the model. The iteration process for selecting explanatory variables continued until no additional variables met the entry or removal criterion.
Results
The passing score set by the Thai Medical Council, based on the minimum passing level and the standard error of measurement. Two hundred and twenty-nine out of 340 (67.4%) Mahasarakham medical students passed the examination.
Predictors of the pass/fail groups are shown in Table 1.The passing rate was slightly higher in the younger age group, male sex and Mahasarakham residence. All students with year-2 GPA of < 2.5 and most of the students with year-3 GPA of < 2.5 failed the examination.
The logistic regression method yielded the following logistic regression equation to predict the MLET Step1 passing status: the estimated probability of passing the MLET Step1 was:
P(X) = ez / (1+ ez)
where e is the base of the natural logarithm, approximately 2.718, and
Z = -14.37 + 0.26*GPAyr1+ 2.54*GPAyr2 + 2.87*GPAyr3
Based on the contribution from each of the explanatory variables, the estimated probability could be derived from this equation for an individual student. If the calculated probability was ≥ 0.5, a student was categorized in the passing group of the MLET Step1. On the contrary, those with a probability of < 0.5 would be classified in the fail group. The prediction accuracy of this equation was as follows: 90% for the pass group, 62.1% for the fail group and 84.1% for the combined pass and fail group (Table 2). The receiver operating curves is shown in Figure 1. The performance of the model was relatively good with an area under curve (AUC) of 0.84.The logistic enter (all variables) analysis serving as a benchmark for the stepwise model yielded similar result.
Table 1 Predictors of the pass/fail groups
|
Predictors |
Pass group
n=229(67.4%) |
Fail group
n=111(32.6%) |
|
Age (mean ± SD) |
21.97 ± 0.66 |
22.28 ± 0.76 |
|
Male (%) |
72.20 |
27.80 |
|
Female (%) |
64.48 |
35.52 |
|
Residence: Mahasarakham (%) |
68.13 |
31.87 |
|
Residence: non-Mahasarakham (%) |
67.07 |
32.93 |
|
Entrance examination score (mean ± SD) |
55.68 ± 4.35 |
53.98 ± 4.36 |
|
GPA year-1 (mean ± SD) |
3.72 ± 0.19 |
3.55 ± 0.23 |
|
GPA year-2 (mean ± SD) |
3.39 ± 0.26 |
3.05 ± 0.27 |
|
GPA year-3 (mean ± SD) |
3.32 ± 0.28 |
2.86 ± 0.26 |
|
GPA year-1: < 3.0 from 4.00 scale (%) |
20 |
80 |
|
GPA year-2: < 2.5 from 4.00 scale (%) |
0 |
100 |
|
GPA year-3: < 2.5 from 4.00 scale (%) |
10 |
90 |
GPA: grade point average, SD: standard deviation
Table 2 Logistic Regression Model for Predicting MLET step1 Pass Status
|
Variables in the equation |
Logistic regression coefficient (β) |
SE (β) |
OR (e β ) |
95%CI of OR |
|
GPA-year 1 |
0.26* |
0.15 |
1.30 |
0.65-14.86 |
|
GPA-year 2 |
2.54** |
1.06 |
12.68 |
1.60-95.72 |
|
GPA-year 3 |
2.87** |
1.09 |
17.64 |
2.99-135.24 |
|
Constant |
-14.37** |
3.42 |
0.000 |
|
95%CI: 95% confidence interval; GPA: grade point average; OR: odds ratio
MLET: Medical Licensing Examination of Thailand; SE: standard error
*p< 0.05, **p< 0.01
Figure 1 Receiver operating curve (ROC)
Discussion
We were dissatisfied with the performance of our medical students in their first attempt sitting for the MLET Step1.
The failure rate of 32.6% prompted us to search for factors influencing success and failure in the examination.The objective data collected for demographic characteristics were age, gender and residence.
Our study found no difference on MLET step1 performance by age. Our medical students ages were close, ranging from 21 to 24 years, which probably explained the non-significant difference in their performance. This finding was consistent with a study carried out by Ramsbottom-Lucier et al1. Of particular note was the matriculation ages of their students, which ranged from less than 23 to 28 years or older. They also noted a modest gender difference on the NBME I result, with males performing better than females. In the new era of USMLE, Case et al2 also reported that males outperformed females in the Step1 examination. Our study also found that males outperformed females in the Step1 examination however there was no statistical significance. The pass rates of our male and female students were 72.2% and 64.5%, respectively. Haist et al3 explored the interaction between gender and age. They found a significant gender effect on age in predicting academic performance. We did not study this interaction due to the narrow age range of our students.
Our study found no differences on the MLET Step1 performance by the students residences. Almost of our medical students residences were rural backgrounds which probably explained the non-significant difference in their performance. This finding was different from a Croatian study carried out by Polasek and Kolcic4 and a Thai study carried out by Samkaew and Supavadee5where students from urbanized backgrounds outperformed those from rural backgrounds. They explained by the nature of developing countries, where access to knowledge and information is markedly different in urban and remote areas. The higher standard of teaching and extra lessons in highly urbanized high schools may have enhanced the students critical thinking skills, resulting in better performance in examinations.
GPAs and scores have been extensively investigated in relation to examination outcome. Veloski et al6 found that MCAT scores and science GPA were good predictors of USMLE Step1 performance. Basco et al7 and Kasuya et al8 reported similar results. Our study included high school GPA, Mahasarakham University Entrance Examination (MUEE) and undergraduate GPAs. It was evident that high school GPA was not correlated with MLET Step1 performance since marking and grading systems in our high schools were still not standardized. MUEE score was not a significant predictor because the subjects tested were purely science subjects e.g. chemistry, physics, biology and mathematics. The freshmen, sophomore and third-year GPAs significantly predicted MLET Step1 results. When the freshmen, sophomore and third-year GPAs increased by an average of 1 point, the odds of passing the MLET Step1 increased by factors of 1.3, 12.7 and 17.6 respectively. The freshmens GPA could predict examination performance less significant (p< 0.05) than second and third-year GPAs (p< 0.01) because it is derived mainly from the science subjects. The obvious reason was that the subjects taught in the second and third years were mainly basic medical sciences compatible with the Table of Specification of the examination. These values indicate that the effect was very significant. This implies that there should be prompt intervention or tutorials of students with low GPAs in their first and second years. Delaying such intervention until the outcome of third-year GPA may be too late and result in a high possibility of failing the MLET Step1 examination.
Our study was retrospective so the limitation was some datas lost for example high school GPA and birth date.
In conclusion, our study found that Thai students low freshmen and sophomore GPAs were at risk of performing poorly on the MLET Step1 and required intensive academic supervision to prevent unsatisfactory outcomes in the medical licensing examination.
References
1. Ramsbottom-Lucier M, Johnson MM, Elam CL. Age and gender differences in students preadmission qualifications and medical school performances. Acad Med 1995; 70: 236-9.
2. Case SM, Swanson DB, Ripkey DR, Ripkey DR, Bowles LT, Melnick DE. Performance of the class of 1994 in the new era of USMLE. Acad Med 1995; 71 (Suppl): S91-S93.
3. Haist SA, Wilson JF, Elam CL, Blue AV, Fosson SE. The effect of gender and age on medical school performance: an important interaction. Adv Health Sci Educ Theory Pract 2000; 5: 197-205.
4. Polasek O, Kolcic I. Academic performance and scientific involvement of final-year medical students coming from urban and rural backgrounds. Rural Remote Health 2006; 6: 530.
5. Samkaew Wanvarie, Supavadee Prakunhungsit. Logistic regression analysis to predict medical licensing examination of Thailand (MLET) Step1 success or failure. Ann Acad Med Singapore 2008; 37: 1024-6
6. Veloski JJ, Callahan CA, Xu G, Hojat M, Nash DB. Prediction of students performances on licensing examinations using age, race, sex, undergraduate GPA, and MCAT scores. Acad Med 2000; 75(Suppl): S28-S30.
7. Basco WT Jr, Way DP, Gilbert GE, Hudson A. Undergraduate institutional MCAT scores as predictors of USMLE step1 performance. Acad Med 2002; 77(Suppl): S13-S16.
8. Kasuya RT, Naguwa GS, Guerrero AP, Hishinuma ES, Lindberg MA, Judd NK. USMLE performances in a predominantly Asian and Pacific Islander population of medical students in a problem-based learning curriculum. Acad Med 2003; 78: 483-90.
|
