SPMC Journal of Health Care Services

Southern Philippines Medical Center COVID-19 Updates

RESEARCH

Relationships of training examination results and patient load to certifying examination results in a Surgery residency training

SPMC J Health Care Serv. 2015;1(1):9-13. ARK: http://n2t.net/ark:/76951/jhcs3ns64f

Mauro Bravo III,1, 2, 3 Bernardo Ang1


1Davao Regional Hospital, Apokon, Tagum City, Philippines;
2Democrito O Plaza Memorial Hospital, Prosperidad, Agusan del Sur;
3San Francisco Doctors Hospital, Brgy Hubang, National Highway, San Francisco, Agusan del Sur


Correspondence Mauro Bravo III, maui_brux00@yahoo.com

eceived 17 April 2015

Accepted 24 July 2015

Cite as Bravo M III, Ang B. Relationships of training examination results and patient load to certifying examination results in a Surgery residency training program. SPMC J Health Care Serv. 2015;1(1):9-13. http://n2t.net/ark:/76951/jhcs3ns64f


Abstract

Background. In the Philippines, Surgery residents-in-training are assessed annually for proficiency through residency in-service training examinations (RITE). Specialty certification, however, is contingent upon the results of the Surgery Certifying Examinations (SCE).

Objective. To determine the relationships of Surgery residents’ annual RITE performance and patient load with the results of their written and oral SCE.

Design. Retrospective cohort study.

Setting. Davao Regional Hospital in Tagum City and Southern Philippines Medical Center in Davao City, Philippines.

Participants. Thirty-eight residents who graduated from the Mindanao Integrated Surgical Residency Training Program (MISRTP) and took the SCE in 2000-2012.

Main outcome measures. Mean percentage of RITE passed, mean annual patient load per resident, SCE performance, and odds ratios (95% CI) of passing the SCE.

Main results. Resident physicians passed 76.32 ± 18.48% of the RITE that they took, and performed a mean of 441 ± 52 surgical operations per resident per training year. Passing rates on first attempt were 25/38 (65.79%) for written SCE and 13/24 (54.17%) for oral SCE. Passing ≥75% of the total number of RITE significantly increased the odds ratio of passing the written SCE (OR=24.44, 95% CI 4.18 to 142.95, p=0.0004), but not the odds ratio of passing the oral SCE. Having a higher than average patient load did not significantly change the odds ratio of passing any SCE.

Conclusion. Good performance in the RITE increased the odds ratio of passing the written, but not the oral, SCE. Patient load did not significantly affect the odds ratios of passing the written or oral SCE.


Keywords. specialty certification, proficiency assessment, written examination, oral examination


Introduction

In 1991, the Philippine College of Surgeons (PCS) formulated a standardized surgical curriculum that was subsequently implemented among surgery residency training programs throughout the country.1 The Philippine Society of General Surgeons (PSGS) assumed the responsibility of overseeing the implementation of the PCS curriculum and accrediting the training programs from PCS in 2002.1

The Mindanao Integrated Surgical Residency Training Program (MISRTP), a consortium of the originally independent surgical residency programs of Southern Philippines Medical Center (SPMC) and Davao Regional Hospital (DRH), has been an accredited training program since 2000. It is the largest training program in General Surgery in Mindanao. MISRTP has produced competent surgeons in General Surgery, as well as in surgical subspecialties within the field. However, like other training programs all over the country, not all its graduates fare well in the specialty certifying examinations.

In the Philippines, residents in Surgery take the annual written residency in-service training examinations (RITE) as part of their proficiency assessment. Residents who graduate from Surgery programs usually appraise their preparedness and ability to pass the surgery certifying examinations (SCE) from their past performances in the RITE. Both the RITE and the SCE are given by the Philippine Board of Surgery (PBS), a body created by the PCS to formulate and organize examinations for residents-in-training and for graduates who are eligible to take the SCE.

The SCE is a two-part examination. The first part is a written examination, which is similar to the RITE. In the second part, which is an oral examination, examinees answer questions based on theoretical surgical cases. The examinees' performance in the oral examination is assessed by a panel of examiners. An examinee must pass both the written and oral examinations by the PBS in order to become a board-certified surgeon.

In-training examinations have been investigated for their ability to predict performance in post-residency certifying examinations, with varying results2 3 4 Work conditions have also been known to influence the performance and proficiency of surgeons.5 6 Surgery residency training in SPMC and DRH, as in many service-oriented government training hospitals, is characterized by high patient load because these hospitals are the end-referral centers in Southeastern Philippines.

We were interested to know whether the training conditions in MISRTP are adequately preparing the residents for the national SCE. Hence, we did this study to determine the relationships of Surgery residents' annual RITE performance and annual patient load with the results of their written and oral SCE.


Methods

Study design and setting
We did a retrospective cohort study by reviewing available records on the annual RITE performance and workload of residents in Surgery, and the subsequent results of their SCE. The study was done within the MISRTP. In the program, Surgery residents typically rotate in two hospitals in Southeastern Philippines. The first hospital, SPMC, is the only tertiary government hospital in Davao City with a 1,200-bed capacity, shared by 13 medical specialty departments, including Surgery. The second hospital, DRH, is located in Tagum City and is 60 kilometers away from SPMC. DRH has a 400-bed capacity, with five other medical specialty departments, apart from Surgery. Surgical residents in the program provide tertiary level care and perform a total of 16,000 emergency and elective operations annually. A training board, composed of consultant specialists from both hospitals, is responsible for planning and implementing activities that promote academic and skills learning among residents, and for regularly evaluating the residents' performance within the MISRTP.

Participants
For this study, we included all residents who graduated from the MISRTP from 2000 to 2012 and took the SCE given by the PBS. A total of 40 residents graduated from the training program, but two of them did not take the written SCE. Of the 38 who took the written SCE, only 24 took the oral SCE.

Data collection
The primary outcome measures in this study were mean percentage of RITE passed, mean annual patient load per resident, SCE performance on first take, and odds ratios (95% CI) of passing the SCE on first take. We obtained data on the annual RITE results from the records of the individual Surgery departments, and the results of the oral and written SCE from the records of the PCS. Data on annual patient load were taken from the annual census of surgical procedures done by MISRTP residents, as annually reported by the program to the PSGS.

We took the annual RITE ratings of each resident and compared each rating to the minimum passing level set by the PBS during the year that the RITE was taken, in order to determine whether the resident passed the RITE during a particular year. We then determined the individual resident's percentage of RITE passed by dividing the number of RITE passed by the total number of RITE taken and multiplying the quotient by 100.

To determine patient load per resident per year, we took the total number of surgical procedures done by the MISRTP residents each year and allocated the sum equally to all training residents for that year. We took the average annual patient load for each resident by taking into account all their annual loads during residency training.

For the SCE performance, we reflected the passing or failure status of each resident during the first attempt to take the written or the oral SCE. Since the numbers of written SCE takers and oral SCE takers were different, we expressed the outcomes of the written SCE and the oral SCE separately in the results.

Statistical analysis
We summarized continuous variables using means and standard deviations, and categorical variables using frequencies and percentages. We predetermined that a 'good performance in RITE' should be any percentage of RITE passed equal to or higher than 75%. We also classified residents with higher than average annual patient load as having 'high patient load during residency.' We used logistic regression to determine the odds ratios (95% CI) of passing the written SCE and the oral SCE on first take for 'good performance in RITE' and 'high patient load during residency.' A two-sided p-value of <0.05 was considered statistically significant.


Results

We included 38 residents who graduated from MISRTP and who eventually took the written and/or the oral SCE in this analysis. The characteristics of the residents are shown in Table 1. Most of the residents (33/38, 86.84%) were males. The residents passed a mean of 76.32 ± 18.48% of the RITE that they took during residency, and performed a mean of 441 ± 52 surgical procedures per resident per year. Of the 38 residents who took the written SCE, 25 (65.79%) passed the examination on first attempt. Of the 24 residents who subsequently took the oral SCE, 13 (54.17%) passed the examination on first attempt.


Table 1   Characteristics of Surgery residents

Characteristiscs
Values
n=38*
Sex distribution, frequency (%)
   Female 5 (13.16)
   Male 33 (86.84)
Mean annual patient load per resident ± SD, patients 441 ± 52
Mean percentage of RITE passed ± SD, % 76.32 ± 18.48
Passed written SCE on first take, frequency (%) 25 (65.79)
Passed oral SCE on first take, frequency (%) (n=24) 13 (54.17)
*Unless specified otherwise.


Table 2 shows the odds ratios (95% CI) of passing the certifying examination on first take. Having a 'good performance in the RITE' significantly increased the odds ratio of passing the written SCE on first take (OR=24.44, 95% CI 4.18 to 142.95, p=0.0004), but not the odds ratio of passing the oral SCE on first take (OR=1.22, 95% CI 0.14 to 10.48, p=0.8548). Having a 'high patient load during residency' did not significantly change the odds ratio of passing either the written SCE (OR=2.59, 95% CI 0.46 to 14.51, p=0.2796) or the oral SCE (OR=0.10, 95% CI 0.01 to 1.06, p=0.0558).


Table 2   Odds ratios (95%) of passing the written and oral SCE on first take

Variables Passing written SCE on first take Passing oral SCE on first take
OR (95% CI) p-value OR (95% CI) p-value
Good performance in the RITE† 24.44 (4.18 to 142.95) 0.0004* 1.22 (0.14 to 10.48) 0.8548
High patient load during residency‡ 2.59 (0.46 to 14.51) 0.2796 0.10 (0.01 to 1.06) 0.0558
*Statistically significant; †resident passed ≥ 75% of total number of RITE taken; ‡mean annual surgical operations >441 patients (average). RITE — residency in-service training examinations; SCE — surgery certifying examinations.


Discussion

Key results
We found out that 'good performance in the RITE' increased the odds ratio of passing the written SCE, but did not affect the odds ratio of passing the oral SCE. Further, 'high patient load during residency' did not significantly change the odds ratio of passing any of the SCE.

Strengths and limitations
This study provides evidence regarding the relationships of academic exercises in the form of RITE and patient load during residency, with outcomes of certifying examinations that residents take after training. We believe that this is the first systematic inquiry on the factors that affect SCE results among residents who graduated from the MISRTP.

In measuring proficiency during residency, we only considered the performance of residents in the RITE. Several other quantified measures of proficiency, such as quarterly evaluations, or annual marks considered in the decision to promote residents to the next level of training were, in fact, obtainable but were not accounted for in this study. Our study was also limited in the sense that, in quantifying patient load during residency, we only took into consideration the number of surgical procedures done during training. Performing non-surgical procedures like admitting patients, doing rounds, doing pre-operative and post-operative care, etc., may also take up a considerable amount of the residents' time and effort, which, in turn, may positively or negatively affect their overall proficiency in Surgery.

Interpretation
Associations between in-training examinations and certifying examinations in Surgery residency have been have been explored in the past.2 3 4 7 However, these studies reported different conclusions, perhaps because of the diverse practices of conducting and evaluating residency training programs.

Our underlying goal for doing this study was to build evidence for or against the way that Surgery residents are being prepared during residency to tackle the SCE, which is an important milestone in their practice. The fact that this study demonstrates that good performance in the RITE is associated with a positive outcome in SCE may mean that the two sets of examinations given by the PBS are similar. In a way, the RITE do prepare the residents for the written SCE. The whole system works because would-be takers of the written SCE can simulate taking the certifying examinations by going through the RITE during training.

It is a common notion among residents that high work load during training gives them less time to study and usually translates into poor performance in examinations. However, the results of this study, as well as those of previous studies, showed otherwise - that the volume of surgical procedures during training does not affect the performance of residency graduates during their certifying examinations.8 9

Generalizability
While on training, residents need to prepare for the certifying examinations, which will eventually assess their competence in becoming surgical specialists. The RITE adequately prepares residents in Surgery for the written certifying examination only. Since the oral portion of the SCE is as important as the written examination, training programs should consider introducing learning activities in residency that would enhance residents' skills in verbally articulating their academic proficiency in Surgery. Modes of instruction that can be considered for this purpose include grand rounds and mock oral examinations. The aim should be to provide during training a learning activity that is comparable to the setup of the oral portion of the certifying examinations.

Conclusion

Performing well in the RITE during training significantly increased the odds ratio of passing the written SCE. However, performance in the RITE was not significantly associated with oral SCE results. Moreover, patient load during residency was not significantly associated with written or oral SCE results.

In essence

Graduates from Surgery residency training must pass both the written and the oral certifying examinations in order to become qualified surgeons.


In this study, in-training examinations prepare residents for the written portion of the certifying examinations only.


Trainers can consider introducing learning activities during residency that adequately prepare residents for the oral certifying examination.


Acknowledgments

We thank the members, the training staff, and consultants of the Mindanao Integrated Surgical Residency Training Program, and Dr Alvin S Concha for their inputs and guidance in this research. We also thank the Philippine College of Surgeons for granting us access to their data on certifying examination results of the MISRTP graduates.


Ethics approval

This study was reviewed and approved by the Department of Health XI Cluster Ethics Review Committee (DOHXI CERC reference P13112701)..


Reporting guideline used

STROBE Checklist (http://www.strobe-statement.org/fileadmin/Strobe/uploads/checklists/STROBE_checklist_v4_combined.pdf)


Article source

Submitted


Peer review

External


Funding

Supported by personal funds of the authors


Competing interests

None declared


Access and license

This is an Open Access article licensed under the Creative Commons AttributionNonCommercial 4.0 International License, which allows others to share and adapt the work, provided that derivative works bear appropriate citation to this original work and are not used for commercial purposes. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/


References

1. Philippine Society of General Surgeons. Standardized Curriculum in General Surgery 4th edition. 2012.


2. De Virgilio C, Yaghoubian A, Kaji A, Collins JC, Deveney K, Dolich M, et al. Predicting performance on the American Board of Surgery qualifying and certifying examinations: a multi-institutional study. Arch Surg. 2010;145(9):852-6.


3. Corneille MG, Willis R, Stewart RM, Dent DL. Performance on brief practice examination identifies residents at risk for poor ABSITE and ABS qualifying examination performance. J Surg Educ. 2011;68(3):246-9.


4. Woloschuk W, Mclaughlin K, Wright B. Predicting performance on the Medical Council of Canada Qualifying Exam Part II. Teach Learn Med. 2013;25(3):237-41.


5. Brandenberger J, Kahol K, Feinstein AJ, Ashby A, Smith M, Ferrara JJ. Effects of duty hours and time of day on surgery resident proficiency. Am J Surg. 2010;200(6):814-8.


6. Friedlaender GE. The 80-hour duty week: rationale, early attitudes, and future questions. Clin Orthop Relat Res. 2006;449:138-42.


7. Grewal SG, Yeung LS, Brandes SB. Predictors of success in a urology residency program. J Surg Educ. 2013;70(1):138-43.


8. Sturm L, Dawson D, Vaughan R, Hewett P, Hill AG, Graham JC, et al. Effects of fatigue on surgeon performance and surgical outcomes: a systematic review. ANZ J Surg. 2011;81(7-8):502-9.


9. Lehmann KS, Martus P, Little-elk S, Maass H, Holmer C, Zurbuchen U, et al. Impact of sleep deprivation on medium-term psychomotor and cognitive performance of surgeons: prospective cross-over study with a virtual surgery simulator and psychometric tests. Surgery. 2010;147(2):246-54.


Copyright © 2015 Bravo M III, et al.


Published
November 15, 2015

Issue
Volume 1 Issue 1 (2015)

Section
Research




SPMC Journal of Health Care Services


           

Copyright © 2019 Southern Philippines Medical Center.
ISSN (Print): 2012-3183. ISSN (Online): 2467-5962
All rights reserved.