For the initial study in 2011 - 2012, 246 questionnaires were completed for both inpatient and outpatient; however, some were not completed leaving 120 inpatient DREEMs and 109 outpatient DREEMS. In 2012 - 2013, 111questionnaires were completed for the inpatient experience, but one was incomplete leaving 110 for analysis. In 2013 - 2014, 102 questionnaires were completed for the outpatient experience.
4.2. Phase1 Results
During the 2011 - 2012 year, students completing the inpatient rotation spent two weeks assigned to a floor at the children’s hospital or at the university hospital inpatient unit. DREEM subscales were reviewed and were not statistically different between the rotation sites (children’s floor 4, children’s floor 5, and children’s floor 6, University tower). Similar results occurred in an analysis of the various outpatient clinic rotations (university clinic, Omaha private clinic, greater Nebraska private clinic). Although we hypothesized times of the year when children are sicker may impact perceptions of the learning environment, there were no significant differences based on the time of year students completed the clerkship.
Pilot data themes focused on clarifying expectations for the rotations. “Clearer objectives on what students are to do and what patients we are to see”, outpatient clinic student comment. This theme extended to the inpatient wards at children’s hospital, particularly with respect to rounding on Saturdays. “Regarding the Saturday assignment for children’s inpatient, each of the students on the 4
th, 5 th, and 6 th floors was given different instructions from residents, showed up at different times and the supervising resident that day didn't know what to do with us!". Perhaps establish what students should expect to do on Saturday (we saw patients but none got to present or even see faculty) ideas include seeing a particular patient to present, staying to take an intake.anything”. 4.3. Phase 2 Results
Based on annual college of medicine course evaluations, changes were made to the inpatient rotation in 2012 - 2013. Students still spent two weeks on inpatient wards, but they moved to a different children’s hospital floor or to the university hospital ward for a week at a time. With these changes it was anticipated that DREEM results may change.
Table 3 shows the mean subscale result differences from the first to the second year. The results remained positive in each DREEM subscale, although the means dropped somewhat after the changes were made. Upon analysis, DREEM subscales were not significantly different across sites. Pairwise comparisons with adjusted p-values showed there were no significant differences on all measured subscales (data not shown).
Table 3. Inpatient Dreem Results
Value 11/12 12/13 b CH UH K-W P CH UH K-W P 1. Student perception of learning 34.29 35.02 1.42 0.234 33.55 34.44 1.28 0.258 2. Student perception of course organizers 29.59 30.00 1.69 0.193 28.26 29.28 0.72 0.396 3. Students’ academic self-perception 22.75 22.09 0.47 0.495 22.43 22.94 0.79 0.375 4. Students’ perceptions of atmosphere 31.96 32.43 1.46 0.227 28.86 28.67 0.01 0.922 5. Students’ social Self-perceptions 17.96 18.29 0.38 0.540 17.71 18.22 0.47 0.495
aSubscale interpretations: 1, 0 - 12: very poor; 13-24: teaching is viewed negatively; 25-36: positive perception of teaching; 37 - 48: teaching highly thought of, 2, 0 - 11: abysmal; 12 - 22: in need of some retraining; 23 - 33: doing okay; 34 - 44: model course organizers, 3, 0 - 8: feelings of total failure; 9 - 16: many negative aspects; 17 - 24: more positive feelings; 25 - 32: confident, 4, 0 - 12: terrible environment; 13 - 24: many issues need changed; 25 - 36: More positive perception; 37-48: overall good feeling of environment, 5, 0 - 7: miserable; 8 - 14: dissatisfied; 15 - 21: not too bad; 22 - 28: very good socially.
b12/13 students spent 1 week at UH and 1 week at CH. In 11/12 they were assigned to UH for 2 weeks.
Five themes emerged from the analysis of the narrative comments about how to improve the inpatient rotation. Two themes were complementary, relating to expectations and scheduling. When students came on the service, there were not clear directions about what they should do in the morning or after rounds. The following was indicative of this theme: “We were never told on the first day what we should do on the floor and then were publicly called out for not doing enough on the first day of the rotation.” Although students eventually learn what is expected in the mornings, the structure of the remainder of the day remained unclear. “Daytime shifts could be more structured after the lunch hour. I always knew what was expected of me from 6 am to noon, after lunch was a toss-up.”
The other themes related to the educational experience, specifically logistics, teaching, and feedback. These themes also relate to the expectations and scheduling themes in that students were expected to prepare mini-topics but never had an opportunity to present them (logistics). The students suggested “more structured afternoon teaching sessions/learning on topics that are directly relevant to patients.” The students also sought specific feedback from the attending physicians about their performance on the wards.
4.4. Phase 3 Results
In January, 2013, the university outpatient clinic came under new management by a private physician group. The new clinic director had a goal to improve the patient care experience and education of the medical students in the clinic. We hypothesized the new management and emphasis on student education would result in higher DREEM ratings by the students.
Table 4 shows the results of the DREEM from 2011 - 2012 versus the 2013 - 2014 groups. The table summarizes the DREEM subscales for the sites. The results from the pilot year were more positive in all of the subscales across sites. “Social self-perception”, which relates to student wellness, indicated students were dissatisfied in both Omaha private clinics and the University clinic in 2013 - 2014. DREEM subscales were not significantly different across sites. Pairwise comparisons with adjusted P values showed there were no significant differences on all measured subscales (data not shown).
Table 4. Outpatient DREEM Results for University Clinic
Value 11/12 13/14 K-W P 1. Student perception of learning 33.06 33.39 0.34 0.561 2. Student perception of course organizers 32.88 33.16 0.02 0.890 3. Students’ academic self-perception 21.88 21.71 0.02 0.883 4. Students’ perceptions of atmosphere 34.13 34.90 0.51 0.477 5. Students’ social self-perceptions 20.06 19.45 0.31 0.579
aSubscale interpretations: 1, 0 - 12: very poor; 13 - 24: teaching is viewed negatively; 25 - 36: positive perception of teaching; 37 - 48: teaching highly thought of, 2, 0 - 11: abysmal; 12 - 22: in need of some retraining; 23 - 33: doing okay; 34 - 44: model course organizers, 3, 0 - 8: feelings of total failure; 9 - 16: many negative aspects; 17 - 24: more positive feelings; 25 - 32: confident, 4, 0 - 12: terrible environment; 13 - 24: Many issues need changed; 25 - 36: More positive perception; 37 - 48: overall good feeling of environment, 5, 0 - 7: miserable; 8 - 14: dissatisfied; 15 - 21: not too bad; 22 - 28: very good socially.
Recommendations to improve the outpatient experience ultimately resulted in four themes: orientation, environment, structured teaching, and inclusion. Orientation to the clinic was a theme identified prior to the change in management and did not change under the new management. Students commented that they needed a “better introduction to expectations on the first day.”
The overall learning environment was of concern for both study periods. This theme focused on the lack of patient care accessibility, either because the clinic had too many learners for the patient population, or the students were allowed to do a portion of the encounter then excluded from the rest. For example, “I would see a patient, present to the attending, and then not continue the encounter.”
New themes that arose after the change in management were structured teaching and inclusion. With the change in management, the clinic was busier, and the faculty had less time to teach or provide feedback. Students would like to “have better scheduled teaching sessions with faculty. They easily got pushed aside with faculty schedules.” Because of their schedules, students did not feel they were included in decisions about patient care. “It would be nice to be more involved by being asked how we think we should handle a patient's plan and treatment."