To date we have used self-reported measures only. We have found retrospective pre-post measures1 of confidence in various tasks to be the most convenient. The basic structure is a survey that includes descriptions of specific tasks or skills and a rating of ability before and after the educational event.
Here is an example:
Please respond to the following items according to your impressions of the impact of the Fellows’ Seminar on your ability in each area. For each item rate your ability before you joined the seminar (Before Seminar) and then how much you think the seminar improved your ability (After Seminar)
- My ability to understand the effect of caring for a child with cancer and blood disorders on medical professionals in ways that will enhance my ability to experience positive personal development over the course of my career.\
- My ability to understand my own and my colleagues’ personal responses to tragic medical situations:
- My ability to communicate with families during times of tragedy:
- My general ability to reflect on and analyze complex situations, formulate working hypotheses about the situations and take strategic action:
We have used similar measures for all of our applications (e.g., workshops, seminars, etc.). Although the outcomes consistently indicate that physicians think that the method improves their understanding and skill in managing challenging situations, the question remains whether or not this method can produce better outcomes on more objective measures. Over time, we hope to provide additional strategies for outcome measurement including those produced by colleagues who are applying the RP&L methods in other settings and with alternative designs.
Our seminar for fellows is mandatory for the first year and elective for the remaining two years of fellowship. We think our best outcome measure is the fact that 75% of our fellows continue to participate on a voluntary basis.
1. Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: A comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Evaluation and the Health Professions 1992; 5(3):350-366.