Foo, J

Purpose An essential yet oft-neglected step in cost evaluations is the selection of resources (ingredients) to include in cost estimates. The ingredients that most influence the cost of physician continuous professional development (CPD) are unknown, as are the relative costs of instructional modalities. This study’s purpose was to estimate the costs of cost ingredients and instructional modalities in physician CPD. Method The authors conducted a systematic review in April 2020, searching MEDLINE, Embase, PsycInfo, and the Cochrane Library for comparative cost evaluations of CPD for practicing physicians. Two reviewers, working independently, screened articles for inclusion and extracted information on costs (converted to 2021 U.S. dollars) for each intervention overall, each ingredient, and each modality. Results Of 3,338 eligible studies, 62 were included, enumerating costs for 86 discrete training interventions or instructional modalities. The most frequently reported ingredients were faculty time (25 of 86 interventions), materials (24), administrator/staff time (23), and travel (20). Ingredient costs varied widely, ranging from a per-physician median of $4 for postage (10 interventions) to $525 for learner time (13); equipment (9) and faculty time were also relatively expensive (median > $170). Among instructional modalities (≤ 11 interventions per modality), audit and feedback performed by physician learners, computer-based modules, computer-based virtual patients, in-person lectures, and experiences with real patients were relatively expensive (median > $1,000 per physician). Mailed paper materials, video clips, and audit and feedback performed by others were relatively inexpensive (median ≤ $62 per physician). Details regarding ingredient selection (10 of 62 studies), quantitation (10), and pricing (26) were reported infrequently. Conclusions Some ingredients, including time, are more important (i.e., contribute more to total costs) than others and should be prioritized in cost evaluations. Data on the relative costs of instructional modalities are insightful but limited. The methods and reporting of cost valuations merit improvement.

Abstract Purpose Both overuse and underuse of clinician referrals can compromise high-value health care. The authors sought to systematically identify and synthesize published research examining associations between physician continuous professional development (CPD) and referral patterns. Method The authors searched MEDLINE, Embase, PsycInfo, and the Cochrane Database on April 23, 2020, for comparative studies evaluating CPD for practicing physicians and reporting physician referral outcomes. Two reviewers, working independently, screened all articles for inclusion. Two reviewers reviewed all included articles to extract information, including data on participants, educational interventions, study design, and outcomes (referral rate, intended direction of change, appropriateness of referral). Quantitative results were pooled using meta-analysis. Results Of 3,338 articles screened, 31 were included. These studies enrolled at least 14,458 physicians and reported 381,165 referral events. Among studies comparing CPD with no intervention, 17 studies with intent to increase referrals had a pooled risk ratio of 1.91 (95% confidence interval: 1.50, 2.44; P < .001), and 7 studies with intent to decrease referrals had a pooled risk ratio of 0.68 (95% confidence interval: 0.55, 0.83; P < .001). Five studies did not indicate the intended direction of change. Subgroup analyses revealed similarly favorable effects for specific instructional approaches (including lectures, small groups, Internet-based instruction, and audit/feedback) and for activities of varying duration. Four studies reported head-to-head comparisons of alternate CPD approaches, revealing no clear superiority for any approach. Seven studies adjudicated the appropriateness of referral, and 9 studies counted referrals that were actually completed (versus merely requested). Conclusions Although between-study differences are large, CPD is associated with statistically significant changes in patient referral rates in the intended direction of impact. There are few head-to-head comparisons of alternate CPD interventions using referrals as outcomes.

Purpose Nearly all health care professionals engage in continuous professional development (CPD), yet little is known about the cost and cost-effectiveness of physician CPD. Clarification of key concepts, comprehensive identification of published work, and determination of research gaps would facilitate application of existing evidence and planning for future investigations. The authors sought to systematically map study themes, methods, and outcomes in peer-reviewed literature on the cost and value of physician CPD. Method The authors conducted a scoping review, systematically searching MEDLINE, Embase, PsycInfo, and Cochrane Library databases for comparative economic evaluations of CPD for practicing physicians through April 2020. Two reviewers, working independently, screened all articles for inclusion. Three reviewers iteratively reviewed all included articles to inductively identify key features including participants, educational interventions, study designs, cost ingredients, and cost analyses. Two reviewers then independently reexamined all included articles to code these features. Results Of 3,338 potentially eligible studies, 111 were included. Physician specialties included internal, family, or general medicine (80 studies [72%]), surgery (14 studies [13%]), and medicine subspecialties (7 studies [6%]). Topics most often addressed general medicine (45 studies [41%]) or appropriate drug use (37 studies [33%]). Eighty-seven studies (78%) compared CPD with no intervention. Sixty-three studies (57%) reported the cost of training, and 79 (71%) evaluated the economic impact (money saved/lost following CPD). Training cost ingredients (median 3 itemized per study) and economic impact ingredients (median 1 per study) were infrequently and incompletely identified, quantified, or priced. Twenty-seven studies (24%) reported cost–impact expressions such as cost-effectiveness ratio or net value. Nineteen studies (17%) reported sensitivity analyses. Conclusions Studies evaluating the costs and economic impact of physician CPD are few. Gaps exist in identification, quantification, pricing, and analysis of cost outcomes. The authors propose a comprehensive framework for appraising ingredients and a preliminary reference case for economic evaluations.

Context High‐quality research into education costs can inform better decision making. Improvements to cost research can be guided by information about the research questions, methods and reporting of studies evaluating costs in health professions education (HPE). Our objective was to appraise the overall state of the field and evaluate temporal trends in the methods and reporting quality of cost evaluations in HPE research. Methods We searched the MEDLINE , CINAHL (Cumulative Index to Nursing and Allied Health Literature), EMBASE , Business Source Complete and ERIC (Education Resources Information Centre) databases on 31 July 2017. To evaluate trends over time, we sampled research reports at 5‐year intervals (2001, 2006, 2011 and 2016). All original research studies in HPE that reported a cost outcome were included. The Medical Education Research Study Quality Instrument ( MERSQI ) and the BMJ economic checklist were used to appraise methodological and reporting quality, respectively. Trends in quality over time were analysed. Results A total of 78 studies were included, of which 16 were published in 2001, 15 in 2006, 20 in 2011 and 27 in 2016. The region most commonly represented was the USA ( n = 43). The profession most commonly referred to was that of the physician ( n = 46). The mean ± standard deviation ( SD ) MERSQI score was 10.9 ± 2.6 out of 18, with no significant change over time (p = 0.55). The mean ± SD BMJ score was 13.5 ± 7.1 out of 35, with no significant change over time (p = 0.39). A total of 49 (63%) studies stated a cost‐related research question, 23 (29%) stated the type of cost evaluation used, and 31 (40%) described the method of estimating resource quantities and unit costs. A total of 16 studies compared two or more interventions and reported both cost and learning outcomes. Conclusions The absolute number of cost evaluations in HPE is increasing. However, there are shortcomings in the quality of methodology and reporting, and these are not improving over time.