Citations

Clear

97 citation(s) found for "Cook, David J".

Reporting quality and risk of bias in randomised trials in health professions education
Horsley, Tanya , Galipeau, James , Zeiter, Jeanie, Hamstra, Stanley J. , Cook, David A. , Hamstra, Stanley J , Petkovic, Jennifer

Mayo Clinic

Medical Education | December 17, 2016 | Volume: 51 | Issue: 1 | Pages: 61-71

10.1111/medu.13130 | PMID 27981660 Get It

Context Complete reporting of research is essential to enable consumers to accurately appraise, interpret and apply findings. Quality appraisal checklists are giving way to tools that judge the risk for bias. Objectives We sought to determine the prevalence of these complementary aspects of research reports (completeness of reporting and perceived risk for bias) of randomised studies in health professions education. Methods We searched bibliographic databases for randomised studies of health professions education. We appraised two cohorts representing different time periods (2008–2010 and 2014, respectively) and worked in duplicate to apply the CONSORT guidelines and Cochrane Risk of Bias tool. We explored differences between time periods using independent‐samples t ‐tests or the chi‐squared test, as appropriate. Results We systematically identified 180 randomised studies (2008–2010, n = 150; 2014, n = 30). Frequencies of reporting of CONSORT elements within full‐text reports were highly variable and most elements were reported in fewer than 50% of studies. We found a statistically significant difference in the CONSORT reporting index (maximum score: 500) between the 2008–2010 (mean ± standard deviation [SD]: 242.7 ± 55.6) and 2014 (mean ± SD: 311.6 ± 53.2) cohorts (p < 0.001). High or unclear risk for bias was most common for allocation concealment (157, 87%) and blinding of participants (147, 82%), personnel (152, 84%) and outcome assessors (112, 62%). Most risk for bias elements were judged to be unclear (range: 51–84%). Risk for bias elements significantly improved over time for blinding of participants (p = 0.007), incomplete data (p < 0.001) and the presence of other sources of bias (p < 0.001). Conclusions Reports of randomised studies in health professions education frequently omit elements recommended by the CONSORT statement. Most reports were assessed as having a high or unclear risk for bias. Greater attention to how studies are reported at study outset and in manuscript preparation could improve levels of complete transparent reporting.
Using In-Training Evaluation Report (ITER) Qualitative Comments to Assess Medical Students and Residents: A Systematic Review
Sawatsky, Adam P. , Ginsburg, Shiphra , Cook, David A. , Hatala, Rose , Dudek, Nancy L.

Mayo Clinic

Academic Medicine | December 7, 2016 | Volume: 92 | Issue: 6 | Pages: 868-879

10.1097/acm.0000000000001506 | PMID 28557953 Get It

Purpose In-training evaluation reports (ITERs) constitute an integral component of medical student and postgraduate physician trainee (resident) assessment. ITER narrative comments have received less attention than the numeric scores. The authors sought both to determine what validity evidence informs the use of narrative comments from ITERs for assessing medical students and residents and to identify evidence gaps. Method Reviewers searched for relevant English-language studies in MEDLINE, EMBASE, Scopus, and ERIC (last search June 5, 2015), and in reference lists and author files. They included all original studies that evaluated ITERs for qualitative assessment of medical students and residents. Working in duplicate, they selected articles for inclusion, evaluated quality, and abstracted information on validity evidence using Kane’s framework (inferences of scoring, generalization, extrapolation, and implications). Results Of 777 potential articles, 22 met inclusion criteria. The scoring inference is supported by studies showing that rich narratives are possible, that changing the prompt can stimulate more robust narratives, and that comments vary by context. Generalization is supported by studies showing that narratives reach thematic saturation and that analysts make consistent judgments. Extrapolation is supported by favorable relationships between ITER narratives and numeric scores from ITERs and non-ITER performance measures, and by studies confirming that narratives reflect constructs deemed important in clinical work. Evidence supporting implications is scant. Conclusions The use of ITER narratives for trainee assessment is generally supported, except that evidence is lacking for implications and decisions. Future research should seek to confirm implicit assumptions and evaluate the impact of decisions.
The monitoring of activity at home after total hip arthroplasty
Toogood, Paul , Spear, Joshua A. , Cook, S. M., Taunton, Michael J. , Abdel, Matthew P. , Cook, David J. , Taunton, Michael J

Mayo Clinic

The Bone & Joint Journal | November 1, 2016 | Volume: 98-B | Issue: 11 | Pages: 1450-1454

10.1302/0301-620x.98b11.bjj-2016-0194.r1 | PMID 27803219 Get It

Aims Total hip arthroplasty (THA) has well known subjective benefits, but little is known objectively about the recovery of mobility in the early post-operative period. Patients and Methods A total of 33 patients aged > 60 years who underwent elective primary THA had their activity monitored for 30 days post-operatively using an at-home (Fitbit) ankle accelerometer. Their mean age was 70.7 years (61 to 86); 15 (45.5%) were female. The rate of compliance and the mean level of activity were determined. Comparisons between subgroups based on age, body mass index (BMI), surgical approach, and the destination of the patients when discharged were also performed. Results The mean compliance over the 30 days was 26.7 days (16 to 30; 89%) of use. The mean number of steps increased from 235 (5 to 1152) to 2563 (87 to 7280) (p < 0.001) between the first and the 30th post-operative day. Age < 70 years and an anterior surgical approach were significantly associated with higher levels of activity (1600 to 2400 (p = 0.016 to 0.031) and 1000 to 1800 (p = 0.017 to 0.037) more steps per day, respectively) between the second and the fourth week post-operatively. There was also a trend towards higher levels of activity in those who were discharged to their home rather than to a nursing facility (a mean of 1500 more steps per day, p = 0.02). BMI greater or less than 30 kg/m 2 was not predictive of activity (p = 0.45 to 0.98). Conclusion At-home remote mobility monitoring using existing commercially available technology is feasible in patients who have undergone THA. It showed a clear trend towards increased activity with the passage of time. Additionally, the remote device was able to detect differences in levels of activity clearly between patients in relation to variables of interest including age, BMI, surgical approach, and the destination of the patient at the time of discharge from hospital. Such monitoring may allow for the early identification and targeted intervention in patients who recover slowly. Cite this article: Bone Joint J 2016;98-B:1450–4.
Motivation to learn: an overview of contemporary theories
Cook, David A. , Artino, Anthony R.

Mayo Clinic

Medical Education | September 15, 2016 | Volume: 50 | Issue: 10 | Pages: 997-1014

10.1111/medu.13074 | PMID 27628718 Get It

Objective To succinctly summarise five contemporary theories about motivation to learn, articulate key intersections and distinctions among these theories, and identify important considerations for future research. Results Motivation has been defined as the process whereby goal‐directed activities are initiated and sustained. In expectancy‐value theory, motivation is a function of the expectation of success and perceived value. Attribution theory focuses on the causal attributions learners create to explain the results of an activity, and classifies these in terms of their locus, stability and controllability. Social‐ cognitive theory emphasises self‐efficacy as the primary driver of motivated action, and also identifies cues that influence future self‐efficacy and support self‐regulated learning. Goal orientation theory suggests that learners tend to engage in tasks with concerns about mastering the content (mastery goal, arising from a ‘growth’ mindset regarding intelligence and learning) or about doing better than others or avoiding failure (performance goals, arising from a ‘fixed’ mindset). Finally, self‐determination theory proposes that optimal performance results from actions motivated by intrinsic interests or by extrinsic values that have become integrated and internalised. Satisfying basic psychosocial needs of autonomy, competence and relatedness promotes such motivation. Looking across all five theories, we note recurrent themes of competence, value, attributions, and interactions between individuals and the learning context. Conclusions To avoid conceptual confusion, and perhaps more importantly to maximise the theory‐building potential of their work, researchers must be careful (and precise) in how they define, operationalise and measure different motivational constructs. We suggest that motivation research continue to build theory and extend it to health professions domains, identify key outcomes and outcome measures, and test practical educational applications of the principles thus derived.
Context-sensitive decision support (infobuttons) in electronic health records: a systematic review
Cook, David A. , Teixeira, Miguel T. , Heale, Bret S.E. , Fiol, Guilherme Del , Cimino, James J. , Del Fiol, Guilherme

Mayo Clinic

Journal of the American Medical Informatics Association | August 7, 2016 | Volume: 24 | Issue: 2 | Pages: 460-468

10.1093/jamia/ocw104 | PMID 27497794 Get It

Objective: Infobuttons appear as small icons adjacent to electronic health record (EHR) data (e.g., medications, diagnoses, or test results) that, when clicked, access online knowledge resources tailored to the patient, care setting, or task. Infobuttons are required for “Meaningful Use” certification of US EHRs. We sought to evaluate infobuttons’ impact on clinical practice and identify features associated with improved outcomes. Methods: We conducted a systematic review, searching MEDLINE, EMBASE, and other databases from inception to July 6, 2015. We included and cataloged all original research in any language describing implementation of infobuttons or other context-sensitive links. Studies evaluating clinical implementations with outcomes of usage or impact were reviewed in greater detail. Reviewers worked in duplicate to select articles, evaluate quality, and abstract information. Results: Of 599 potential articles, 77 described infobutton implementation. The 17 studies meriting detailed review, including 3 randomized trials, yielded the following findings. Infobutton usage frequency ranged from 0.3 to 7.4 uses per month per potential user. Usage appeared to be influenced by EHR task. Five studies found that infobuttons are used less often than non–context-sensitive links (proportionate usage 0.20–0.34). In 3 studies, users answered their clinical question in &gt; 69% of infobutton sessions. Seven studies evaluated alternative approaches to infobutton design and implementation. No studies isolated the impact of infobuttons on objectively measured patient outcomes. Conclusions: Weak evidence suggests that infobuttons can help providers answer clinical questions. Research on optimal infobutton design and implementation, and on the impact on patient outcomes and provider behaviors, is needed.
Twelve tips on writing abstracts and titles: How to get people to use and cite your work
Cook, David A. , Bordage, Georges

Mayo Clinic

Medical Teacher | June 2, 2016 | Volume: 38 | Issue: 11 | Pages: 1100-1104

10.1080/0142159x.2016.1181732 | PMID 27248314 Get It

When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments
Cook, David A. , Ginsburg, Shiphra , Kuper, Ayelet , Hatala, Rose

Mayo Clinic

Academic Medicine | April 19, 2016 | Volume: 91 | Issue: 10 | Pages: 1359-1369

10.1097/acm.0000000000001175 | PMID 27049538 Get It

Quantitative scores fail to capture all important features of learner performance. This awareness has led to increased use of qualitative data when assessing health professionals. Yet the use of qualitative assessments is hampered by incomplete understanding of their role in forming judgments, and lack of consensus in how to appraise the rigor of judgments therein derived. The authors articulate the role of qualitative assessment as part of a comprehensive program of assessment, and translate the concept of validity to apply to judgments arising from qualitative assessments. They first identify standards for rigor in qualitative research, and then use two contemporary assessment validity frameworks to reorganize these standards for application to qualitative assessment. Standards for rigor in qualitative research include responsiveness, reflexivity, purposive sampling, thick description, triangulation, transparency, and transferability. These standards can be reframed using Messick’s five sources of validity evidence (content, response process, internal structure, relationships with other variables, and consequences) and Kane’s four inferences in validation (scoring, generalization, extrapolation, and implications). Evidence can be collected and evaluated for each evidence source or inference. The authors illustrate this approach using published research on learning portfolios. The authors advocate a “methods-neutral” approach to assessment, in which a clearly stated purpose determines the nature of and approach to data collection and analysis. Increased use of qualitative assessments will necessitate more rigorous judgments of the defensibility (validity) of inferences and decisions. Evidence should be strategically sought to inform a coherent validity argument.
Consequences Validity Evidence: Evaluating the Impact of Educational Assessments
Cook, David A. , Lineberry, Matthew , Lineberry, Matt

Mayo Clinic

Academic Medicine | February 3, 2016 | Volume: 91 | Issue: 6 | Pages: 785-795

10.1097/acm.0000000000001114 | PMID 26839945 Get It

Because tests that do not alter management (i.e., influence decisions and actions) should not be performed, data on the consequences of assessment constitute a critical source of validity evidence. Consequences validity evidence is challenging for many educators to understand, perhaps because it has no counterpart in the older framework of content, criterion, and construct validity. The authors’ purpose is to explain consequences validity evidence and propose a framework for organizing its collection and interpretation. Both clinical and educational assessments can be viewed as interventions. The act of administering or taking a test, the interpretation of scores, and the ensuing decisions and actions influence those being assessed (e.g., patients or students) and other people and systems (e.g., physicians, teachers, hospitals, schools). Consequences validity evidence examines such impacts of assessments. Despite its importance, consequences evidence is reported infrequently in health professions education (range 5%–20% of studies in recent systematic reviews) and is typically limited in scope and rigor. Consequences validity evidence can derive from evaluations of the impact on examinees, educators, schools, or the end target of practice (e.g., patients or health care systems); and the downstream impact of classifications (e.g., different score cut points and labels). Impact can result from the uses of scores or from the assessment activity itself, and can be intended or unintended and beneficial or harmful. Both quantitative and qualitative research methods are useful. The type, quantity, and rigor of consequences evidence required will vary depending on the assessment and the claims for its use.
American Board of Internal Medicine and Maintenance of Certification Standards—Reply
Holmboe, Eric S. , Cook, David A.

Mayo Clinic

JAMA Internal Medicine | August 3, 2015 | Volume: 175 | Issue: 8 | Pages: 1425

10.1001/jamainternmed.2015.1115 | PMID 26236981 Get It

A contemporary approach to validity arguments: a practical guide to K ane's framework
Cook, David A. , Brydges, Ryan , Ginsburg, Shiphra , Hatala, Rose

Mayo Clinic

Medical Education | May 19, 2015 | Volume: 49 | Issue: 6 | Pages: 560-575

10.1111/medu.12678 | PMID 25989405 Get It

Context Assessment is central to medical education and the validation of assessments is vital to their use. Earlier validity frameworks suffer from a multiplicity of types of validity or failure to prioritise among sources of validity evidence. Kane's framework addresses both concerns by emphasising key inferences as the assessment progresses from a single observation to a final decision. Evidence evaluating these inferences is planned and presented as a validity argument. Objectives We aim to offer a practical introduction to the key concepts of Kane's framework that educators will find accessible and applicable to a wide range of assessment tools and activities. Results All assessments are ultimately intended to facilitate a defensible decision about the person being assessed. Validation is the process of collecting and interpreting evidence to support that decision. Rigorous validation involves articulating the claims and assumptions associated with the proposed decision (the interpretation/use argument), empirically testing these assumptions, and organising evidence into a coherent validity argument. Kane identifies four inferences in the validity argument: Scoring (translating an observation into one or more scores); Generalisation (using the score[s] as a reflection of performance in a test setting); Extrapolation (using the score[s] as a reflection of real‐world performance), and Implications (applying the score[s] to inform a decision or action). Evidence should be collected to support each of these inferences and should focus on the most questionable assumptions in the chain of inference. Key assumptions (and needed evidence) vary depending on the assessment's intended use or associated decision. Kane's framework applies to quantitative and qualitative assessments, and to individual tests and programmes of assessment. Conclusions Validation focuses on evaluating the key claims, assumptions and inferences that link assessment scores with their intended interpretations and uses. The Implications and associated decisions are the most important inferences in the validity argument.