Simovitch, Ryan W.

Background: When missing data are present in clinical outcomes studies, complete-case analysis (CCA) is often performed, whereby patients with missing data are excluded. While simple, CCA analysis may impart selection bias and reduce statistical power, leading to erroneous statistical results in some cases. However, there exist more rigorous statistical approaches, such as single and multiple imputation, which approximate the associations that would have been present in a full dataset and preserve the study’s power. The purpose of this study is to evaluate how statistical results differ when performed after CCA analysis versus imputation methods. Methods: This simulation study analyzed a sample dataset consisting of 2204 shoulders, with complete datapoints from a larger multicenter total shoulder arthroplasty database. From the sampled dataset of demographics, surgical characteristics, and clinical outcomes, we created five test datasets, ranging from 100 to 2000 shoulders, and simulated 10–50% missingness in the postoperative American Shoulder and Elbow Surgeons (ASES) score and range of motion in four planes in missing completely at random (MCAR), missing at random (MAR), and not missing at random (NMAR) patterns. Missingness in outcomes was remedied using CCA, three single imputation techniques, and two multiple imputation techniques. The imputation performance was evaluated relative to the native complete dataset using the root mean squared error (RMSE) and the mean absolute percentage error (MAPE). We also compared the mean and standard deviation (SD) of the postoperative ASES score and the results of multivariable linear and logistic regression to understand the effects of imputation on the study results. Results: The average overall RMSE and MAPE were similar for MCAR (22.6 and 27.2%) and MAR (19.2 and 17.7%) missingness patterns, but were substantially poorer for NMAR (37.5 and 79.2%); the sample size and the percentage of data missingness minimally affected RMSE and MAPE. Aggregated mean postoperative ASES scores were within 5% of the true value when missing data were remedied with CCA, and all candidate imputation methods for nearly all ranges of sample size and data missingness when data were MCAR or MAR, but not when data were NMAR. When data were MAR, CCA resulted in overestimates of the SD. When data were MCAR or MAR, the accuracy of the regression estimate (β or OR) and its corresponding 95% CI varied substantially based on the sample size and proportion of missing data for multivariable linear regression, but not logistic regression. When data were MAR, the width of the 95% CI was up to 300% larger when CCA was used, whereas most imputation methods maintained the width of the 95% CI within 50% of the true value. Single imputation with k-nearest neighbor (kNN) method and multiple imputation with predictive mean matching (MICE-PMM) best-reproduced point estimates and intervariable relationships resembling the native dataset. Availability of correlated outcome scores improved the RMSE, MAPE, accuracy of the mean postoperative ASES score, and multivariable linear regression model estimates. Conclusions: Complete-case analysis can introduce selection bias when data are MAR, and it results in loss of statistical power, resulting in loss of precision (i.e., expansion of the 95% CI) and predisposition to false-negative findings. Our data demonstrate that imputation can reliably reproduce missing clinical data and generate accurate population estimates that closely resemble results derived from native primary shoulder arthroplasty datasets (i.e., prior to simulated data missingness). Further study of the use of imputation in clinical database research is critical, as the use of CCA may lead to different conclusions in comparison to more rigorous imputation approaches.

Background A variety of clinically important benchmarks of success (CIBS) have been defined for total shoulder arthroplasty (TSA) to quantify success. However, it is unclear how the preoperative status of the patient influences their likelihood of achieving each CIBS. Questions/purposes (1) What proportion of patients achieve commonly used CIBS after TSA? (2) Is there a relationship between a patients’ preoperative function and their probability of achieving different CIBS? (3) Does there exist preoperative ranges for each outcome measure that are associated with greater achievement of CIBS? Methods We retrospectively queried a multicenter shoulder arthroplasty database for primary anatomic TSA (aTSA) and reverse TSA (rTSA). A total of 2041 aTSAs and 3205 rTSAs were included. The mean ± SD age at time of surgery was 66 ± 9 years for aTSAs and 72 ± 8 years for rTSAs. In all, 51% (1049 of 2041) of aTSAs and 61% (1955 of 3205) of rTSAs were in patients who were women. For aTSAs, osteoarthritis with an intact rotator cuff was the most common preoperative diagnosis (92% [1869 of 2041]). For rTSAs, the most common preoperative diagnoses included rotator cuff tear arthropathy (40% [1280 of 3205]), osteoarthritis with an intact rotator cuff (24% [780 of 3205]), osteoarthritis with a torn rotator cuff (20% [632 of 3205]), and massive rotator cuff tear (10% [309 of 3205]). Outcomes were evaluated at latest follow-up (aTSA 61 ± 36 months, rTSA 47 ± 26 months) and consisted of ROM (abduction, forward elevation, external and internal rotation) as well as the most commonly used outcome scores: the Simple Shoulder Test (SST), Constant score, American Shoulder and Elbow Surgeons (ASES) score, University of California, Los Angeles (UCLA) score, Shoulder Pain and Disability Index (SPADI), and the Shoulder Arthroplasty Smart (SAS) score. The CIBS that we evaluated included the minimum clinically important difference (MCID), substantial clinical benefit (SCB), patient acceptable symptom state (PASS), minimum clinically important percentage of maximal possible improvement (MCI-%MPI), and substantial clinically important percentage of maximal possible improvement (SCI-%MPI). Prosthesis-specific anchor-based CIBS were adopted from prior publications on patients from this database. Multivariable regression was performed to identify the relationship between preoperative outcome measures and achievement of CIBS. Additionally, receiver operating characteristic (ROC) curve analyses were performed to determine whether thresholds in preoperative outcome measures were associated with achieving CIBS. Results For all ROM measures and outcome scores, poorer preoperative ROM was associated with greater odds of achieving the MCID and SCB but lower odds of achieving the PASS. For the SST and the two scores without ceiling effects (the Constant and SAS scores), poorer preoperative outcome scores were associated with greater odds of achieving the MCI-%MPI and SCI-%MPI, but no association was demonstrated for the ASES, SPADI, and UCLA scores. Graphical analysis demonstrated that patients with greater preoperative ROM and outcome scores had a lower probability of achieving the MCID and SCB but a higher probability of achieving the PASS. For outcome scores with known ceiling effects, patients with more favorable preoperative outcome scores were more likely to achieve the MCI-%MPI and SCI-%MPI than the MCID and SCB, respectively. For outcome scores without ceiling effects, patients undergoing aTSA were more likely to achieve the MCID, MCI-%MPI, and the SCI-%MPI than the SCB, but no clear trend was identified for patients undergoing rTSA. On ROC curve analysis, identified thresholds were reasonably accurate (area under the curve > 0.7) for achievement of measures of absolute improvement (the MCID and SCB), but not for absolute postoperative status (the PASS) or relative improvement (the MCI-%MPI and SCI-%MPI). Conclusion While most patients reported being “much better” (aTSA 75%, rTSA 76%) or “better” (aTSA 15%, rTSA 18%) compared with before surgery, 10% of aTSAs and 6% of rTSAs were either “unchanged” or “worse.” Patients’ likelihood of achieving CIBS depends in part on their baseline function and whether success is defined as absolute improvement, absolute postoperative status, or relative improvement. Patients with more favorable preoperative status are more likely to achieve CIBS that evaluate relative improvement (the MCI-%MPI and SCI-%MPI) than absolute improvement (the MCID and SCB). Future studies are needed to determine which individual CIBS or combinations thereof most accurately represent clinically relevant benefit. Level of Evidence Level III, therapeutic study.